." ~ \ -T-'V ( «« «BVV'«' "• mmm^mmB^mmmmm^^w-*-'j *~J '

ARI Research Note 86-87

>

.v

.-

CNJ

&

CM

<

i

D

<

AN ANALYSIS OF COSTS OF COMPUTER BASED

TRAINING HARDWARE AND COURSEWARE DEVELOPMENT

FOR THE MODEL TRAINING PROGRAM FOR

RESERVE COMPONENT UNITS

John Begg and Kenneth Bernstein

Scientific Systems, Inc.

for

ARI Field Unit at Fort Knox, Kentucky

Donald F. Haggard, Chief

TRAINING RESEARCH LABORATORY

Jack H. Miller, Director

o

o

es

jELECTE]

DT 0 61988

U. S. Army

Research Institute for the Behavioral and Social Sciences

August 1986

Approvefl for pubiic r«iti$e: dittr bution unlimitK).
." -.— - "- .".-."

U. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIA1 SCIENCES

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

FIX;AR M. JOHNSON

Technical Dui-clor

WM. DARRYL HENDERSON

COL, IN

Commanding

Peer Reviews by:

Scott E. Graham

Donald R. Lampton

Accession For

DIIC '

I

ion .

^

By

Distribution/

Aval.: " )dei

/or

Dlst

m

Thii rtport, *i lubmittid by th« contractor, hd batn cltartd (c raleaic to Dcfenie Tichnical Information Center

IDTICI to comply with regulatory requirementi. It hat been given no primary dittribution other than to DTIC

and will be available only through DTIC or other reference icrvicei tuch as the National Technical Information

Service (NTIS). The vi:wi. cpinicnt, and/or finding« contained in thii report are thote of the authorltl and

ihould not be comtrueu at an official üepaitmem of the Army poiition, policy, or deciiion, unlen 10 detignated

by other official documentation.

i..:

- - . ••'-..- .' _'•

--.• .-••"./.-• '.
p«.1»»' i . 11. I . |j l Kl i .niJ»Jiii»g»|H.iinnp<fB>ij<.i..nM>.nijt

ij<.iiy>ij|MlMl-l>ll-'..

UNCLASSIFIED

SECURITY CLASSIFICATION OF THIS PAGE fWhtn Dmlm Enltrtd)

REPORT DOCUMENTATION PAGE

1 REPORT NUMBER

ARI Research Note 86-37

2. GOVT ACCESSION NO

AD-A L22

4 TITLE (trtd Submit)

An Analysis of Costs of Computer Based Training

Hardware and Courseware Development for the Model

Training Program for Reserve Component Units

7. AUTMORf«;

John Begg and Kenneth Bernstein

9 PERFORMING ORGANIZATION NAME AND ADDRESS

Scientific Systems, Inc.

35 Cambridgepark Drive

Cambridge. MA 02140

II. CONTROLLING OFFICE NAME AND ADDRESS

U.S. Army Research Institute for the Behavioral

and Social Sciences, 5001 Eisenhower Avenue,

Alexandria. VA 22333-5600

U MONITORING AGENCY NAME A AODRESSC/' dllttrmrl from Canlrolllnt Olllct)

READ INSTRUCTIONS

BEFORE COMPLETING FORM

3 RECIPIENT'S CATALOG NUMBER

J7^

5 TYPE OF REPORT a PERIOD COVERED

Interim Report

October 83 - December 85

6 PERFORMING ORG. REPORT NUMBER

S CONTRACT OR GRANT NUMBERf»)

MDA 903-83-C-Ü483

10 PROGRAM ELEMENT, PROJECT. TASK

AREA a WORK UNIT NUMBERS

2Q263743A794

12. REPORT DATE

Augusi 1986

'3 NUMBER O" PAGES

62

IS. SECURITY CLASS, (ol Ihli fport)

Unclassified

15« DECLASSIFICATION DOWNGRADING

SCHEDULE . .

I« DISTRIBUTION STATEMENT (ol thlm Rmporl)

Approved for public release; distribution unlimited.

17. DISTRIBUTION STATEMENT (ol '.he mbtlrmzl tnfrtd In Block 30, II dllltrml from Rmporl)

1» SUPPLEMENTARY NOTES

Donald F. Haggard, contracting officer's representative

19. KEY WORDS (Confinu« on r«v«ra* midm II nacoaavy and identity by block numbmr)

Computer Based Training

Maintenance Training

Reserve Components

Tanks

Ml Tanks

Technical Training

Computer Cost

Tracked Vehicles

Training

Armor

Army Training

V ABSTRACT rCooftoua an rarara« a/«

This report analyses

computer-based training

Reserve Component Units

system configurations w

numbers of workstations

Per Student Hour In A !

:

is identified. Part Two

(to H rmcvmrnmrj mail Idtuitr by block numbmr)

the costs of producing and delivering MicroTICCIT

system courseware for the Model Training Proaram for

(MTP-RC). Part One of the report analyses the costs of

ith various host computers, amounts of storage, and

. Costs are reported in the format: Total System Cost

ive-Year Lifecycle, and the most economical configuratior

reports the cost of developing five types of lessons,

(over)

00 1 JSM'T» 1473 EDITION OF I NOV SS IS OBSOLETE UNCLASSIFIED

1 SECURITY CLASSIFICATION OF THIS P «GE fBfwn Dmlm Bnlmrmd)

.^..^^£^..: ;; ...:

..: ••■■.•■■. ^• >'.-. . . ■■•. .•.' -■■ . .: v.-.--..-...
II 11* III I ^T^^T^^^I^I > ^H.II*fl*.ll».IH«l..

UNCLASSIFIED

SECURITY CLASSIFICATION OF THIS PAGE »T>«fi Da(a Ent.r««;

ARI RESEARCH NOTE 86-87

20. Abstract (continued)

each based on MTP-RC instructional templates. Costs are reported in four labor

categories for each type of lesson. Variations in cost between types ere discussed and three variables which can affect cost are identified.

' ^

UNCLASSIFIED

n SECURITY CLASSIFICATION OF THIS PAGflfhon Dalm f.nlered)

" .-'•..•'•.•"• -' '- '• - • -' '-"- . ."- - -',, •'."-.. • >Mrik>'>. • ." .V ,■■ ."> jv - »- .vi ■■ -■■ '
^'^^^^^^^^•W^^^F^^^^^w^^»wp»fw

'

w

'

,'WT*?»'"*'^"

AN ANALYSIS OF COSTS OF COMPUTER BASED TRAINING HARDWARE AND SOFTWARE

DEVELOPMENT FOR THE MODEL TRAINING PROGRAM FOR RESERVE COMPONENT UNITS

EXECUTIVE SUMMARY

Requirement:

The Model Training Program for Che Reserve Component Units (MTP-RC) Is

a research project conceived by the US Army Research Institute (ARI) and

the Training Doctrine Command's Training Technology Agency to develop and

validate an effective computer-based Instructional response to the training

challenges created by the introduction of new equipment into the Army

inventory. The purpose of this report is to present a cost analysis of

MTP-RC hardware delivery system configurations and to present a cost

analysis of MTP-RC courseware development.

Procedure:

Approximately 200 hours of Interactive courseware were developed for

MTP-RC to train four Military Occupation Specialities (MOSs) who maintain

the Ml Abrams tank. A two part cost analysis was performed. Part One

contains a cost analysis of different configurations of the delivery

system; Part Two contains an analysis and comparison of the cost of

producing the courseware. The courseware was authored and will be

delivered using the MicroTICCIT computer-based training system produced by

the Hazeltlne Corporation. Different configurations of the host computer

with varying numbers of workstations and disk storage were analyzed.

Hardware costs are reported in system cost per student hour. The second

part of the report presents an analysis of the cost of producing five

different types of lessons, each based on instructional templates. A

template is composed of a carefully prescribed instructional strategy, a

predetermined set of screen display characteristics, and a structured

system for student interactions. Costs are reported in the approximate

number of hours required by each of four labor categories to produce

lessons based on each of the five lesson templates. Variations in

production cost are discussed and three variables affecting development

time are identified.

Findings:

Following the analysis of the hardware systems, the authors report

that a MicroTICCIT system with the Desktop host computer is the model best

suited for the typical RC training site, if MTP-RC courseware is to be

in

-"•.•-. -- .• • .■■. -V-^>,<VV/VV'V^/-JS . '- •' -"- -'- " .-•" .-H-V-VvV.~ - -.. ■■ - .--v- '•"•• 1-- '' •" ' ■■ -' •' •- ••'--
,"%.,

widely fielded In the future. The Desktop models with 1A2 megabytes of

disk storage compare favorably In cost per student hour to models that have

less disk capacity. The added disk capacity allows greater training

schedule flexibility. Following their analy is of courseware production,

the authors report that Troubleshooting Simulation lessons required the

most labor to produce, followed by Maintenance Simulations, Name, Locate

Function, and Input, Process, Output lessons and Troubleshooting

Introductions In descending order. Production time per lesson Increases as

the complexity of the Interaction Increases, as the complexity of the

content Increases, and as the amount of content Increases.

Utilization of Findings:

Application of the findings reported In this paper provides guidance

for evaluating delivery systems for fielding MTP-RC courseware. The

Information reported on courseware development costs will assist In

estimating the cost of producing courseware of a similar technical nature

that uses the same lesson templates.

• !.

iv

'. -". V. A

:v;-.vvv.:^v: .•:-'->-:-. :.-;:• ,

K - * - m t.
(«VW«"- w-V-"l^" V-« VW'.-* .-» '-WTWT-WTWl-WlTTVVWT.- - ---' ^.- ^.T^-^IW-W"-»''-!*- ^ ^*

AN ANALYSIS OF COSTS OF COMPUTER BASED TRAINING HARDWARE AND COURSEWARE

DEVELOPMENT FOR THE MODEL TRAINING PROGRAM FOR RESERVE COMPONENT UNITS

CONTENTS

Page

OVERVIEW OF THE REPORT 1

PART ONE: HARDWARE COST ANALYSIS 3

Components of the Delivery System 4

MicroTICCIT System Configurations 5

Summary and Recommendat Ions 15

PART ONE: COURSWARE DEVELOPMENT COSTS 17

Overview 17

Courseware Templates 18

Labor Categories 19

The Development Process 21

Findings: Courseware Production Costs 24

Utilization of the Findings 32

Summary 33

REFERENCES 35

APPENDIX A HAZELTINE MICROTICCIT SYSTEMS AND CATALOG PRICES A-l

B FORMULAS FOR COMPUTING SYSTEM COST PER STUDENT-HOUR..B-l

C STRUCTURE OF MTP-RC COURSEWARE C-l

D WORK SUMMARY D-l

LIST OF TABLES

Table 1 Comparison of Three MicroTICCIT Systems 5

2 Cost Per Student Hour of MicroTICCIT System 7

3 Number of Workstations and Cost Per Student Hour 11

4 Disk Storage and System Cost Per Student Hour 13

5 Two Fielding Scenarios 14

6 Labor Categories and Major Production Tasks 21

7 Courseware Production: Average Labor Figures in Hours..25

m

•*• •.«
f " '■■'• V"»" W^V» V

^ V'^i~"\.-W\_»V^ .- V- r .-

M

,".'- CONTENTS -continuedLIST OF FIGURES

Page

Figure 1. MTP-PC trial fielding sites 3

2. Cost per student hour-lOO Megabyte S>stem II 8

3. Cost per student houi—142 Megabyte Desktop 9

C-l. rlow of Instruction C-2

V ".

vi

VV-

^-.i/v:-.' :

'. '.'. ■■.

. -. • .-. .-•. .
"If ""^T - Tf—

AN ANALYSIS OF COSTS OF COMPUTER BASED TRAINING HARDWARE AND COURSEWARE

DEVELOPMENT FOR THE MODEL TRAINING PROGRAM FOR RESERVE COMPONENT UNITS

OVERVIEW OF THE REPORT

Estimating the cost of designing, producing, and delivering

instructional materials is a difficult task, and the use of sophisticated

computer based training (CBT) systems iucrpases the difficulty. CBT

technology brings new factors into the cost of instructional programs, such

as the cost of the computer system for development and delivery of the

training and the hours required for the programmers to develop Che complex

code required. In addition, the tasks of instructional designers, graphic

artists, and subject matter experts may be very different in CBT programs

than they are in programs in other media, further increasing the difficulty

of estimating costs. Analysis of the cost experience of current and past

CBT programs can aid the process of estimating the cost of future programs.

\r>.s

. .-

This report presents an analysis of the costs of the Model Training

Program for Reserve Component units (MTP-RC), a research and development

program sponsored by the Army Research Institute (ARI) and the Training and

Doctrine Command's Training Technology Agency to evaluate computer based

training as a means of delivering troubleshooting and maintenance training.

The cost of hardware delivery systems and configurations are analyzed and a

recommendation made for a configuration to delivar the MTP-RC training. The

courseware development costs are also analyzed to determine the relative

costs of different types of labor and different types of lessons. This

analysis will be useful to planners of future sophisticated technical

training courseware. The term "courseware," as it is used in this report,

refers to computer programming that integrates text, graphics, and video in

the presentation of interactive instruction to soldiers.

This report is composed of two parts. The first part is a cost

analysis of the hardware system used to deliver MTP-RC courseware. This

inalysls is the basis of recommendations of the most cost-effective hardware

configurations for fielding of delivery systems. It includes a discussion

of:

(a) the hardware components that compose the delivery system,

(b) delivery system costs, including calculation of the system

cost per student hour,

(c) the effect of system configuration on system utilization, and

(d) the effect of disk storage capacity on training schedules.

The second part of the report contains an analysis of the the lahui costs

for five different lesson types developed for MTP-RC. About three quarters

of the 123 lessons produced for the program are included in these five

,'V-'.. . Xtiti . . . . .
•-"••-. »-

• \ » "T" 7 1_ i i^ IJ •1 " »' '. " '. ' " »i:»' TV»'.

1^'1

»'.

1! '. » »'.T'^T'^I^'I^."», ' HJ»,«»-^'. ^,-»- ^.' l/»-^

types. This report examines the labor expended by the prime contractor for

lessons during a period when all personnel were trained and the production

process was established. Approximately twenty-five large lessons are

Included. The remainder of the lessons do not fail neatly Into types

because they have varied structures and designs. The courseware and

production steps used In MTP-RC are described, and variation^ In the five

lesson tvpes bv labor categories are discussed. The discussion Includes the

effect of type of content and complexity of Instruction on production costs.

Another factor In production costs, the development system Itself, Is not

addressed In this report. All production was done on the same system,

providing no basis for comparison of systems. This analysis of the cost

experience of the Model Training Program provides useful guidance for

estimating the cost of producing courseware of similar technical nature that

uses the same lesson templates.

The Model Training Program for Reserve Component Units Is developing

and evaluating state of the art, computer-based troubleshooting and

maintenance Instruction. The training was designed specifically for Reserve

Component units that lack equipment or experienced Instructors for

traditional training on new equipment. Approximately 200 hours of

compute!—based training were developed for this program. The sophisticated

design of the courseware combines CBT with state-of-the art Interactive

videodisc.

The MTP-RC Instruction Is designed to train and sustain skill level I

and 2 troubleshooting and maintenance skills for four Military Occupation

Specialties (MOSs) related to the Ml Abrams tank. The target audience

Includes two organizational MOSs (45E and 63E) and two Direct

Support/General Support MOSs (45K and 63H) In RC units which have CAPSTONE

assignments as Ml support units. Computer-based training and simulation are

particularly appropriate for these unts, as they typically do not have Ml

tanks available for training at their home sites.

The MTP-RC courseware Is organized Into five courses: four

MOS-speclflc courses and one course of common skills such as safety and

test, measurement, and diagnostic equipment. The courses train tasks

selected by the US Army Research Institute from the respective MOS critical

task lists. Each course comprises approximately one third of a complete

skill level course for the MOS. The skills and knowledge required to

perform the tasks were Identified through a front-end training analysis.

Course syllabi were then developed and Instructional strategies appropriate

to the training objectives were selected prior to production of the

courseware.

The courseware development process used new design strategies and state

of the art courseware authoring and delivery techniques. The instruction Is

presented with a standard student Interface and lesson designs to reduce the

amount of time spent learning how to deal with the training system Itself.

The courseware presents realistic video and graphic Images and requires

extensive soldier Interactions for gaining knowledge and for practicing

skills on troubleshooting and maintenance simulations.

.-. .•.• •'•" • -i- •-•-••• • •-.. ■■■•-,-„•-- - . . • . , • . •. . - A •.. ^ • • - • J - '.• . -^»i i 'V w -
« 1 I 11 « 1 I

1

'T » .»—> >,'«. • ji j » , u"'r'^'V"1 mvm umim^my mi*,.*, m^mV9\*':V ^ ' I

1

'1 I »

Plans for the one-year trial fielding and evaluation of the MTP-RC

courseware and hardware call for Installation of systems at three Reserve

Component units In the second and third quarters of fiscal year 1986. Each

unit has soldiers assigned to MOSs with Ml maintenance responslhi 111les, as

listed below.

Unit Location MOSs and Maintenance Level

195th Company

2198th Company

1st Detachment,

2 52nd Armor

Westminster MD

Dagsboro DE

Red Springs, NC

45K, b3H Direct Support/General Support

45K, 63H LIrect Support/General Support

45E, 63E, Organizational and

45K, b3H Direct Support/General Support

Figure 1. MTP-RC trial fielding sites.

The 1st Detachment at Red Springs Is primarily an organizational maintenance

unit, but it also has some Direct Support/ General Support (DS/GS) personnel

assigned to It. This site Is the only one of the three to have Ml tanks

fielded to date.

Soldiers at each site will receive hands-on and written pretests on Ml

skills and knowledge prior to beginning their training on computer system.

The soldiers will receive their training during weekend drills and annual

training. At the end of the fielding. In third quarter fiscal 1987,

soldiers will be posttested on both hands-on and written skills. In

addition to evaluating the effectiveness of the CBT program by analyzing

soldier performance scores, data will be collected on the suitability of the

computei—based training system to the Reserve Component training

environment. Some factors to be considered are time required for the system

operator, ease of scheduling soldiers, and overall utilization of the

system.

PART ONE: HARDWARE COST ANALYSIS

Different configuration of the Hazel tine MlcroTICCIT system provide

various advantages and varying degrees of cost effectiveness. Several

configurations possible for the MTP-RC delivery are analyzed In this section

of the report to determine the total system hardware cost and system cost

per student hour. A recommendation Is made for the most cost effective

system. In addition, the components of the system are considered for their

effect on courseware delivery. The method for calculating system cost Is

discussed In some detail to provide a model for such calculations for other

programs.

~:

tttäi&^^
.^.i^ t i» . . j i. . j pii-n- tr in».

1

•'m-mjimjimj^^ IHIJ^I^IU^P IJ»I.H u ■■. »'. "i. « ! »'« » I»»-»»»IHII.»II^H.»I Ji i » i . ; FJ J

1

;

Components t)f the Del'.very System

MTP-RC courseware was designed for d

MlcroTICCIT was selected as the delivery

most sophisticated systems available at t

contract In 1983. The Hazel tine Corporat

presentations of video, graphics, and the

the system Includes courseware management

records of soldiers' performance. Three

this report: the Desktop, the System II,

major components of each system are works

disk storage device. Each system can be

iipon the number of workstations and disk

The Workstation,

ellvery on a MIcroTICCIT system,

system because It was one of the

he time of the award of the MTP-RC

Ion system allows Screen

combination of both. In addition,

capabilities required to maintain

MIcroTICCIT systems are analyzed In

and the System III. The three

tatlons, a host computer, and a

configured differently, depending

storage connected to the system.

The workstation Is the equipment that displays the

courseware to the student and accepts student fnput. The workstation Is

connected by coaxial cables to the host computer, allowing the student to

Interact with the courseware stored on the disk storage device. The

workstation used for MTP-RC delivery Is composed of:

o an IBM VC microcomputer with a graphics overlay boa-d designed

by Hazeltlne,

o a Sony videodisc player,

o a Sony color monitor,

o a keyboard with a keypad and special function keys,

o a light pen, and

o a high-speed communications link to the host computer.

The IBM microcomputer contains the communications Interface card and a

display controller. The display controller Is a circuit board that permits

videodisc Images, text, and graphics to be displayed singly or In

combination on the color monitor. Every MIcroiICCIT system must have one

workstation equipped as a network controller to manage the flow of

Information between the workstations and the host computer through a special

network controller card. Additional details about Hazeltlne MIcroTICCIT

systems are contained In Appendix A.

The Disk Storage Dov'.ce. Disk storag? Is ; magnetic medium upon which

courseware and student record data are written. Data written to the disk

are accessible by the workstations through the central processing unit (CPU)

In the host computer. Disk storage capacity Is measured In millions of

bytes or megabytes (Mbytes). The disk storage capacity of each MIcroTICCIT

system depends on Its configuration. The amount of disk storage capacity

determines how much courseware Is accessible at one time. In addition to

the disks, a tape drive Is used for loading or unloading courseware and

student record data to the disk. The MTP-RC systems are all equipped with a

15 Mbyte cartridge tape drive for loading and unloading programs.

r* ** ".> v

.• .

frwV.V.V.-". /.v.v-A.'--, A <..-•■■ ^^•^^^^^Ä:ÄÄ?
-',•*

The Host Computer. Each of the three Hazeltlne systems uses a

different central processing unit (CPU). The processing capacity determines

the number of workstations that the host computer can support. The CPU Is

the part of the host computer that takes Information stored In Its raw form

from the disk storage device and processes It for display at the

workstations. It also takes student responses from the workstatlo-is and

stores them as student records on the disk.

MlcroTICCIT System Configurations

The capacities of each of three MlcroTICCIT systems are listed In Table

1. Detailed specifications of each system are contained In Appendix A.

Desktop systems can support as many as ten workstations. The other two

systems can be used for both delivery and authoring. The System II supports

up to forty workstations and System III up to sixty-four workstations. The

number of workstations the host computer ^an support Is an Important

constraint when an Institution Is considering which MlcroTICCIT system to

field. The number of workstations affects utilization rates and therefore

system cost per student hour. These costs are discussed beloj.

Table 1

Comparison of Three MlcroTICCIT Systems

System Maximum dl>k Tape

drive

Megabytes

Maximum ilsk

workstatIons storage

10 71 or 142

40 50 or 100

b4 147 or 295

Desktop 10 71 or 142 15.4 Mbyte cartridge tape

System II 40 50 or 100 15.4 Mbyte cartridge tape

System III 64 147 or 295 15.4 Mbyte cartridge tape

or

25 Mbyte streaming tape

Another consideration In choice of system Is the ability to write or

revise courseware on the system. Systems II and III permit authoring at any

workstation while the Desktop system does not. This authoring capability Is

not considered necessary for the MTP-RC field sites. In fact, even If the

systems fielded do support authoring, no personnel at the sites will be

authorized to make changes In the courseware. Personnel at the sites lack

the Ml technical expertise required to change content and the programming

expertise required to change the code In the MTP-RC courseware. In

addition, the special tape sets developed to deliver the courseware maximize

the courseware available on-line by having only the code needed to run the

lessons. They do not contain the "source" code required for authoring and
revlslr? the on-line lessons. A limited number of emergency changes could

be made by field representative at the sites, but most changes would have to

be made in a central locations and shipped to the sites.

Dellverv Svstem Costs

Method of Calculating System Cost Per Student Hour. Orlansky and

String (1979) recommend that dellverv system costs be ci-culated using the

"system cost per student hour methoa" because It accounts for the life—cycle

costing variables of maintenance, amortization, and system use. The cost

per student hour method was used for this report because this method

accounts for most of the variables of Interest In military training,

The steps In calculating system cost per student hour are:

1. Total system hardware costs. Including iialntenance,

over the life of the system.

2. Annual .,ost per workstation.

3. Annual workstation utilization rate.

Table 2 lists the costs of three different MIcroTlcclt configurations

and the Intermediary steps In the calculation of system cost per student

hour and hardware costs. The hardware and maintenance prices used in this

report are based on Hazeltlne's Decerabti , 1985, ctalog prices and do not

Include training and support. Tlw columns bre£.V. down the system costs Into

main steps In the computation of system cost per student hour. Each step

will be described below. Appendix B lists all the formulas for computing

system cost per student hour.

Column 4 In Table 2 shows the total system cost over 5 years. It Is

the Initial acquisition price of the central processing unit with a

designated amount of storage as shown In column 3, plus the acquisition

price of the number of workstations given In column I, plus maintenance

costs. The workstation cost Including the required videodisc player and

associated equipment Is approximately $10,ÜÜO. The cost of each system,

therefore, depends on the host computer, the amount of disk storage, and the

number of workstations. In the next section of the report, typical Reserve

Component training situations will be presented to Illustrate how the system

cost Is affected by these variables.

Orlansky and String (1979) compared system cost per student hour using

a five year amortization period. The five year maintenance cost of the host

computer depends on the specific system. The cost ranges from $4,100 per

year for a Desktop system with 71 Mbytes to $12,000 per year for a System

III system with 294 Mbytes. The workstation maintenance cost over five

years Is based on $1,000 per year per workstation and Includes maintenance

on all workstation components Including the videodisc player.
Table 2

Cost per Student Hour Of MicroTICCl ' Systems

HOST ^EGRBYTES (5 YtTR) (5 YEACS) ANNUAL SYSTEM

NUMflEP OF COMPUTEP DISK TOTAL SYSTEM COST COST PER

WORKSTATIONS MODEL STOPflGE SYSTEM CIST PEP WORKSTATION WORKSTATION

3 DESKTOP 71 110,500 36,933 7,367

DESKTOP M2 131,500 43,833 8,767

SYSTEM II BO 130,000 43,333 8,667

SYSTEM II ICO 155,000 51,667 10,333

5 DESKTOP 71 140,500 29,100 5,620

DESKTOP 142 161,500 32.300 6,460

SYSTEM II 50 160,000 32,000 6,400

SYSTEM 11 100 185,000 37,000 7,400

10 DESKTOP 71 215,500 21,550 4,310

DESKTOP i- 3 236,500 23,650 4,730

SYSTEM II 50 235,000 23,500 4,700

SYSTEM II 100 260,00^ 26,000 5,200

SYSTEM III 147 292,500 29,250 5,850

SYSTEM III 2

C

>4 335,000 33,500 6,700

COST PEB COST PER COST PER .

STUDENT HOUP STUDENT HOUR STUDENT HOIP SI p

1000 HRS/YEAP 500 HRS/YEAR 367 HRS/YEfR

7.37 14.73 20.07

8.77 17.53 23.*

8.67 17.33 23.61

10.33 20.67 28.16

5.62 11.24 15.31

6.46 12.92 17.60

6.40 12.90 17.44 2^

7.40 14.90 20.16 33.b

4.31 8.62 11.74 19.59

4.73 9.46 12.99 21.50

4.70 9.40 12.81 21.36

5.20 10.40 14.17 23.64

5.95 11.70 15.94 26.59

6.70 13.40 18.26 30.45

.•••;• •
System cost per student hour Is a function of the annual system cost

per workstation and the annual workstation utilization rate. Annual

workstation utilization rate Is computed by dividing the total number of

hours per year the workstations are used by the number of workstations. For

example, If 100 soldiers each use a workstation 10 hours a year, the

workstations are In use 1,000 hours a year. If there are five workstations,

then the workstation utilization rate Is 200 hours per year. System cost

per student hour (Columns 7 through 10) Is calculated by dividing the annual

system cost per workstation by the annual workstation utilization rate.

Workstation Utilization Rates. System cost per student hour Is

Inversely proportional to workstation utilization rates, to system life

span, and to the number of soldiers a workstation can accommodate. When

system use is constant (e.g., 1,100 hours per year) and the number of

workstations Is Increased (e.g., from three to five) the annual workstation

utilization rate decreases, thus Increasing the system cost per student

hour.

Figure 2 illustrates the relationship between system cost per student

hour and workstation utilization rate for a System II with 100 Mbytes as an

example. System cost per student hour was computed for the system

configured with 3, 5, and 10 workstations at three different workstation

utilization rates. For a given number of workstations, as workstation

utilization increases, the cost per student hour decreases. Also, If the

workstation utilization rate remains constant as the number of workstations

increases, then cost per student hour decreases. The same relationship can

be seen in the comparison of various configurations of the Desktop system

with 142 megabytes of disk storage (Figure 3).

Analysis of MTP-RC fielding situations with typical RC training

schedules and student loads will Illustrate the Influence that different

system configurations have on system cost per student hour. It also will

Identify factors other than capital expenditure that should be considered

when selecting a system confIg.jratIon.

It Is possible to calculate system cost per student hour for any system

configuration given the data contained In Table 2, the formulas In Appendix

B, and the Hazel tine catalog prices for maintenance In Appendix A. The

situations presented, although not exhaustive, are Illustrative of typical

system configurations. The purpose of these examples Is to Illustrate the

effects on cost of the number of workstations and the amount of disk

storage.

Number of Workstations. System cost per student hour Increases with

the number of workstations If the total hours of system use per year Is

constant. Consequently, as the number of terminals Increases, the system

cost per student hour Increases. This relationship between system cost per

student hour and number of workstations can be Illustrated using a situation

similar to that of the 1st Detachment In Red Springs, NC.

.-.-•.•

:■■. : / " ' • ':-:":v:>•• ; ": : " ' • '. - -->::■■■
CO

tr

<

_i

o

Q

30-

25-

20-

15-

10-

28.16

Workstation

Utilization Rate

• =367 HOURS

=500 HOURS

=1000 HOURS

14.17

10.40

5.20

1

i ' 4 ' ' '

NUMBER OF WORKSTATIONS

r

~^o

Figure 2. Cost per student hour-100 Megabyte System II.

9

,,. -'..V-v,v. •>,'r..-V. v. ■■■•- •,

• .•" ..-","./","'v"'.-'Vv.-"".- ', ""

■■.,- . .-
Workstation

Utilization Rate

• = 367 HOURS

- 500 HOURS

=1000 HOURS

26-

20-

<

_J

O

Q

15-

10-

5-

23.89

8.77

6.46

12.89

9.46

4.73

T T T

3 5

NUMBER OF WORKSTATIONS

10

Figure 3. Cos.: per student hour-lA2 Megabyte Desktop

10
In this situation, each of twenty-five soldiers In the unit receives

four hours of Individual CBT training each month for eleven months. They

therefore require a total of 1,100 workstation hours per year In order for

each soldier to receive 44 hours MTP-RC training, the approximate number of

hours of MTP-RC training for each MOS. If the RC training site has three

workstations connected to a System II with 100 megabytes of disk storage,

the annual workstation utilization rate would be 367 hours per workstation.

Such a system for delivering MTP-RC rourseware Is scheduled for fielding at

the 1st Detachment at Red Springs, North Carolina. If, however, a five

workstation configuration were fielded, the annual workstation utilization

rate would fall vo 220 hours per workstation. As workstation utlllzat'on

rates fall, system cost per student hour Increases. In this case, the

system cost per student hour would Increase by 19% from $28.16 per hour to

$33.64 per hour with the addition of two workstations.

Table 3 presents the costs of two configurations of a System II with

100 megabytes disc storage and two configurations of a Desktop with 142

megabytes disk storag«-. In this table the total workstation hours per year

was held constant at 1,100 hours while the number of workstations was

Increased from three to five. Although adding workstations In this case Is

not cost effective In terms of hardware costs, there are other factors to

consider, such as convenience of scheduling training during the drill

period. In fact, sections below will show that it is Infeaslble to schedule

twenty-five soldiers to receive four hours of training per month with only

three workstations.

Table 3

Number of Workstations and Cost Per Student Hour

System II with 100 megabytes

disk storage Desktop with 142 megabytes

3 workstations 5 workstations 3 workstations 5 workstations

$151,500

Total system hardware

cost (5 years) $155,000 $185,000 $131,500

Annual cost per

workstation $10,330 $7,400 $8,767 $6,460

Annual workstation

utilization rate 367 hrs/year 220 hrs/year 367 hrs/year 220 hrs/year

System cost per

student hour $28.16 $33.64 $23.89 $29.36

Note: Total workstation hours per year = 1,100

11

:-:■■'-
Disk Storage Capac-tty. Disk storage capacity Is another factor to

consider when selecting a hardware configuration. MTP-RC courseware is

recorded on tape sets, each of which includes several units from the

syllabus. The system operator at a training site will load the courseware

required by a majority of soldiers from the appropriate tape set. Systems

with more disk storage allow more of the courseware syllabus to be accessed

at one time. This gives greater flexibility for scheduling students based

on their rates of individual progress.

Soldiers who proceed through the courseware faster or slower than

average soldiers would be unable to access their lessons because the system

operator could load only the courseware that was needed by a majority of the

students. Training for the faster or slower students would have to be

rescheduled and the required courseware reloaded. Having more courseware on

line also demands less operator time because the courseware needs to be

changed less often.

The advantages of a system with larger storage for a program like

MTP-RC with a large amount of courseware are reduced operator time for

changing courseware and greater accessibility of courseware. Fewer soldiers

would require rescheduling to use the CBT system. The advantage of a system

with smaller storage Is mainly in reduced cost of the system.

The disk storage requirements of an RC training site depends on the

number of MTP-RC courses that must be available at one time. Soldiers

training at the DS/GS level should use the courseware targeted for the

organizational level to supplement the DS/GS courseware. Therefore, the

Westminster and Dagsboro sites with DS/GS soldiers need access to courseware

designed for all four MOSs. These DS/GS training sites would benefit more

from delivery systems with greater amounts of disk storage than

organizational training sites. The Red Springs detachment is unusual In

that It is an organizational maintenance unit with direct support

maintenance personal also assigned to It. Red Springs has the greatest need

for courseware storage of all three MTP-RC sites.

The cost figures of two example systems are presented In Table 4, which

compares two configurations of the System II and two configurations of the

Desktop. In this table, the variable Is the amount of disk storage in each

system configuration. The number of workstations, total workstation hours

per year (1,100), and aniual workstation utilization rate (367 hours/year)

are constant. The workstation utilization rate Is constant because in all

cases the total workstation hours per year Is divided by three workstations.

The system cost per student hour Increases 19% from $23.61 per hour on the

50 megabyte systtra to $28.16 per hour on the 100 megabyte system. The

system cost per student hour also Increases 19% from $20.07 on the Desktop

with 71 megabytes to $23.89 on the 142 megabyte system. Although the system

cost per student hour is higher with the system with greater disk storage

capacity, one must balance this cost with the possible benefits derived from

decreased operator time and greater scheduling flexibility which would allow

training to proceed faster and more efficiently.

12

-. .-_ .. -.V- --.' ■■. -'. -. --- . .•-•- - - . f.'.-^^-J'.f ^-- •—---.^-■■^..■■•.■■.--.-•.•-- ..•-V->-V..V . ..•.:
Table 4

Disk Storage and System Cost Per Student Hour

System II with three workstations

50 Mbytes

Total system cost

(5 years) $130,000

Annual cost per

workstation $8,667

System cost per

student hour $23.61

100 Mbytes

$155,000

$10,333

Annual workstation

utilization rate 367 hrs/year 367 hrs/year

$28.16

Sote. Total workstation hours per year 1,100

Desktop systems with three ^^

Workstations !£•!

71 Mbytes 142 Mbytes

$110,500

$7,367

$20.07

$131,500

$8,767

367 hrs/year 367 hrs/year

$23.89

The data In Table 4 also show that when the workstation utilization

rate ".s constant (367 hours per workstation) the cost for the Desktop system

per student hour Is between 9% and 18X lower than for the System II. The

Desktop cost Is lower even though It has th? advantages of larger disk

storage capacity. Therefore, If authoring capability Is not required and

the system to be fielded Is only for courseware delivery, the Desktop is the

cost effective choice.

Increasing System Utilization and Its Effect on Training Schedules. A

third factor that influences system cost per student hour is system

utilization. System cost per student hour is Inversely proportional to

system utilization if the number of workstations is constant. Consequently,

cost per student hour can be decreased by increasing system utilization.

The typical RC unit's schedule allows for twenty-two days of computerbased training per year during weekend drills. This schedule gives fairly

low utilization of an expensive system. System utilization can be Increased

by using the same system at two training sites if the training schedule can

be coordinated. Another way is to locate the MicroTICCIT system at a

regional training center '..'^ere it can be shared by several RC unit;.. The

following paragraphs ise a possible MTP-RC fielding plan for Westminster,

Maryland, and Dagsboro, Delaware, to Illustrate the advantages and

disadvantages of sharing a system between two sites. Both units have

approximately 25 soldiers eligible for MTP-RC trailing.

13

*

.-j - - .- - .--J . ; - ..j-..- .- /- ■■.•..-. i. i.V.f..........-» IM ,.,...-.•,.,. .A. •.

s J ' - v v vv^-- y '- ". •.-.'.
Installation of a System II with 50 megabytes of storage and three

workstations Is planned for Dagsboro. Each soldier Is typically scheduled

for one four-hour drill period of individual training per weekend. If each

soldier spends four hours per weekend on the CBT courseware, the total

system utilization would be 110Ü hours. The annual utilization would be 367

hours per workstation, and the cost per student hour would be $23.61, as

shown on Table 3.

Table 5

Two Fielding Scenarios

System II with 50 megabytes disk storage and three workstations

2198th 2198th and 195th

System available U mos/yr 6 raos/yr 6 mos/yr

Soldiers Assigned 25 1 25 21

Drill Periods 11 1 f 6

Hrs per drill

period for Individual 4 1 8

training

Total CBT hours

per soldier 44 4« 48

Total workstation

hours per year 1, 100 2,024

Annual workstation

ut 11IzatIon rate

367 hrs per

workstat Ion

per year

675 hrs per

workstat fon

per year

System cost per

student hour $23.61 $12.£4

It would be Impossible for all 25 soldiers to use the system on one weekend,

however. If only three were workstations available. Dividing the unit Into

two groups which drill on different weekends would allow all soldiers four

hours of CBT time, but only If some soldiers are excused from group drill

activities. This plan would allow all soldiers to receive CBT training In

one year at the expense of group activities. In addition, each soldier

would have to spend eight hours, half of the monthly drill time, on the CBT

system to complete the training In about one year.

14

f:,

täM^m&tä :-:.v,v:^-:-:--v:-/:-/.v.-.-:-;:::-^. ^i . .- .v. -. -v^- ^ _._% - ,• .;-.' •_■■ •- v o ..
Moving the system to Westminster after six months would require that

both units be split Into four groups each, meeting on four different

weekends. In addition, each soldier would have to spend eight hours, half

of the monthly drill time, on the CBT each month to complete the training.

This plan obviously Is Infeaslble. This plan for sharing systems between

sites would Increase system utilization and bring the system cost per

student hour down 46% from $23.61 to $12.84, but It cannot be recommended

because of the adverse Impact on other activities of the unit. Another

disadvantage to this fielding plan Is that It subjects sophisticated

electronic equipment to the hazards of moving. Some down-time would be

required for disassembling, transporting, reassembling, and repairing the

system.

Another fielding strategy Is the deployment of two small systems, such

as a Desktop with four workstations, at each of two training sites.

Additional workstations could then be shifted from one RC training site to

another as fluctuations In training loads occurred. The advantage of this

plan Is that only the workstations would be moved, and only on demand rather

than on a fixed schedule. The decrease In system cost per student hour

would be mlnlraaJ , however, unless a large number of workstations were moved.

The workstations would still face the hazards of moving.

The result of sharing a system between two sites Is that system cost

per student hour decreases, but doing so places unacceptable restraints on

the training schedules of the two RC units. Moving the system to Increase

system utilization Is not a recommended fielding plan because It Interferes

with unit drills and subjects sophisticated electronic equipment to the

hazards of moving.

Although system cost per student hour can be lowered through Increased

system utilization, attention must be focused on the effect It has on the

resulting training schedules. The constraints Inherent to RC training

schedules means that the system utilization rate at a typical RC inlt will

be lower than that expected at a site where large numbers of tralrees would

have dally access to the system.

.-.-- Summary and Recommendations

The system cost per student hour method of measuring computer-based

delivery systems was used In this analysis because It accounts for the

life-cycle costs of the system. System cost per student hour was calculated

using the following steps:

1. Total system hardware cost over the life of the system.

2. Annual cost per workstation.

3. Annual workstation utilization rate.

13

,

_;_- V—_a!--'-j

, fvc-ft. .•-.- . .-.• .

iün . - . . - ■■-.

• . . ~- ..- - . .:.-..•.:.•. .
,%" V

The cost of various configurations and training srhedules was

presented. From the analysis of hypothetical situatlois, the following

polnta were highlighted.

1. Attempts should be made to increase workstation utilization rates,

2. When the workstation utilization rate has reached ehe point of

negatively affecting the training scheduie, then additional

workstations should be considered.

-.

Consideration should be given to future training needs and how many

workstations and how much disk storage capacity may be needed

later.

Recommendat: tons.

To field MTP-RC courseware on a wide basis with MicroTICCIT systems,

the Desktops offer the lowest cost per student hour of the three MicroTICCIT

systems analyzed. When the cost per student hour of a System II and a

Desktop with the same number of workstations are compared, the Desktops are

lower In system cost per studsnt hour than the System IIs. Even when a

Desktop with 142 megabytes is compared to the System II with only 50

megabytes, the Desktop is only a few percent points higher In price. The

Increased amount of disk storage of the Desktop with 142 Mbytes makes

scheduling of training easier and will allow the delivery system to support

additional courseware. Although the Desktop does not support authoring of

courseware, the delivery sites do not require authoring capability.

System cost per student hour is a useful tool for analyzing CBT

delivery systems, but the selection of the right system configuration for a

RC training site must balance these hardware costs with scheduling needs and

requirements. More information is required to determine the Impact of

particular system configurations on the training schedules at actual reserve

training units. Such information will be collected during the trial field

Implementation (Phase III) of MTP-RC.

16

^•;^v^<^A..-:.v.^.r:. .•. •. •■■: v.

/ . .- A-' • •v-^-v ■■"<--'. • :';- ■■■■. ' JK"

J."....,,.. .-..
PART TWO: COURSEWARE DEVELOPMENT COSTS

Overview

Labor costs for five courseware templates developed for the Model

Training Program for Reserve Component units are presented In this part of

"^e report. Factors that affect courseware development costs are

luentlfled. The results should be useful In predicting labor costs when

similar courseware development stages and templates are used for the

presentation of complex technical content. The following sections are

presented:

1. Courseware Templates.

2. Labor Categories in Courseware Development.

A. The Development Process.

4. Findings.

5. Utilization of the Findings.

6. Summary.

Because MTP-RC Is a research project designed to determine the efficacy

of computer-based training In RC training, It Is desirable to know how much

It cost to produce different MTP-RC lessons. The costs reported In this

document are based on the production records for the program. The cost of

developing courseware Is Influenced by a number of factors. They Include:

1. The steps Included In the development process,

2. The complexity and quantity of the content.

3. The Instructional strategy selected.

4. The courseware authoring language used.

5. The experience of the production personnel.

The discussions of courseware development cost focus on the steps In

the development process, but the Influence of the other factors are also

discussed. The discussion will show how complexity of the content, quantity

of the content, and Instructional strategy affect the amount of labor and

resources required to produce CBT courseware. Because all courseware

authoring for this program was done on the ADAPT system In MIcroTICCIT, the

discussions in the report will reflect only ADAPT authoring. The discussion

of the labor hours required for courseware development (Findings) will refer

to the experience of the production personnel during peak production;

supervisory and support personnel are not Included.

17

^Msisw
Courseware Templates

Understanding the production process requires certain knowledge of he

structure of the MTP-RC courseware. MTP-RC courseware Is unusual In that

most of the lessons were produced using extensive and detailed templates.

The templates, which were the resalt of courseware engineering (Stacy,

1984), wore strategies and code for types of lessons. For example, to teach

the cognitive objective for understanding the functions of a tank system, a

template was developed with Interactions to allow soldiers easy access to

the Information about the system ar.d to test the soldiers' acquisition of

the knowledge.

A functional diagram of the system was presented, and by marking a

component on the diagram as If pointing to a menu, the soldier received

Instruction on that component. The next part of that template would be a

practice or test session allowing the the soldier to demonstrate mastery of

the content.

Although templates are becoming common as a tlnresavlng CBT production

tool, the MTP-RC templates are unusual In their sco'je. Templates on other

projects might be simply a reusable screen design format and code for

multiple choice questions. An iMTP-RC template, on the other hand, would

Include reusable format an.Jcode for Introductions, Instruction, special

assistance, menus, review, and multiple choice tests, ill of which are

components of an integrated Instructional strategy for a type of performance

objective. For a more detailed description of the MTP-RC courseware

templates and the structure of the courses, refer to Appendix C. The five

templates that compose most of the MTP-RC courseware were reviewed for

analysis In this report. The five templates are:

1. Name, Locate, Function (N^F).

2. Input, Process, Output (IPO).

3. Troubleshooting Introductions

4. Troubleshooting Simulations.

5. Maintenance Simulations.

The variations In labor required tc produce lessons based on each

temolate are discussed In the Findings section of this report.

18

^.^M" k^d •' - '

.■■ -..-. .- 'V-V-

--•- - . .v.-- — -^-
Labor Categories

MTP-RC courseware was designed and developed by a team with expertise

'.n a variety of areas. The production team Included Managers, Instructional

designers, Courseware Developers, Production Supervisor, Subject Matter

Experts, Video Specialists, Graphics Specialists, a MIcroTICCIT System

Operator, and Technical Coordinator. Four labor categories were established

to document the courseware production costs In this report.

Time sheets were maintained by all participants on the program to

record the hours spent on tasks. The purpose of these records was to allow

billing and some tracking of production, but they also provide some useful

Information for analyzing costs of production. During the last half of the

production period, Courseware Developers (CDs) and Graphics Specialists

(GSs) also kept detailed production logs, recording the hours spent in

production of specific lessons. At least once a day, CDs and GSs recorded

the amount of time spent on each lesson that day and the amount of time

estimated to complete the work. It was typical for the CDs and GSs to have

worked on several lessons each day, often with each lesson In different

stages of production, review, or revision.

An example of a Courseware Developer's work log Is presented In

Appendix D. On this sheet, the CD recorded the time spent on tasks for

lessons. The log lists tasks and subtasks In the development process broken

down Into logical subtasks which allowed the Production Supervisor to track

the work flow and estimate the time remaining to complete the lesson. The

major tasks are explained In the section below on the development process.

A time sheet for an Instrictlonal Designer Is also presented iu

Appendix D. The time records for IDs and SMEs are not as detailed because

they were Intended only as time sheets for keeping track of hours spent on

the contract. Nonetheless, they allow reasonably accurate estimates of time

per segment. IDs were usually working on a small number of segments at one

time and had fewer subtasks than the CDs, so It Is possible to estimate the

amount of hours spent on ID tasks for a lesson by looking at when the lesson

scripts were completed and when review wera completed and assigning hours

from the time sheet to the lessons under development at that time. The time

sheets for IDs and SMEs were not designed to keep precise records for

lessons, so the estimates developed for these categories must be considered

very approximate. They can be considered accurate within 20%.

For other labor categories, detailed logs were not maintained and It

was not possible to attribute hours to specific lessons. For example, when

Courseware Developers were working on specific single lessons, the

Production Supervisor was coordinating the workflow for as many as thirty

lessons simultaneously. The Supervisor's hours were then charged to

management task categories without assignment to specific lessons. The

production cost report Is limited to the following labor categories and

19

^. ......

, *.

• -" -i -«• ,-. .•.<• •-i.l'. ". i". i. . . :. '»--'. . , — .- ,■■»-. >--
tasks for which reasonably accurate records or estimates of time spent per

lesson are available:

1. Instructional Designers.

2. Subject Matter Experts.

3. Courseware Developers.

4. Graphics Specialists.

The general responsibilities of these labor categories will be defined

briefly before more detailed explanations are given about the roles they

played In the development of MTP-RC courseware.

Instructional Designers develop the overall plan for the design of the

learning activities. IDs select and analyze content for Inclusion in the

courseware before designing Instruction and the Interaction between the

soldier and the courseware. In performing these tasks, IDs consult with

Subject Matter Experts about content and consult with Courseware Developers

about programming code. Once the overall design for a lesson template Is

completed, IDs write lesson content for specific lessons In that template.

Subject Matter Exnerts have technical expertise in the content area and

usually are familiar with the target audience. SMEs play an Important role

In the content selection process, in generating content for lessons, and in

reviewing content materials for subject matter accuracy.

Courseware Developers have expertise in the authoring language, the

authoring system, or the programming language used to translate the ID's

content and Interaction specifications Into courseware code. Tney generally

work from a script or storyboard prepared by an ID for a specific lesson.

CDs use the template code for the type of lesson, enter the text from the

script, and adjust the code for unusual presentations or Interactions

demanded by the content of that lesson. MTP-RC CDs were generally

responsible for the making and implementing final screen design decisions,

such as color of text and location of touch spots on the menus. They also

were responsible for the "mechanical" operation of the lesson—making sure

that the computer responded appropriately to soldiers' Input, encoding

scoring criteria, coding menus, and coding video comnands. Although the

template provided nearly all the code for a "standard" lesson, no lesson was

exactly standard. All lessons required some modifications of the code.

Graphics Specialists are experts in visual design who create the

technical drawings, charts, menus, diagrams and cartoons used to Illustrate

the courseware content. GSs generally start with the script and graphics

request prepared by the designer of the lesson. They work closely with the

ID to understand the Instructional Intent, with the SME to ensure technical

accuracy, and with the CD to make the graphic work properly with the code.

Some of the MTP-RC graphics weie created with the on-line MlcroTIcclt

graphics editor, but many, especially cartoons, were drawn by hand,

transferred Into the graphics storage of the computer with a digitizing

video camera, and then modified through the graphics editor.

20

'-'-'- -- r- -' --

> ."-A '-> '^ -LA fL> ILW AZAZ
The major production tasks for each of the labor categories are

summarized In Table 6. The listed tasks are the activities during the main

production phase, that Is, after the task analysis and template design are

completed.

Table 6

Labor Categories and Major Production Tasks

Category

Instructional Designer

Subject Matter Expert

Main Tasks

write scripts, specify video and graphics,

review scripts, review lessons

generate content, review technical accuracy,

advise IDs and GSs

Courseware Developer

Graphics Specialist

enter data, modify code. Integrate video,

review lessons, debug code

design and draw functional graphics and

cartoons

The Development Process

One major problem In evaluating or comparing courseware development

costs between projects or organizations Is that there are neither

standardized labor categories nor definitions of the development process

Itself. Most definitions of the development process Include writing the

lesson and coding it on-line, but some organizations have both these tasks

performed by one person (e.g., a Courseware Author) and some use two people

(e.g., an Instructional Designer and a Courseware Developer). Other steps

of production may or may not be Included by other organizations In their

program descriptions. The steps Included affect the calculation of the

number of hours required to produce a unit of Instruction. When comparing

courseware production figures. It Is essential know which steps were

Included In the cost figures.

The process used to develop MTP-RC courseware conformed to the Systems

Approach to Training (SAT) model with some enhancements. In the early

stages, content selection, task analysis, and training task analysis were

conducted by the design team. The design process used In MTP-RC differed

from the standard SAT process in that a process of generating templates was

used. The process called, "courseware engineering," resulted In the

development of courseware templates, each of which Is a generic, thoroughly

structured approach for teaching a specific category of learner behavior.

Each template consisted of a carefully prescribed Instructional strategy, a

21

- - -

....-•- . r.--..\. ..•.---- .•-. -. -' -- ^•■■^ .......... ^:^.. '.-■■ -" ■■
predetermined set of screen display charac-ter1stles, and a structured system

for student Interaction. In this process, an analysis of course content

during the development of program objectives ^ to the Identification of

the templates required for a given course. Thv details of the template

design process are described by Marco, Begg, Israelite and Bernstein (1985).

Once templates were developed and approved by the design ti?am, lesson

proüuctlon began. Consequently, this cost analysis of the production

process does not Include template design and coding. The courseware

production steps used in the production ">f MTF-RC and which are the basis of

the cost report were:

1. Generate lesson content.

2. 1D/SME review.

3. Prepare lesson for production.

4. Input lesson data and adjust code,

5. CL review.

6. ID review.

7. Senior ID review.

8. Army review of lessons.

9. Integrate video and graphics.

Each of these stages of the production process are described below with

the relevant activities of each labor category described.

Generate Lesson Content. The IDs wrote the content using the lesson

templates and content Information provided by the Institution and the SMEs.

The MTP-RC SMEs were also sometimes used to write lesson content. The

lesson content Included descriptions of graphic and video Images to be used.

The descriptions included the Image content, any response windows that were

required, and any other Information to make the graphic easier to produce.

ID/SME Review. Lessons were then submitted to another ID for design

review and to a SME for content review. Lessons were returned to the author

for revisions. This review process continued until the lessons were

approved.

After lesson specifications had undergone their final revision and

approval, they were released for on-line production. The production phase

consisted of two separate tracks, courseware production and video

production. The courseware production phase will be discussed first.

22

: : I--:-.--:-. .<. :^.^..:.- ..:.:-*.-..^.. -. :- _- ■■ _ . - - -. . ^-. .-:-^.-:-.:.v. . .- . ...-■■■ :-.:. ..v^
Prepare Lesson for Production. When a lesson was approved, the ID

submitted It to the Production Manager. The Production Manager channeled

the various components of the lesson specifications to different personnel.

Graphics requirements were directed to the GSs for design and Input Into the

system. Components of the lesson that make use of video were turned over to

the Video Specialist (VS). The VS took the IDs' video descriptions and

developed the storyboards and scripts for video production. The CD reviewed

the lesson, gathered any additional necessary Information, and prepared the

lesson for entry Into the system.

Input Lesson Data. It was during this phase of the production process

that Lesson data were entered Into the CBT system by the Courseware

Developers. Code templates were used as foundations for all lessons, but

typically an Individual lesson would require minor modifications to a

template. Modifications to a template were made as the lesson data were

entered. The Graphics Specialists designed, drew, and edited the various

kinds of line drawings, cartoons and other graphics. Graphics were

Integrated with lesson data during this step In the production process.

CD Review. Each lesson underwent two CD reviews. First, the lesson

was reviewed by the CD who entered It. The purpose of the review was to

verify that the data entry was accurate and that any modifications to the

code template worked as Intended. Revisions were made until the lesson was

correct. The second CD review was designed as a confirmation that the

lesson worked: branches were correct, graphics appeared In the correct

place, spelling was correct, etc. Correction and revisions then were made

by the atthorlng CD.

ID Rv.-v'.ew. When the data entry and review/debug process were complete,

a lesson was released for ID review. The ID's main objective was to verify

that the on-line lesson accuratel / reflected the lesson specification, but

the ID was also checking to assure that the lesson or graphic as conceived

In the specifications was actually effective on screen. This review was

essential because tie CD sometimes lud to rewrite text to fit within the CBT

screen format. This review was not .ntended as a major design review.

Major design modifications would have greatly Increased development time and

generally were avoided by the thorough ID and SME review conducted earlier.

Modifications and revisions were usually made by the CD who entered the

lesson.

Senior ID Review. The senior ID conducted the final lesson review.

This review typically verified that lessons were complete. Occasionally,

however, design and production flaws were identified. In such cases,

lessons were submitted for revision and modification.

Army Review. When lessons were completed, they were sent to the

Ordnance School for review. Because video production was concurrent with

early courseware production, some video segments were not available for Army

review at this phase of development. Consequently the courseware the Army

reviewed at this time sometimes Included the graphics but lacked the video.

Lesson designs had been previously approved, therefore, changes In the

23
lesson designs were seldom required as a result of the Army review. The

major focus of this review was content accuracy.

At the same time that courseware was being produced, video production

was also underway. When lesson content was generated, the author ID wrote

video specifications that consisted of detailed descriptions of the video

Images required for the lesson. These specifications were given to the

Video Speclallst. It Is at this point that the video production process

began. It should be noted that not all of the steps required for video

production were Included In this cost report. The video scripting,

storyboardlng, shoot, post production work, and cataloging the disc were not

Included In this report because these tasks relate to the whole of MTP-RC

courseware. Consequently, It was not possible to report the amount of time

required for Individual lessons. It was possible, however, to tabulate the

amount of time per template the CDs spent to Integrate the video Into the

courseware.

Integrate Video and Graphics. The vld^o Integration process required

that the video sequences specified by the 10 be Inserted Into the courseware

by a CD. Integrating the video sometimes required adjusting the code,

especially If there was motion video or audio. Graphic overlays also had to

be Integrated Into the courseware at this stage of development. Ideally,

video would be available when lesson data were first entered Into the CBT

system. Because of the long lead time required for video production, video

Integration typically occurred near the end of the development process,

prior to final reviews and client acceptance. Once video Integration was

complete It was possible for the client to review the courseware In Its

completed form.

:•:•:

»

FIndlngs: Courseware Production Costs

Cost Informat

7. The number of

reported for each

!abor involved In

reasonably be assl

includa the labor

task analvsis, and

Include tl.ie spent

The figures also d

generate the tempi

code. Finally, th

Specialist on the

ion figures for the five templates are presented In Table

hours required by each of four labor categories Is

template. The figures contained In Table 7 reflect the

the actual courseware production process which could

gned to specific lessons. The figures, therefore, neither

performed by the IDs and SMEs In the content selection,

training task analysis procedures, nor do the figures

writing evaluations, reports, and general correspondence,

o not Include the labor required by the design team to

ates or the time required to translate the templates Into

e figures do not Include the time spent by the Video

video shoot, video post-production, and disc cataloging.

^4

K

f.A. .. --I . •... , . - . . •.-... .. .. i". -: - '.". r'. v. ^. '- -•- ---V. -'- -.'- .' i'- -•-"»•- >'- Cdjtd i"- -"- "-
.>v

Table 7

Courseware Production: Average Labor Figures In Hours

LESSON TYPE

LABOR nr IPO INTROS TROUBLE- MAINTENANCE

CATEGORY SHOOTING

N=6 N»4 N-5 N-6 •4

VJ 110 110 4(1 120 155

CD 100 100 60 155 125

GS 50 50 AO 80 50

SME 30 30 20 S9 J5

Note. All fig ures +/- 20%

The analysis of these records will help for a very rough gauge of

relative production costs by lesson type and labor category. The

recordkeeplng, however, was not Intended to provide data for such an

analysis, so both the number of lessons for analysis and the accuracy of the

data are limited. The figures are most reliable In comparing relative labor

per lesson type or labor r^Legory. A more carefully controlled study

designed to accurately record the hours for a number of segments would be

necessary to determine the hours per segments more precisely. Such a study

Is planned for Phase III of MTP-RC. Nonetheless, a review of the MTP-RC

production experience to date provides useful Insight Into the production

process.

The number of lessons for which usable records were available Is

limited for other reasons In addition to the lack of Intent to analyze the

production. Many of the Name, Location and Function; Input, Process,

Output; and Troubleshooting Introduction lessons were produced with the

templates by Hazeltlne Corporation under subcontract to Scientific Systems.

Hazeltlne was not asked !"o keep production logs by lesson. All of these

lessons, about thirty, are "herefore excluded from the data base. Other

lessons have been excluded because they were not produced during periods of

routine production. During the first months of the program, SSI Courseware

Developers were learning the ADAPT language, so training time confounds

production time. The first few lessons of each type were affected so much

by revisions being made to the templates as they went Into production that

they are not Included In the report of routine production. Other lessons

simply had missing or obviously Inaccurate Information; these were not used.

Lessons which required extensive modifications to the template too anomalous

for Inclusion. Even with the exclusion of these lessons, the estimates for

23

-•- •-. f. -f. •'.

^ • ^ - * *

J s s

_
lessons may vary +/- 20% from the average. The approximate number of

lessons of each type used In the cost breakdown Is Included In Table 7.

The figures represent the hours spent by each labor category on each

template during the courseware production phase of MTP-RC. The figures

represent the amount of time to write and to revise lessons, scripts and

generate content. The figures Include the amount of time entering code,

processing code, debugging, revising code and Integrating video and making

changes based on Army reviews and Information obtained from formative

evaluatIons.

The figures In Table b represent approximate averages. Actual

production times may vary by 20%. The variation Is attributable to a number

of variables, Including the difficulty of the content, the length of the

procedure being taught, and the actual number of Items that composed the

segment. For example, some Troubleshooting lessons are composed of as few

as 200 Items while others were composed of as many as 450 Items. Some tank

systems Include five components requiring Instruction; others Include 10 or

12, so their lessons are longer and require more production time.

The averages for the NLF and IPO lessons Include some lessons produced

before all the production staff were working at the productivity rates they

achieved on subsequent templates. The NLFs and IPOs were the first kinds of

segments to be produced. Therefore, IDs were beginning to learn about the

content, CDs were still Increasing their knowledge of the authoring

language, and GSs had not yet developed all of the labor saving techniques

they eventually created. The data listed for the other three templates are

based on average times when personnel were working at higher productivity

rates.

NLF and IPO Lessons. Although the hours required for the NLFs and IPOs

are the same, the nature of the work performed by IDs differed between the

two templates. NLF lesson content was broader than IPO lesson content. A

typical NLF lesson might discuss seven Ml tank system parts while a typical

IPO lesson might discuss three or four of those parts In depth. IPO lessons

extended the explanations given In the NLF lesson for parts that required

further explanation. So, although the number of parts the ID had to learn

about were fewer In number, the ID had to spend more time reading the

reference materials and consulting the SME In order to be able to explain

the parts In depth. Therefore, the amount of time the ID spent on both

types of templates averaged out to the same number of hours.

The time required by the CDs for entering data, debugging, and

processing Items for both templates was quite similar. Although the CDs had

fewer Items to encode for IPO segments, the coding was more complex than for

NLF segments because of the required changes In screen displays. The amount

of t^me required by the GSs and SME was the same for both templates. The

SME consulted with the IDs and explained the operation of the systems. The

SME worked with the IDs to Identify the majors parts of the system, explain

their functions, and generate functional descriptions of system which the

IDs used. The SME also was responsible for locating and obtaining existing

lb

.-* . ^'-/l..--.S.. . ■■ .V.V.V.-V.

:-;• ; .-: :-;'-: :.;•: • :•

•. -.

.-'.-•.. ..«.i. »• > j-.-._..

v.'A"-".. ■■' •'".• -vv'v'.A . !""!-''" .

.-_ .; ._
visuals of the system parts and was responsible for describing those parts

which would require video Images. Once the ID had written the segment

specifications, the SME reviewed them for technical accuracy.

The SME worked with the GSs to Identify what was required ot the

graphics, which functioned both as menus and Instructional content In NLF

and IPO segments. There are two typei of such graphics: realisticrepresentations of a system (e.g. the engine or transmission) or diagrams

of the functional relationships of parts not In close proximity to each

other on the tank (e.g. the Laser Range Finder and the Line of Sight

Electronics Unit). The SME Identified to the GSs the major parts that had

to be Included In the graphics. The GSs created the graphics based on the

guidance of the SME and the ID's segment specifications. The GSs also

created many motivational cartoons for the NLF and IPO segments. When video

Images were not available, the GSs created realistic Hie drawings of the

parts.

Troubleshooting Introduction Lessons. Troubleshooting Introductions

required fewer personnel hours to produce than any of the other templates,

because they have fewer pages and simpler branching options. The IDs were

able to use Information that already had been generated for NLFs, IPOs and

the "conceptual help" material designed for use within the Troubleshooting

Simulation lessons. Typically, the ID worked concurrently on writing the

conceptual help material for the Troubleshooting Simulation and on writing

the Troubleshooting Introduction.

Compared to NLF and IPO lessons, CDs required fewer hours to enter the

data Items because there was less material to enter. What material had to

be entered required simple branching and contained no tests. GSs also

required fewer hours to produce the graphics needed for Troubleshooting

Introductions because the GSs were able to modify graphics that had been

developed for previous templates. Including the graphics required for

conceptual help In the Troubleshooting Simulation lessons.

The SME's role In producing Troubleshooting Introductions was different

than the role he performed In the production of NLF and IPO lessons. The

SME had to Identify the feasible set of faults that might be responsible for

the symptom that was the focus of the lesson. The SME had to generate the

explanations of what each troubleshooting test did. This Information was

not contained In the soldlers' TM, therefore, the SME had to establish the

logic of the troubleshooting procedures, using wiring diagrams and often

consulting vendors of tank components. Because of the relative simplicity

of the template, the SME followed models created by the IDs to write content

for Troubleshooting Introductions. Consequently, based on the average

number of pages in this type of lesson segment, the SME spent

proportionately more time on the Troubleshooting Introductions than on the

NLFs and IPOs.

Troubleshooting Simulation Lessons. Instructional Designers generated

content for the Troubleshooting Simulation template by decomposing TM

procedures Into their component steps. Many ID hours were needed because

27

K

._•■■ -
each of Che 250 to 450 steps of a procedure required generation of up to

nine separate Information bits, such as graphics requests, feedback

responses, help messages, etc.

The large number of Items In Troubleshooting Simulations also required

considerable effort on the part of CDs, who were responsible for entering

all the data, reviewing the lesson, and debugging the code. To review the

lesson the CD had to process the code Into the student version and then

reprocess to Inspect the revisions. Each processing took over one hour.

Another factor responsible for CD time was modifying the template code for

unforeseen requirements, such as timed displays on automated test equipment.

Many CD hours were also spent modifying the code to achieve effects that

were not explicitly built Into MlcroTlcelt's ADAPT authoring language.

ADAPT makes some kinds of lessons very easy to author, such as

Rule-Example-Practlce lessons. Using ADAPT to create Interactions for which

It was not designed, however, required that CDs spend many hours modifying

ide, testlr.g, and debugging untl' the required Interaction performed

flawlessly.

Troubleshooting Simulations required more Graphics Speclallst hours per

segment than any other type of lesson because they needed realistic graphic

Images to represent the Ml stations, components, and test equipment. These

simulations were desl;ne( to use graphics Instead of video images to shorten

the machine response ' Ime anu to eliminate distracting visual information

from the screen. Production of hundreds of realistic Images for each

simulation was a very labor intensive effort.

Methods were developed to Increase graphics production efficiency. The

GSs created an on-line character library, a set of drawings of parts such as

plugs and cables. The GSs were able to retrieve these images and use them

repeatedly as baric construction blocks when they needed to create a new

Image. They also created an on-line library of complete drawings, such as

line replaceable units and test equipment, which could be retrieved and

modified for different lessons. These two techniques saved many hours

compared to created all drawing anew. Many drawings were created on paper

and then transferred to the graphics editor with a digitizing video camera.

The Troubleshooting Simulation template provided the IDs with

considerable structure. The IDs generated Troubleshooting Simulation

content largely by following the procedures detailed In the TMs. This task

of decomposing procedures Into their component steps Is conceptually easy,

but the task was time consuming because of the large number of steps or data

items Involved. The number of steps in Troubleshooting Simulations

typically ranged between 200 and 450 steps. Each Individual step required

the generation of approximately nine separate bits of Information, Including

graphic requests, at least two different help messages, feedback responses,

error messages, etc.

t i

The large number of data Items also required a considerable effort on

the part of the CDs. In addition to entering the data, the CDs were

responsible for debugging the code as well. Debugging requires the

28

>i*iä

»tt

M^J
processing of the segment t-ode. This irotedure can take up to one hour.

The CD then checked the reprocessed code to confirm that the code was

bug-free; If tt was not, the CD had to continue debugging the code and

repeat the time consuming process. Another factor responsible for the

amount of CD time, was the adjustment of the template. Over the life of the

courseware production phase, CDs found It necessary, depending on the

content, to modify the template code In order to achieve special effects.

For example, the original template did not Include a timed display. When It

became necessary to Include a timed display In order to simulate a a readout

of a test set, a senior CD had to modify the code template. More

slgnlfI cantly,

Graphics Specialists had to create a substantial number of graphics to

represent all of the Ml stations, components, and test equipment used for

the Troubleshooting Simulations. Troubleshooting Simulations contained no

video Images; all the Images were graphics. Therefore they required more GS

hours per segment than any of the other segment types. Troubleshooting

Simulation lessons required realistic Images of high visual fidelity. The

simpler line drawings used In the NLF and IPO lessons were inappropriate for

this type of Instruction, Production of realistic graphic Images was labor

intensive. The GSs devised methods for increasing the efficiency of

croatlng Images and thereby reduced the number of hours that might otherwise

nave been required. GSs, instead of creating images on-line, were able to

produce many kinds of images faster by rendering them on paper first then

using an image digitizing camera to convert the image into data that could

be stored in the computer. The GSs created a character library, a set of

related parts, such as cables and plug ends that compose a system or part.

By storing these Images In the computer, the GSs were able to retrieve them

and use them repeatedly as basic construction blocks when they needed to

create a new Image. The new image did not have to be created from a blank

page, instead the Items in the character family were used as needed to

construct the new image. In a similar fashion, the GSs created a visual

library of elaborate drawings that were used repeatedly In MTP-RC

courseware. Images of complex line replaceable units and test equipment,

such as the multimeter, and breakout box were stored in computer memory and

recalled and modified as required. Modifying these detailed Images was less

time consuming than the time that would have been required to create new

Images each time.

The SME helped generate the conceptual help messages and checked the

large number of Items for technical accuracy. This technical review was

done once during the Initial review and a second time during the final

review. The SME also was responsible for locating appropriate graphics from

reference material and if the graphic was unobtainable, the SME was

responsible for sketching one- to aid the GS in producing a finished version

of the graphic.

Maintenance Simulation Lessons. Maintenance Simulation lessons were

composed of fewer Items than Troubleshooting Simulation lessons, but

Maintenance Simulation lessons were composed mainly of video Images. The

large amount of video required more ID time than had been required to

J9
produce the average Troubleshooting Simulation segment. The ID had to spend

considerable time locating the best frames to use. Sometimes this occurred

because the ID had to choose" from a number of possible frames. At other

times, the ID was faced with the challenge of finding a frame Intended for

another screen that could be used to Illustrate a step for which specific

video was not available. This process was time consuming and required

technical guidance from the SME.

In addition to the time required to select the necessary video frames,

ID time was spent making Instructional decisions at the segment level. The

IDs had to make many decisions about what material to Include In the lessons

and what material was of less Importance and could be deleted. The

Troubleshooting Simulations were basically a re-creatton of every step In

the procedure of the Skill Performance Aids (SPAs) manuals. The Maintenance

Simulations, on the other hand, deleted many steps that were common

mechanic's skills, such as removing bolts. The ID and SME had to make

decisions about the Inclusion of steps and this required Intensive review of

the procedure In light of expected entry level skills. Because the template

provided multiple ways for the soldier to Input a response, the IDs spent

time deciding when It was more appropriate to use one response mode than

another. For example, the ID could offer the soldier two ways to Initiate

the simulation of an action, such as the removal of a shaft. The soldier

could request the action by marking the Image on the screen or by marking a

line of text that best describes the desired action. The IDs had to decide

which Interaction was appropriate depending upon whether the part was

clearly visible In the video Image.

favsv

Fewer hours of CD time were required to produce Maintenance Simulation

lessons than Troubleshooting Simulation lessons, because there were fewer

Items to Input Into the CBT system. Another reason why less CD time was

required was the template Itself. When the template was coded, the design

of the code was so complete that very few modifications of the code were

required. This was true because this was the last template to be designed

and It was possible to anticipate variations and exceptions from a standard

lesson. In addition, the Maintenance Simulation template avoided the coding

problems In the Troubleshooting Simulation template. When production

started on Maintenance Simulation lessons, the production team had achieved

their greatest familiarity with the content and the authoring system. Fewer

CD hours were required to produce Maintenance Simulation lessons than

Troubleshooting Simulation lessons because fewer data Items required entry,

the template facilitated coding, and the CDs were more familiar with the

content and with the authoring system.

i

The primary Images used In Maintenance Simulation lessons were video,

but GSs spent time enhancing those video Images with overlays. The GSs

created a library of overlay shapes. Accurately positioning the graphicoverlay on the screen required the GS to find the specified video frame and

then verify that the overlay was In Its proper place. Not as many graphicImages had to be produced for Maintenance Simulation lessons compared to the

number required for Troubleshooting Simulation lessons, but the Images that

were required could not be obtained from other lessons, as had sometimes

30

. -•-.-. ^-•--.•-- ^ .. --•-^•-''
been the case with other lessons. The graphics and diagrams required for

Maintenance Simulation lessons had to be original creations. Producing

these original Images and producing the overlay shapes took as much time, on

average, as the time the GSs needed to produce graphics for the NLF and IPO

lessons.

The SME's role In producing the Maintenance Simulation lessons was

quite similar to the role he played In the production of the Troubleshooting

Simulation lessons. But, because these lessons use many still frame video

Images, the SME spent many additional hours assisting the IDs in selecting

video Images that were appropriate and technically correct. The SME also

spent time In the Initial and final reviews checking the technical accuracy

of each Instructional screen and help messages available to the sold'er on

each screen. The SME played an Important role In determining which steps

were critical to Include In these lessons.

Content and Complexity of Instruction. Production costs of lessons are

Influenced by the complexity of the content contained within the lesson, the

amount of content Included In the lesson, and the Instructional strategies

employed to teach that content. It Is not possible, with the current data,

to quantify the Impact of these factors, however their Influence on the

production of MTP-RC courseware can be discussed. Further research Is

planned for Phase III of MTP-RC to develop a cost gauge for courseware

development.

Designing Instruction for NLF lessons was less labor Intensive than

designing lessons for Troubleshooting Simulations, because the content for

NLF lessons required less complex Interactions. For example,the content of

an NLF lesson might teach the soldier the names of the external parts of an

engine transmission while a Troubleshooting Simulation might teach the

soldier the exact series of procedures required to diagnose a fault In an

electronic component. Generally, content designed to teach tasks composed

of many steps and choices required more resources than content designed to

teach simpler tasks. Therefore, the complexity of the content and quantity

of content Influenced production costs. As the complexity of the content

Increased, the amount of time IDs needed to understand the content

Increased. Complex technical content affectd the amount of SME time

required. The SME must consult with the IDs, help Identify key points,

verify content accuracy, locate reference materials, locate visuals, consult

with the GSs to verify the technical accuracy of the graphics, etc. Because

MTP-RC courseware presents content about troubleshooting and maintaining

sophisticated electronic and mechanical systems, the production of the

courseware required extensive resources and labor.

The amount of content Included In a lesson also affects the number of

labor hours requlrec to produce the lesson. The more content Included In a

lesson the more resources are required to produce that lesson. In NLF

lesson content was based on a tank systems or subsystems. Therefore, a

lesson about a system composed of many parts had to cover more content than

lessons composed of fewer parts. Additional content affects all four labor

categories. For example, IDs had to write more lesson specifications, SME's

il

^Nv>:}":^-^:^:^ . . . . . .
had to review more content, CD's had to code and debug more data, and GS's

had to create more graphics.

A third factor that affects production costs Is the Instructional

strategy selected to teach the content, because the strategy determines the

type of Interactions that will be required. Content that requires drill and

practice as an Instruct il strategy can be taught with courseware using a

simpler Interaction thai ontent which requires simulation. MTH-RC was

explicitly Intended to demonstrate and evaluate state of the art CUT

capabilities, such as complex branching. Interactions with videodisc,

multicolored graphics, graphic overlays on video Images, etc. For example,

In a Troubleshooting Simulation lesson a soldier may mark a part on the

screen, he may mark one of several action Icons to simulate the performance

of a task, he may select to receive help, he may move ahead, go back, and

exit the lesson. The number of possible Interactions per screen Is large.

A template design with all of these possible Interactions requires complex

program coding. In addition, all the possible branches, help messages,

error messages, graphics, video, and menus had to be specified and created.

The selection of simulation techniques to teach some MTP-RC content was

labor Intensive.

The Authoring Language. Authoring language Is another factor which

Influenced the amount of labor required to produce courseware. Hlllelsohn

(1984) sho„ d that the authoring language used to produce courseware

Influences the amount of time required to develop a lesson and that some

authoring tools are better and quicker than others at producing particular

types of displays. The authoring language used to produce MTP-RC courseware

was Hazeltlne's ADAPT language. Because ADAPT was the only authoring

language used. It Is not possible within this report to quantify how It

affected CD and GS productivity and production costs. Research Is planned

within Phase III of MTP-RC that would quantify the effect that authoring

languages have on production costs. It Is clear that ADAPT defined what

Interactions are possible, and defined how difficult they were to produce.

The graphics editor In the authoring language also determined the ease of

producing graphic Images. As Improvements are made In graphics editors,

coding will become more efficient and the associated cost^ could be expected

to decline.

Utilization of the Findings

The number of hours required by each of four labor categories to

produce each of five templates was Influenced by several factors. The

numbers were primarily Influenced by the courseware development process, the

type and complexity of Instruction, and to a les.'.er extent the authoring

system.

Gauging courseware development costs Is directly Influenced by the

courseware development process. The courseware production costs reported In

the findings reflect the labor performed by IDs, CDs, SMEs, and GSs during

the courseware production process. The steps Included In the KTPHtC

32

•/ '.

--- --••• . - ... . _k.
courseware development pro-ess were listed and each step In the process was

described briefly. The reported costs do not Include the labor required by

personnel during the content analysis, task analysis, training task

analysis, and some video production phases. Therefore, If these costs are

used to estimate the partial cost of future production projects, care should

be exercised to Insure that the production process Is defined the same way

and that cost of additional steps not Included In these figures are added to

the estimates. The applicability of the data reflects the experience of the

production personnel. The number of labor hours required to produce

T

roubleshootIng Introductions, Troubleshooting Simulation lessons, and

Maintenance Simulation lessons are based on the fact that when these were

produced, the personnel were working at their highest productivity rate.

The personnel were familiar and experienced with the ADAPT language, the

MlcroTICCIT system and the coding templates themselves.

The complexity of the five templates varied and this affected the

number of hours that were required to produce lessons. The courseware

production costs are also a reflection of the technical nature of courseware

content, the amount of content, and the authoring language used to code the

courseware. These cost figures could provide a basis for gauging the cost

of producing courseware that Is of similar complexity and that teaches

similar troubleshooting and maintenance skills. If the material has greater

technical complexity, then It Is likely that more time would be required of

all four labor categories. The technical difficulty of the content

Influences the amount of time necessary for the ID to understand the

material. Complex content requires more time for the ID to read the

reference materials and more time to consult with SMEs. If the technical

material Is of greater complexity, then more CD time would be required to

enter the data Items. More data Items require more time to review and debug

the material. The degree of complexity also would determine the number and

kinds of visual Images required. More video would require more ID time,

while more graphics would require more GS time. The authoring language

affects the hours required for entering program code, debugging, processing

code, entering and editing graphics. The use of an authoring language other

than ADAPT would alter the number of hours required to code a lesson or

create a graphic. For example, some authoring languages might facilitate

graphics production but be unable to produce the kinds of Interactions that

ADAPT can, while others might facilitate revisions. Each authoring language

has Its strengths and weaknesses and It Is difficult to quantify how the

authoring language affects production costs without further st'dy.

Sununar1!

The second part of the report contains descriptions of the five lesson

templates that compose most of the MTP-RC lessons and descriptions of the

responsibilities of the Instructional Designers, Courseware Developers,

Subject Matter Expert, and Graphics Speclaxlsts who produced the lessons.

The steps Included In the courseware production process were listed and

described. The cost. In terms of labor per labor category, showed how many

hours were required to produce lessons based on each of the five templates.

n

F*
Troubleshooting Simulations were the most labor Intensive lessons, closely

followed by Maintenance Simulations. Name, Locate, Function and Input,

Process, Output lessons were Intermediate In labor cost, while

Troubleshooting Introductions required the least amount of labor. Each

lesson template was analyzed to determine why the number of hours of labor

varied from template to template. The results of this analysis were

discussed. Finally, the Information extracted from the analysis was

generalized. The number of hours required by each labor category varied

from lesson template to lesson template depending upon the complexity of the

content, the amount of content required by the lesson, and the Instructional

strategy that determined the Interactions the template required. Complex

content, lengthy content, the high degree of Interactivity required to teach

the content, and the use of state of the art delivery techniques, all tended

to Increase the amount of labor to produce a lesson.

Additional courseware can be produced within the same range of hours as

reported In Table 6 provided that:

1. Similarly experienced production personnel are used.

2. The courseware treats content of similar complexity and length.

3. Courseware Is produced using the same or similar authoring

language, lesson templates, and delivery system.

The cost analysis of the MlcroTICCIT delivery system for MTP-RC was the

focus of the first part of this report. Different systems and

configurations of Hazeltlne's MlcroTICCIT computer systems were analyzed

with reference to total system hardware costs and system cost per

student-hour. The second part of the report analyzed courseware production

costs. The report explained the number of labor hours required to develop

five types of lessons. The report examined the Influence of critical

variables In development time: the steps Included In the development

process, the complexity and quantity of the content, and the level of

Interaction required to support the Instructional strategy.

The hardware delivery systems cost analysis provides guidance for

planners considering the selection and fielding of MlcroTICCIT

computer-based delivery systems at other reserve training sites. Courseware

development cost analysis provides planners with guidance for predicting the

cost of developing additional courseware of a similar nature to that

developed for MTP-RC. Further research to study and quantify the factors

which Influence CBT courseware production Is planned for Phase III of the

Modi^l Training Program for Reserve Component Units.

34

-• -•. ■■. -, •>

'-•. >.' • .' . -•. •

..- ..--..•
REFERENCES

Htllelsohn, M., J. (1984). Beru-hmarklng authoring systems. Journal of

Conputei—Based Instruction, 11.

Marco, R. \., Begg, J., Israelite, L. & Bernstein, K. (1985). Instruct tonal

technology report. Technical report In preparation for tne U, S, Army

Research Institute, Fort Knox, KY.

Orlansky, J., & String, J. (April, i979). Cost effectiveness of computei—based

Instruction In military tralnl ig. IDA P-1375, Arlington, VA. Institute

for Defense Analysis.

Stacy, E. W,, Jr. (1984). Impact of courseware engineering on design and

production of code. Paper presented at the Society of Applied Learning

Technology Conference on Simulation and Training Technology for Increased

Military Systems Effectiveness, Arlington, VA.

ti

T)

'1 '.'^
APPENDIX A

HAZELTINE MICROTICCIT SYSTEMS AND CATALOG PRICES

CATALOG PRICES*

Desktop with 71 megabytes storage 43,000

Desktop with 142 megabytes storage 60,000

System II with 50 megabytes storage 55,000

System II with 100 megabytes storage 71,000

System III with 147 megabytes storage 95,000

System III with 294 megabytes storage 125,000

4,100

5,300

6,000

8,000

9,500

12,000

W0RKSTATIONS*

ACQUISITION

COST

ANNUAL

WARRANTY

3 workstations

5 workstations

10 workstations

30,000 3,000

50,000 5,000

100,000 10,000

* These prices were quoted as of December 1985.

Most prices were rounded to nearest $1,000 by Hazeltlne,

+ Workstation prices Include videodisc players, network controller,

hubs and nodes.

A-l

-• '•''-' "-" '•", -". -"• ',"

<_Lif_LM-L^i*«iii*i'iA;^i'rtnaiV - • -" - - - - - - -:■■ - ^ - -
. .-

0 TICCIT

WORKSTATION

coo

»-•.T1CCIT |

llllllllll V'»X/yM

^^^^m

BorfcJtition Characterittiea:

The MicroTICCIT workitition it • aultipurpoie itatioo deiigned for utt by

• tudentt, initmctora, countwarc devclopen and training adminiitraton on

all MicroTICCIT ayatca configurations. It conaiata of an IBM Peraonal

Computer, a Sony color aonitor. a detached keyboard with an integral

imtructiooal keypad and special function keys, a lightpen, and a high-speed

connunications link to the aysten host processor. The display aonitor

supports color presentations of computer-generated text and graphics, as well

as full-color still-frame and motion sequences from videodisc. Onder author

control, foreground and background colors csn be eelccted from a palette of

4096 colors for use in the creation of each frame of conputcr-gencratcd text

and graphics. In addition, an option is availablt to allow computer-generated

displays to overlay videodisc frames to form a combined display on any

MicroTICCIT workstation. This espability allows unprecedented freedom for

courseware authors to take advantage of the special attributes of each medium

for training.

A-2

*■■'- •- '■■ - .^ - -• - - --^ -•-' --r- -

f'- - -

•. . - .•. .•.. .
'.V

BgrkjtatioB IlMMt*:

o Aa IBM Pcrtoiul Computer (PC) vitb 256 Kbytci KAN. Standard vorkttationi

will b« provided without diakctce drivct in the PC. A tpccially prograaaed

lead-Only Meaory (ROM) chip it included to control the workatation'a

interface to tht Network Contrcller workatation on each ayatea.

o A iaaeltioe Model 430 diaplay interface controlling • Sony 12" (diagonal)

full-color aonitor. The picture eleaent (piiel) reeolution ia 430B a

204V. The acrcen rcfrcabct at aisty fraaet per aecond to provide a atable

display auitable for extended viewing without eye fatigue. The diaplay

aupporta preaentatioot of coaputcr-generated text and graphica at well at

ttill-fraae and aotion video.

o A detached IBM-ttyle keyboard with an expanded keyaet. The keytet includet

a atandard upper- and lower-caae alphanuaeric keypad, an inatructional

keypad for direct acceaa to OBJECTIVE, RULE, EXAMPLE, PRACTICE, and HELP

coaponentt, an editing and curtor control keypad for acceaa to word

procetting functiont tuch at INSERT and ERASE, and eleven programmable

tpecial function keyt which can be uaed to define up to thirty-three

tpecial cbaractert or tyabolt for unique training applicationt.

o A light pec to tupport intcractiont via acreen touching

o A coaaunicationt interface board to link the workatation to the tytten hott

procettor. The link it ettablithed via an ARCNET Local Area Network (LAM)

coaaunication ayttea. ARCNET utet a tingle coaxial cable to connect

cluttert of workttationt to the central boat. Dittancet up to 2000 feet

between cluttert can be tupported with additional cabling and a maximum

diatance of four ailea can be accomodated with the addition of optional

active repeatert.

Optioaa:

o One or two 360Kbyte ditkettc drivet can be aided to the IBM PC.

o A videoditc overlay option can be added to any MicroTICCIT workatation to

allow tiaultaneout ditplay of computer-generated text and graphica and NTSC

video combined on a tingle ditplay. The videodiac image, which can include

either ttill-fraae or aotion tequencet, can be preaented on all or any part

of the ditplay tcreen and can be ovcrlayed by coaputer-generated text or

grapbict to aingle pixel retolution. The option includet a Sony later

videoditc player and an overlay interface board that retidet in the

workttttion PC.

o A raodom-acceaa EIS audio tyttem can be interfaced to the MicroTICCIT

workttttion to provide voice-qutlity audio aettagea in coordination with

vitual pretentationt. Audio can be uaed to aupplement computer-generated

at well at video ditplaya. The audio can be freely intermingled with the

training aateriala aa deaired by the courteware authort.

o A graphica authoring tupport package to incrcaae the productivity of the

graphica tpecialiat. Thit includet a three button aoute and 256 Mbytes of

additional RAN.

iNVinONMEMT/U. •»EBFICATOMt

WorVstation Specifications SSSSXST mmmV^JSüm

/ OperaimgAimud« upiotOOOFwi

CUCTKICU. SPfCIFICATONS

jj A.CVOW.B. Wm rMYSICAlWfCinCAT»»«

^S^ KAmnNECOnpoMTiON.HESTON.vmaiNiA«»i(rM)MMa« mmmmtmfmm

• - . -' --.•.• ."•. •.■■.-.-. .• • .••..• .-.•• > . -. -.• .••. -"'-.•- .••,-• .•-V'V-^

- •.•-. .'-.•. ' .. .•. •,"-'■■.•', ."-. . ." .• ."-.•, ." .•.•.".' -" ■■. •.•."'> .•. • ..'•'." .

. m. . . •'. -. '. V» • - r . ,'.. -... .-". ^, -/ . ^ - ^ . . " ."-V- ."- «'^ M', *'- ..'. V- ' ti > »*- » - **iC* £ . «'.-,•.
TICCIT

System I Configuration

mmiin

- •

J

System Characteristics:

The System II configuration is designed to optimize courseware development and delivery for medium-scale

computer-ba^ed training efforts. The system will support up to forty MicroTICCIT'vworkstations, each of which can

be used for student, instructor, or autnor functions. The full MicroTICCIT software package, including all courseware

development, graphics, and management software, is supported by System II. In addition, any courseware developed

on System II may be used on any other MicroTICCIT system configuration without modification.

System Elements:

• Data General microECUPSE'** S/20 host processor with 256 Kbytes RAM

• 50 Mbytes of fixed disk storage for on-line system software, student records, and courseware

• 15 4 Mbyte cartridge tape drive for loading or dumping data from fixed disk

• Data General ooerator console CRT for performing system startup, shutdown, and backup routines

• ARCNET Local Area Network communications interface which allows workstations to be located up to 2000 feet

from the host processor and can be extended to four miles with optional active repeaters

A modem for interfacing to Hazeltmes MicroTICCIT maintenance service network

A-4

'-'-^'•-'-"-•'- -•

.-.-.^... ^^...f.A^....-. 1 1 T r-,..,,. .-... ^:.<...:^..-:.-^v>.-^.^..-J
Options:

• Printing console In lieu of the standard operator console CRT to support hard-copy printing of student management data

• Fixed disk capacity expanded to 100 Mbytes with the addition of another disk unit

• 25 Mbyte strtammg tape drive in lieu of the cartridge tape drive

System II Specifications

ELECTRICAL SPECIFICATIONS*

A C Voltage

Line Ffequancy.

Power Consumpncv

'Also available tof 50 H; power standards

•04 to 127 VAC

eOHerli, 5 Hertz

15 Amperes

ENVIRONMENTAL SPECIFICATIONS

Temperature Range

Humtdity Range

Operating Altitude

PHYSICAL SPECIFICATIONS

Dimensions

Weignt

HOST PROCESSOR

Type

65*10 75-F(90'ma«)

40 to SO11 s (Non condensing)

Up to 8000 Feel

69"L x 32"W x M'H (Desk Unit)

378 Pounds (172 Kilograms)

WordLengtn

Memory CycieTime

Memory

Data General microECLIPSE S/20

w/Burst Multiplexer

16 bits

500 nanoseconds

256 KB. byte parity checking standard

DISK STORAGE (STANDARD)

Fixed Disk

Capacity

Head Positioning Time

Rotational Latency

Data Transfer Rale

TAPE STORAGE

Cartridge Tape

Capacity.

Data Transfer Rate.

Speed

Density

LOCAL AREA NETWORK (LAN)

Type.

Speed.

Cable Length

Error Checking

Error Recovery

Maximum Terminals per System

50 7 MB. "Winchester" type

60 Milliseconds/Average

12 5 Milliseconds. Averdge

971 000 Bytes per second

(Disk to Controller)

881 000 Bytes pur second

(Controller to Memory)

15 4 MB/450 loot tape

48,000 Bytes per second

60 inches per second

6400 Bits per inch

ARCNET

2 3 Mi, is per second

Maximum 2000 teet

16 Bit Cyclic Redundancy Check

iCHCi on data and addresses

Automatic Retransmission

40 Recommended

:•:

The materials contained herein are summary in nature, subject to change, and intended tor general information only

^||"jj« HAZELTINE CORPORATION, RESTON VIRGINIA 22091 (703) 620-6800

A-5

PHINItO IN USA 4/84 80

--.-. r.

i, -L.^.- ,v>jiVy;L-\/'-'.'-VvV'.iv»,W'vV

->';
DOTTICCU

TM

System III Configuration

i

System Characteristics:

The System III configurations provide a family of systems designed to optimize courseware development and delivery

for large-scale computer based training efforts Each configuration will support up to sixty-four MicroTICCIT'

worKstations which can be used for either student, instructor, or author functions The full MicroTICCIT software

package, including all courseware development, graphics, and management software, is supported by the System III

configurations In addition, any courseware developed on a System III configuration may be used on any other

MicroTICCIT system configuration without modification

System III Elements:

• Data General microECLIPSE S/120 host with 256 Kbytes RAM

• 15 Mbytes of fixed disk storage for rapid "swapping" of data from disk to processor

• Additional fixed disks for on-line storage of software, student records, and courseware

System lll-A 50 Mbytes

System lll-B, 73 Mbytes

System III C 147 Mbytes

• Tape drive for loading or dumping data from the fixed disks

System III A, 15 4 Mbyte Cartridge Tape

System III B, 25 Mbyte Streaming Tape

System lll-C: 25 Mbyte Streaming Tape

• Data General operator console CRT for performing system startup, shutdown, and backup routines

• ARCNET Local Area Network communications interface, which allows workstations to be located up to 2000 feet

from the host processor and can be extended to four miles with optional active repeaters

• A modem for interfacing to Hazeltme's MicroTICCIT maintenance service network

ht

•*

X-L-f

K-6

*

L - .\-- . ."L ^ •- . u*,m^i ^\ rnjij^.: -.^ . ^ * n» >. - * - . - »jrfVii - < - V_ tr* T- V^. <
Options:

• Printing console In lieu of the standard operator console CRT to support hard-copy printing of student management data

• High-speed dot matrix printer to support hard-copy printouts of software and courseware files

• Graphics digitizer to facilitate the entry of large numbers of graphics into the courseware database

• Device control interfaces to support control of custom devices, including random-access audio cassettes and 3-D

panels

System III Specifications

ELECTRICAL SPECIFICATIONS* DISK STORAGE (STANDARD)

AC Vollage 104IO127VAC

Line Frequency 60Herl2, i 5 Hertz

Powef Consumplion 25 Amperes

'Also avdiUbie lor SO Hz power standards

ENVIRONMENTAL SPECIFICATIONS

Temperature Range

Humidity Range

Operating Altitude

PHYSICAL SPECIFICATIONS

Dimensions

Weignt

HOST PROCESSOR

65* ^S" F|90* max I

40 to 80**o (Non-condensing)

Up to 8000 Feel

iVH «24'Wx30,D(Racki

460 Pounds (209 Kilogramsi

Type

Word Lengtn

Memory Cycle Time

Memory

Data General microECLIPSE S/120

t6Dils

500 nanoseconds

256KB ERCC 122 Bill

LOCAL AREA NETWORK (LAN)

Type

Speed

CaPie Lengtn

Error Checking

Error Recovery

Manmum Terminals per Syslem

ARCNET

2 5 Moils Per Second

Maximum 2000 Feel

16 Bil Cyclic Redundancy ChecK

(CRCl on Data and Addresses

Automatic Retransmission

64 Recommended

Swapping DISK

Capacity

Head Positioning Time

Rotational Latency

Average Access Time

OataTranster Rate.

Fixed Disk

Syslem lll-A

Capacity

Head Positioning Time.

Rotational Latency

Data Transler Rale

System III B/System MIC

Capacity

Head Positioning Time

Rotalionai Latency

Data Transler Rale

TAPE STORAGE

System III A

Capacity

Data Transler Rale

Speed

Density

Syslem III 8 and System III C

Capacity

Data Transfer Rale.

Speed

Density

15MB. Wincnesler Type

66 Milliseconds/Average

9 5 Milliseconds'Average

75 5 Milliseconds

266 000 Bytes Per Second

(Controller to Memory)

50 7 MB, "Winchester" type

60 Milliseconds'Average

12 5 Viliseconds/Average

242 000 Bytes per second

(Controller to Memory!

73 7/147 4 MB, "Winchester" Type

30 Milliseconds/Average

8 33 Milliseconds/Average

600 000 B> les per second

(Contro ler loMemoryi

15 4 MB Cartridge Tape

48 000 Bytäsper second

60 Inches per second

6400 Bits per inch

26 7 MB Slreanmg Tape

48.000 Bytes per second

30 Inches per second

1600 Bits per inch

The materials contained herein are summary tn nature. suDiecl lo change, and intended lor general inlormation only

Ha«ltine HAZELTINE CORPORATION. RESTON VIRGINIA 22091 (703) 620-6800 PRINTED IN USA 4(84 80

A-7

^V-y._--.--

^•-.'•-»jj, .-^AW-J

-• - -'
ooroTICCir

NETWORK CONTROLLER

WORKSTATION

— ^~ »

| »OWY t OOP

fc-.-.TlC&T

llllllllll

U T „T I T I 1 » T T 1 J,J T .

etvork Costrollit Sorkatatiom Characteriatlet:

Tbc MicroTICCIT network controller workttation eontrolt all interectiom

between between the MicroTICCIT boat proccctor and tbe ctaodard MicroTICCIT

workitation* on each ayatra. A bigh-ipeed Direct Memory Acceti (DMA) interface

ia used to citablieh tbe link between tbe boat proceaaor and tbe network

controller workatation. Although the network controller workttation aupportt

the full aet of ctudent, inatructor, couraeware development and training

adniniatrition functiona provided by all atandard MicroTICCIT workatationa,

the DMA link aakei it ncceeaary for tbe network controller workatation to be

located within ten (10) feet of tbe MicroTICCIT boat proceaaor.

Tbe network controller workatation conaiata 6f an IBM Peraonal Computer with

one (1) 360K byte diakette drive, a Sony color aonitor, a detached keyboard

with an integral inatructional keypad and apecial function keya, a lightpen,

and a communicatiosi link to all of tbe atandard workatationa on the ayitem.

Tbe diaplay monitor aupporta color prcacntationa of computer-generated text

and grapbiei, ac well «a full-color atill-fraae and motion aaquencea from

Tideodiac. In addition, an option ia available to allow computer-generated

diaplaya to overlay videodiac framec to form a combined diaplay.

A-8
VorUtatioB Ilawnti:

An IBM Perioml Computer (PC) with 2S6 Kbyte* RAM and on« (I) 360 Kbyte

diikette drive.

o A Betel tine-Model 430 diiplay interface controlling a Sony 12" (diagonal)

full-color aonitor. The picture claaent (piiel) resolution is 430B i

204V, The screen refreshes at sixty frames per second to provide a stable

display suitable for extend'd viewing without eye fatigue. The display

aupporta presentstions of conputer-generated c«xt and graphica as well as

still-frame and motion video.

o A detached IBM-style keyboard with an expanded keyset. The keyset includes

a standard ipper- and lower-case alphanumeric keypad, an instructional

keypad for direct access to OBJECTIVE, RULE, EXAMPLE. PRACTICE, and HELP

components, an editing and cursor control keypad for access to word

processing functions such ss INSERT and ERASE, and eleven programmable

apecial function keys which csn be used to define up to thirty-three

specisl chsracters or symbols for unique trsining spplications.

o A ligbtpen to support interactions via screen touching

o A high-speed conunicatioiis interface board to link the network controller

workstation to the system host processor.

o i network communications interface board to link the network controller

workstation to the rest of the workststions on the system network. The

link is estsblished via en ARCNET Local Area Network (LAN) communication

aystem.

Oftiomt:

o An additional 360 Kbyte diskette drive can be added to the IBM PC.

o A videodisc overlay option can be added to allow simultaneoua display of

computer-generated text and graphics snd NTSC video combined on a single

display. The videodisc image, which can include either still-frame or

motion aequencea, can be presented on sll or any part of the displsy screen

and can be overlayed by computer-generated text or graphica to aingle pixel

resolution. The option includes a Sony laser videodisc plsyer and an

overlay interface board that reaides in the workatation PC.

o A random-acceaa EIS audio system can be interfaced to the controller

workstation to provide voice-quality audio messages in coordination with

visusl presentations. Audio can be uaed to supplement computer-generated

at well aa video displaya. The audio can be freely intermingled with tie

rraininf materials ss desired by the coursewsre authora.

o An AXIOM screen-image printer can be added to the workatation to provide

black-and-white hard-copy printouts of any computer-generated text or

graphic displsy.

ENVIRONMEMT/U. tPECIFCATIONt

Workstition SpecllicaUons SSSÄt-" v**'"* *?* ^.

KLICnUCAL »MClFICATONt

ZSmiSSm 'mnmmm

»HVSICAL VCCIFICATIONt

Acvense« i»vo«i

POTConsum*«* 4»Walls 1 Ws7»f TOPoüna.

MAZELTIME COMOMTION. AESTON. VmOINlA 22M1 (TM) I2MM0 «•NTlD ••«•••

A-9

'-V-'»"-'.-• '-'-"-'-'■■•." •.-. . -1. -. ." - -' - ' - \ " - - '<- l'«-'.^.-.'.^ I'AÜ \',..-\*'\ •"> •"i--".-•,% <. C -.". .

,

. -.'. -•. I*. .' - - .' .- '.•.-. i. i
yi

JICCIT

VIDEO OVERLAY OPTION

The videodiic overlay option allovi iiaultaneoui display of coaputer-generated

text, graphic», and NTSC video from videodisc conbiacd on a single MicroTICCI"

workstation display. The video, which can include either still-frame or

motion sequences, can be presented on all or any part of the diaplay screen

and can be overlayed by computer-generated test or graphics to single picture

element (pixel) rcaolution.

The videodisc overlay option includea a Sony laaer videodisc player, a

speciaHy-designed video overlay interface board that resides in the

workstation IBM PC, and all necessary cablea.

Each videodisc option is dedicated to a aingle terminal and cannot be ahared.

The option may be ordered for one, a few, or all MicroTICCIT workatationa on

any MicroTICCIT system.

A-10
.- . •

DWTICCIT

8

SERIES 10

Li

Oparator Contol« Cartridge Power

Bard

Tape Drhra Dlak DQ-10 Supply

CPU

Syttaa Charactcriitica:

The MicroTICCIT Series 10 provide« a cost-effective configuration for

delivering courseware developed on larger MicroTICCIT systems for snail-scale

training applications. The system will support up to ten MicroTICCIT

workstations, each of which can he used for student, instructor, and training

management functiona. The Series 10 System is ideally auited for thoae

applications that require courseware developed at a central authoring facility

to be delivered to one or more satellite facilities. All software required

to support the efficient delivery of coursewsre and the collection of atudent

progress data is provided with the system. The Series 10 system is not

designed to support courseware development functions.

The MicroTICCIT Series 10 is available in two atandard configurations differing

only in the amount of disk storage provided:

Model 38 provides 38 Mbyte of storage

Model 71 providea 71 Mbyte of atorage

Syatea Eleaenta:

o Data General DG-10 desktop processor with 768 Kbytes RAM

o 38 Mbyte fixed diak (Model 38) or 71 Mbyte fixed diak (Model 71) for

on-line storage of system software, student records, and couraeware

A-11

ylv^^^ly^IÄ:-^^-.^^
o 15.* Mbyte cartridge tape drive for loading or duaping data fro« the fixed

diak

o 368 Kbyte floppy ditkette drive for runnioi, ayaten diagooatica

o Data General operator console CRT for performing yiteo itartup, ahutdovn,

and backup routines

o ARCNET Local Area Network (LAN) communication» interface to link HicroTICCIT

workstations to the Desktop System bolt proceaaor

c A modem for interfacing to Hazeltine'i HicroTICCIT maintenance service

network

Optiona:

o Printing console in lieu of the standard operator console CRT to support

off-line printing of student management data

o Add-on 38 Kbyte fixed disk (Model 38) or 71 Mbyte fixed disk (Model 71) to

double the system's on-line storage capacity

•..-

&T1 lAZELTIiri COtPOtlTIOI. USTOi TUCHIl 22091 (703) M0-4I00

A-12

Printed in USA
APPENDIX B

FORMULAS FOR COMPUTING SYSTEM COST PER STUDENT"-HOUR

SYSTEM ACQUISITION PRICE - [(HOST ACQUISITION PRICE + WORKSTATION ACQUISITION

PRICE * NUMBER WORKSTATIONS)]

TOTAL SYSTEM COST = [((HOST ACQUISITION PRICE * (CPU ANNUAL MAINTENANCE RATE *

YEARS)) + ((COST OF WORKSTATIONS +

(ANNUAL WORKSTATION MAINTENANCE RATE «WORKSTATIONS * YEARS));

SYSTEM COST PER WORKSTATION = TOTAL SYSTEM COST/ NUMBER OF WORKSTATIONS

ANNUAL SYSTEM COST PER WORKSTATION » SYSTEM COST PER WORKSTATION / NUMBER

OF YEARS

TOTAL WORKSTATION HOURS PER YEAR = (NUMBER OF SOLDIERS * NUMBER OF DRILL

PERIODS PER YEAR * HOURS PER DRILL PERIOD)

ANNUAL WORKSTATION UTILIZATION RATE = TOTAL WORKSTATION HOURS PER YEAR /

NUMBER OF WORKSTATIONS

SYSTEM COST PER STUDENT HOUR = ANNUAL SYSTEM COST PER WORKSTATION /

ANNUAL WORKSTATION UTILIZATION RATE

B-l

:••>:-^
APPENDIX C

STRUCTURE OF MTP-RC COURSEWARE

The MTP-RC courseware Is structured In such a way as to be compatible

with the MicroTICCIT Computer-Based Management System, Each course — one

for each MOS — Is divided into 3 instructional categories:

1. Units - One for each tank system covered in the course.

2. Lessons - Two for each Unit within a course.

I. Segments

lesson.

One for each instructional objective being taught within a

Figure A-l shows the flow of Instruction within an MOS course. There

is, In addition, one course which provides Ml skills sustainraent training.

The content of this course and a general description of the Instructional

structure of the MOS courses follows.

The Ml Skills Sustainraent Training Course Is divided Into two units.

Unit One serves as an Introduction to the Ml tank, the MicroTICCIT CBT

system, and the structure and content of the MTP-RC courseware. Unit One

also Includes Instruction on how to use the MicroTICCIT student workstation

to complete the courseware.

Unit Two Is designed to provide RC soldiers with the skills and

knowledge that are prerequisite to successfully complete the MTP-RC

courseware. For those who have completed Level 1 training, this unit can

be considered as sustainraent training. For those who have not acquired the

necessary skills. Unit Two will serve as prerequisite skills training.

Topics covered in this unit Include use of technical manuals, use of the

STE-M1, DSESTS, and Digital Multimeter.

As Figure C-l shows, Units comprise the tank systeras covered in a

course. In each unit there are two lessons: Principles of Operations and

Troubleshooting/Maintenance. Each lesson has one to three segraents.

Instructional Interactions are defined at the segment level. The design of

each of these segments was based on one of five templates.

Principle of Operations segments are based on either the NLF and IPO

templates. The lessons are designed to provide student with an

understanding of the purpose and function of a tank system, its component

parts, and Its general operation. Such information should assist the

student In troubleshooting selected system symptoms. This lesson has one

or two types of segments, depending on the complexity of the system: Name,

Location, Function (NLF) ind Input, Process, Output (IPO).

C-l
w

x\

•_—

00B

•HU:

B

B —*

N

0

—3

•f-4

X •H

V«

W

V 4J

M -.fl

u

— (0 •^ •^

- IM

O

u

-3

-

U

u

b

- 0)J 1/

h

u

c

XX

^i '—

s UJ UJ

e

e .—. " —1

—

—

a

«

r.

u

u

3

a

— 1-1 'M

4J ki ij ->

u

a—

au

Ri C3 Ifl >. -- 9 u kl

C MS

en

og

<uuou

^-rac

c eo o B u c 3 Q

-a£

0«

u«

— —* B -c

-.' c

t/5

ch

0.

B0

•H

4-1

!8UV—-

4Jc0—/.u3

e

0)

_

E1

9E

ww tr.

i

D

u

l-> in F: tn

U CA

u

i ><

3

—

UM

HH

9 »1

P

0

0

>

b

IIu

u

«

V

^

—u •H -3 ^' J C u h s 1^ "1 P II •H M w W- m B 9 (V > h •.-j B M i b £3 B 3 —. n »- h J ^ A

tfl u

J

cr j—i M M

VWX

8

in

m

«m

-.T ^T

c vC

I0—

«i8•H

C-2

'--' -"*- v'^'. ..% . .• w. .\:\:\:.
*A

Name, Locate Function Template

The NLF template defines the interactions contained within NLF

segments. The primary purpose of a Name, Location, Function segment is to

teach the name, functional location and function of each of the important

components of a single tank system. The student learns the function of the

entire system, how the components work together in the system, and what

happens if the system Is not working correctly. In this segment, the

student is presented with a functional block diagram of the entire system.

This diagram shows the functional relationship among the components in the

system, so the student learns the functional position of the component in

relation to the others. In a few systems, such as the engine, the student

also learns the actual physical location. After selecting on individual

component, the students see a video or graphic image of the component and a

brief description of its function. An system the components are stulo In

this manner. The student then demonstrates he has learned by completing a

practice exercise in which he identifies the name, location and function of

each component. At the end of the segment, the student reviews what he has

learned.

The Input, Process, Output Template

The Input, Process, Output (IPO) template structures the set of

possible interactions within IPO segments, AN IPO segment is designed to

accomplish two specific goals. First, information about each system part

is presented in greater depth than in the NLF segment. In the IPO segment,

soldiers learn:

1. Those elements (electricity, data, fuel) that come into a part and

the sources and those elements.

2. What happens to those elements while in the part.

3. What elements (electricity, data) are output by the part and the

destination of those elements.

For example, a computer might receive data from three electronic

sensors. These data comprise the INPUT. Then, the computer performs a

clearly defined set of mathematical operations on those data. The results

are then evaluated. This represents the PROCESS, The computer sends

signals to three other tank parts. The signals represent the OUTPUT, By

studying the NLF and IPO for any part, the soldier learns both what a part

does functionally, and how this function Is accomplished,

A second, and no less important, goal of the IPO segment is to

reinforce the concept of the tank as an Integrated system. While the NLF

segment focuses on one system or subsystem, the IPO segment shows the

relationship among several systems. The reason, of course, is that input

and output is not limited to a specific system; it is the input and output

C-3

. •.- v.. •.■■■■ i'.- yv - -. - - " • ."•-.• .■■•"•- ;:-•■■• ;•-•: • viv^; ■■.-''. -•" ••>. : •.. •, --: ■■■■■->;.-; ;-•. ••>! ■■
of parts, subsystems, and systems that shows the integration of all tank

systems.

The second lesson In each unit Is Troubleshooting/Maintenance, which

uses part-task simulations of troubleshooting or maintenance tasks.

Troubleshooting lessons are defined by the Troubleshooting Simulation

template and Maintenance lessons are defined by the Malntenace Simulation

template. There are two segments within ea^h Troubleshooting/Maintenance

lesson: Introduction to Troubleshooting and Troubleshooting/Maintenance

Exercises. For the 45E, b3E and 45K, the simulations are mainly electronic

troubleshooting and for the 63H, mainly mechanical maintenance procedures.

This content reflects the principal skill level 2 critical tasks for the

respective MOSs.

Troubleshoot'ig Introductions Template

The first segment. Introduction to Troubleshooting, provides a

transition between the Principles of Operation segments and the simulations

the student will perform. The Troubleshooting Introduction template Is

composed of two sections: Symptom Introduction and How to do

Troubleshooting.

In the Symptom Introduction, the system diagram presented In the

Principles of Operation lesson serves as the foundation on which

troubleshooting knowledge Is built. All of the components which could not

be responsible for the symptom are deleted from the diagram, and components

outside the system which could cause the symptom are added. This provides

the student with the "feasible set" of all possible causes of the fault.

The student then learns how each component could cause the fault and what

tests will be performed in order to Isolate the fault. The Symptom

Introduction also Includes a review of the rules that must be applied while

completing troubleshooting simulations.

The troubleshooting approach presented In the Introduction models the

cognitive processes of experienced troubleshooters and can help students

develop the ability to locate system components that could a fault. The

approach may lead some soldiers to develop skills and knowledge necessary

for becoming superior techicians. For other students, the introduction

will provide understanding off why specific tasks are being performed and

will Illustrate the relationship between symptoms that occur within the

same system.

The second section of this segment, called "How to do

Troubleshooting", Is the same in every lesson. Its purpose Is to teach the

student how to use the troubleshooting simulation it is always available

for students In need of review. This section consists of a short section

which explains how Guided Demonstrations and Practical Exercises work, and

two sections which explain how the student will Interact with the Technical

Manuals (TMs) and the computer.

C-4

. •-■■ v..-
t^

Troubleshooting and Maintenance Simulation Templates

The Troubleshooting Simulation and Maintenance Simulation templates

define the set of possible interactions within the troubleshooting and

maintenance exercise segments. These segments generally consist of three

sections: Guided Demonstration, Practical Exercise 1 and Practical

Exercise 2, All exercises are based on the Ml TMs. The student is

required to use the appropriate TM as a constant job did when performing

troubleshooting actions. MicroTICCIT provides the tank and test equipment,

as well as help, prompts, and explanations. Maintenance Simulation lessons

are primarily Intended for 63Hs who are responsible for repairing the

engine and the transmission. The actual performance of troubleshooting

these systems is heavily dependent on the soldier's visual Inspection of

the equipment. Because actual equipment is unavailable for training, the

maintenance lessons are designed to allow the soldier to visually Inspect

video images of the systems to Identify the fault responsible for the

symptom. In addition to familiarizing the soldiers with the equipment and

how to manipulate it, maintenance lessons teach the soldier specific

procedures for removing, repairing, and installing engine and transmission

components.

The Guided Demonstrations (GDs) and Practical Exercises (PEs) cover

the same symptom but follow different paths through the troubleshooting

procedure and lead to different faults. This approach enables the student

to repeat some steps within a procedure as well as performing new steps.

The goal of the troubleshooting exercises is for the student to simulate

troubleshooting the fault symptom using the TM as a job aid. The student

is not expected to memorize the steps required to troubleshoot a particular

symptom, but rather is expected to understand the use of the TMs for

troubleshooting symptoms.

V'.>.

The design of the template defines the types of interactions that are

permitted within the lesson. The following section describes the main

Interactions permitted by the templates.

The main part of the screen, called the work area, displays detailed

graphics showing the tank, system components, and test equipment. It is in

this area that the student locates his workplace, identifies the required

components, and performs tests and checks. Continuous guidance is provided

on a feedback line. Below this line are both action icons (such as remove,

replace, connect) and tool icons (such as the breakout box, cables,

adapters and jumper cables). The electronic troubleshooting procedures are

presented with computer generated graphics, while the maintenance

procedures (63H) primarily use videodisc images.

The student Is required to follow the TM word-by-v/ord, completing each

step by marking the appropriate component, actions, and tools. After each

action, the display changes in whatever way the student has indicated as

long as he continues to mark correctly. When the student makes a mistake,

he receives Immediate corrective feedback and cannot continue until the

correct step have been completed. The student continues on the path

C-5
through the procedure until he finds the fault. He then orders the

replacement of the faulty component and verifies that the problem Is

solved.

Throughout the Troubleshooting Simulation, two kinds of help available

to the student including:

1. What to do next if the student gets lost or does not know what

to do.

2. Reminders on how to use the simulation icons if the student

forgets.

C-6

tl:

A

.--.<. --. <-. «•. '. • . r.
APPENDIX D

WORK SUMMARY

(transcribed from Courseware Developer's handwrlnten log)

Name Jeff

Da t e Hour« Spcnent ('onpient'; (e.e. video, Hati entrv,

debus, finished)

5/13 3 E4.2.2.3 final error corrections

5/13 5 H6. 1.6.1 graphics and code debug

5/13 -4 H6.1.6. 1

N ti n H H

5/14 2.5 E6.5.2.1 final CD review

5/U 1.5 E6.5.2.2 final CD review

5/15 5.5 E6.5.2.2 M H ii

5/15 1 division meeting

5/15 1.5 E5.2.2.3 error corrections

5/16 1.5 — meeting-company

5/16 6.5 16.2.2.3 error corrections

5/17 4 E6.5.2.3 final CD review

5/17 •4 E4.2.2.3 error correction

5/20 3 H6.1.7.1 data entry and graphics

5/21 6 H6.1.7.1

n n M M

5/21 1 H6.2.1.4 video integration

5/22 8 H6.2.1.4 video integration and debug

5/23 2,5 E6.5.1.7 final CD review

5/23 2.5 E6.5.2.1 M ii II

5/23 3.0 E6.5.1.6 final CD review and learning environment specifications

5/24 1.0 tl H H II it II n "

5/24 4.0 E6.5.2.2 II i " "

5/24 3.0 E6.5.2.3 M tl It It II M M

5/28 1 E6.5.2.3 It It It It tt ,, ,,

* '-•'-'. . .'- _'• • -' •'. -'. •''..-V-'-

D-l

•^- '-.-•-- -•

- . •^ ;:::-:y.v^^:v:v^^:

--/-y>:,

Id

- > '-J
s

/I5

<

-£

Su

•<<OC 1

1F

-

"3oI

zo

o: < 1 1 H i 1 & w 1

6° I i

U7 1 b*

0^ b* 1

0° l>

boU

t>o 1 ^

Uo 1 bO— 1

6« b- ^

Oc

6» i <c

uo 1

WO 1 1 1^ 1 1 i — 7s 1 1 1 l>

(y 1 ! i u>

Ö* 1 1 t>-

0' ii &

^c 1 i 0° _ 1 ! 1 1 i ^ i 1 «^ 1 1! Hw ii i |

i

i

111

m t

-

5I

ta

III II

3--

3

5*M<

B

tt

^£

ss

Bc : nJ E

D-2

.•.