* i I I I l I

LEVEW

Technical Report 475

ADAPTIVE DECISION AIDING IN

COMPUTER-ASSISTED INSTRUCTION: ADAPTIVE

COMPUTERIZED TRAINING SYSTEM (ACTS)

Rosemarie Hopf-Weichel, Denis Purcell, Amos Freedy, and Luigi Lucaccini

Perceptronics, Inc.

MANPOWER AND EDUCATIONAL SYSTEMS TECHNICAL AREA

AU . 4 19 ~

U. S. Army

Research Institute for the Behavioral and Social Sciences

September 1980

Approved for public release; distribution unlimited.

- 81 8 14 022 .... .. , • ~ ~ ~ ~~~~~.... ..... - .. ...-.. . ... ' . _ '1
U. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

FRANKLIN A. HART

JOSEPH ZEIDNER Colonel, US Army

Technical Director Commander

Research accomplished under contract

to the Department of the Army

Perceptronics, Inc.

NOTICES

DISTRIBUTION Primary distribution of this report has bean made by ARI. Plea.R address correapondence concerning distribution of reports to U. S. Army Research Institute for the Behavioral and Social Science.,

ATTN: PERI*TP, 5001 Eisenhowr Avenue, Alexandria. Virginia 22333.

FINAL DISPOSITION This report may be destroyed when it is no longer '"eded. Plesaw do not return it to

the U. S. A rmy Research Institute for the Behavioral ad Social Sciences.

PigQZL The firndings in this report are not to be construed as an official Depaertent of the Army poeition, unless so desigrnated by other authorizeid documnent,.

am kI
] /"

Unclassi#feW ' ' /

SECURITY CLASSIFICATION OF THIS PAGE (When Dole Entered) " / R S O

REPORT DOCUMAENTATION PAGE BEFORE COMPL..ETING FORM

I. REPORT NUMBER 2. GOVT ACCEONNO. 3. RECIPIENT'S CATALOG NUMBER Technical Report 475%; ,

ADAPTIVE DECISION &IDING IN OMPUTER-ASSISTED *JECHNICAL REPOT

INSTRUCTION: ADAPTIVE COMPUTERIZED TRAINING )16 Sep 1078- 15 Sep 179 -- SYSTEM (ACTS). - 4- "w -=.-.:-: ft 1 ,,,-NPUMis-OF = , .MER r- -

-- -/, :1PATR-1076-79-11 U

7. AUTHOR(@) 8. CONTRACT OX GRANT NUMBERI4)

Rosemarie/'Hopf-Weichel, Denis/Purcell, /. MDAg03-78-C-2039 -

Amos/Freedy, a Luigi Lucaccini 9. PERFORMING ORGANIZATION NAME AND ADDRESS 10. PROGRAM ELEMENT. PROJECT. TASK

AREA & WORK UNIT NUMBERS

Perceptronics, Inc. /

6271 Variel Avenue _ . 2, 2762722P764 ) 

Woodland Hills, CA 91367

It. CONTROLLING OFFICE NAME AND ADDRESS 12. REPO.T1"ATE

U.S. Army Research Institute for the Behavioral Sep~m~r-i98O

and Social Sciences NUBE OF ACE

5001 Eisenhower Avenue, Alexandria, VA 22333 125 14. MONITORING AGENCY NAME & ADDRESS(it different from Controlling Office) IS. SECURITY CLASS. (of thl report)

-Unclassified

1 I |S1. DECLASSI FICATION/ DOWNGRADING

SCHEDULE

I. DISTRIBUTION - FNT (of thle Report)

Approved for public release; distribution unlimited.

17. DISTRIBUTION STATEMENT (of the abstract entered in Block 20, if different from Report)

1. SUPPLEMENTARY NOTES

This research was monitored technically by Bruce W. Knerr, Manpower and

Educational Systems Technical Area, U.S. Army Research Institute.

19. KEY WORDS (Continue on reveree side It necessary end Identify by block number)

Computer-assisted instruction Artificial Intelligence

Adaptive models Electronic circuits

Multi-attribute utility models Troubleshooting

De cision theory Military training

De~jsion making Task simulator

20. A3:*ACT (Cmautuw n pevree stf if neneev ald Identify by block number)

This report describes the results of the first year's effort of a 3-year

program to develop and evaluate a new Adaptive Computerized Training System

(ACTS). This combines principles of artificial intelligence, decision theory,

and adaptive techniques for teaching the procedures necessary to troubleshoot

I: electronic circuits. ACTS emphasizes the realistic simulation of maintenance

problems and focuses on enhancing the acquisition of decision-making skills

which underlie successful electronic troubleshooting performance. The ACTS

(Continued)

O I 1473 EDITION Of I NOV65 IS OBSOLETE Unclassified ^ " , '

SECURITY CLASSIFICATION OF THIS PAGE (W?,en Det Entered)
Uncl assi fied SECURITY CLASSIFICATION OF THIS PAGZ(WIh Data Bntere)

Item 20 (Continued)

tracks a student's diagnostic and decision value structures, compares them to

that of an expert, provides individualized feedback and help, and structures

subsequent learning experiences. It is designated to facilitate transfer of

these skills to an actual military environment. Preliminary studies indicate

that the ACTS does teach students to make more effective decisions troubleshooting electronic equipment. A large-scale study to further evaluate the

training potential of the ACTS is planned for the second year, as well as to

assess transfer to actual equipment following training with the ACTS.

Unclassified SECURITY CLASSIFICATION OF THIS PAGE(nen Deta Entered)
Technical Report 475

ADAPTIVE DECISION AIDING IN

COMPUTER-ASSISTED INSTRUCTION: ADAPTIVE

COMPUTERIZED TRAINING SYSTEM (ACTS)

Rosemarie Hopf-Weichel, Denis Purcell, Amos Freedy, and Luigi Lucaccini

Perceptronics, Inc.

Submitted by:

James D. Baker, Chief

MANPOWER AND EDUCATIONAL SYSTEMS TECHNICAL AREA

Approved by:

Edgar M. Johnson, Director

ORGANIZATIONS AND SYSTEMS

RESEARCH LABORATORY

U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES

5001 Eisenhower Avenue, Alexandria, Virginia 22333

Office, Deputy Chief of Staff for Personnel

Department of the Army

.5 September 1980

Army Project Number Computer-based

20762722A764 Instructional Systems

Apvoad for Dublic relem distribution unlimimd.

iii

, •~
ARI Research Reports and Technical Reports are intended for sponsors of

R&D tasks and for other research and military agencies. Any findings ready

for implementation at the time of publication are presented in the last part

of the Brief. Upon completion of a major phase of the task. formal recommendations for official action normally are conveyed to appropriate military

agencies by briefing or Disposition Form.

v

$ iv
FOREWORD

The Computer-Based Instructional Systems Team of the Army Research Institute for the Behavioral and Social Sciences (ARI) performs research and

development in areas of instructional technology that apply to military

training. A special research focus is the use of computer-based systems,

which can provide highly individualized training and can therefore improve

training effectiveness as well as reduce training costs and time.

This report describes the results of the first year of a 3-year effort

to develop and evaluate a new Adaptive Computerized Training System (ACTS).

The ACTS combines the principles of Artificial Intelligence, decision theory,

and adaptive Computer-Assisted Instruction to provide improved maintenance

training. In order to accomplish this research, ARI's resources were augmented by contract with Perceptronics, Inc., an organization selected as

having unique capabilities for research and development in this area.

The research effort is responsive to the requirements of RDT&E Project

2Q762722A764, Training and Education, as described in the ARI FY 79 Personnel Performance and Training Program.

nical Director

v

t

i

& • • J

I A - .- q -- " ': :' i , ' I ,I ,
ADAPTIVE DECISION AIDING IN COMPUTER-ASSISTED INSTRUCTION: ADAPTIVE

COMPUTERIZED TRAINING SYSTEM (ACTS)

BRIEF

Requirement:

The Adaptive Computerized Training System (ACTS) is being developed to

provide generalizable diagnostic skills to maintenance trainees.

Current Army maintenance training is largely equipment-specific. The

student first learns the step-by-step procedures for locating a malfunction

in a specific item of equipment, then practices and is tested on the equipment itsplf. Skill thus learned does not transfer readily to other equipment. Also, equipment that could be used operationally is required for

training, instructors must spend time inserting malfunctions into equipment

instead of teaching, and students spend time assembling and disassembling

equipment that should be spent experiencing a variety of faults.

ACTS Training:

The ACTS applies principles of artificial intelligence, decision theory,

and adaptive computer-assisted instruction to Army maintenance training. The

student's task in ACTS training is to troubleshoot an item of equipment by

making various test measurements and replacing the malfunctioning part. ACTS

simulates the electronic malfunction, with no actual equipment required. It

also uses artificial intelligence techniques to develop mathematical models

of both the student and the expert performer. These models can serve as a

basis for evaluating student performance. Previous research has shown the

feasibility of the ACTS approach. This effort focused on improving ACTS software and courseware, and on initiating evaluation of the improved system.

The new software developed permits simultaneous use of the ACTS by multiple fied techniques students, simultaneous use of different items of equipment, and simpli- for modeling new items of equipment. Revised procedures for

modeling student and expert performance permit the presentation to the student of feedback which is based on a comparison of student and expert models.

A plan for the installation of the ACTS at the U.S. Army Signal Center at

Fort Gordon, Ga., was developed, which will permit the evaluation of the

ACTS in an operational training setting. Finally, a pilot experilent was

performed to de-bug ACTS courseware, software, and experimental procedures.

Utilization:

ACTS research and development will continue for an additional 2 years,

culminating in a cost and training effectiveness evaluation of the system

in an ongoing course of instruction at the U.S. Army Signal Center and Fort

Gordon. If successful, it is expected that the system will be implemented

in Army schools providing maintenance training.

vii pMXI %J PAGE UAm-Na( ni"D

9. "'-: .
TABLE OF CONTENTS

Page

1. OVERVIEW 1-1

1.1 Objectives of Phase I 1-1

1.2 Approach 1-2

1.3 Accomplishments 1-3

1.3.1 Development of New Software 1-3

1.3.2 Multi-Attribute Utility (MAU) Decision Model 1-4

1.3.3 Instructional Text 1-5

1.3.4 ACTS Transfer to Military Training Equipment 1-6

1.3.5 Experimental Study 1-6

1.3.6 Directions for the Future 1-7

2. THE ADAPTIVE COMPUTERIZED TRAINING SYSTEM 2-1

2.1 Overview 2-1

2.2 Background: CAI and Decision Making 2-2

2.2.1 Individualized Instruction 2-2

2.2.2 Adaptive Instruction 2-3

2.2.3 Adaptive Decision Modeling 2-4

2.2.4 Decision Models in Maintenance Training 2-6

2.3 ACTS System Description 2-7

2.3.1 Task Simulator 2-7

2.3.2 Student Decision Model 2-9

2.3.3 Instructor Decision Model 2-12

2.3.4 Instructional Logic 2-13

2.4 Hardware Configuration 2-15

2.5 Instructional Approach 2-19

* 2.5.1 Training Procedure 2-19

,4 2.5.2 Consideration and Help 2-23

2.5.3 Action Selection and Help 2-23

2.5.4 Evaluation (Feedback) Phase 2-25

2.6 Software 2-25

3. RESEARCH ISSUES 3-1

3.1 Objectives 3-1

ix

It FMIMMam o PA03 M .D r n

t.
TABLE OF CONTENTS (CONTINUED) _

Page

3.2 Subjects 3-1

3.3 Instructions 3-2

3.4 Sequencing 3-3

3.5 Results and Analysis 3-4

3.5.1 General Results 3-4

3.5.2 Expert's Model 3-6

3.5.3 Number of Measurements 3-8

3.5.4 Help Option 3-8

3.5.5 Utility Difference 3-10

3.5.6 Time Per Problem 3-16

3.5.7 Debriefing Questionnaire 3-19

3.6 Discussion 3-21

3.6.1 Learning Effects 3-21

3.6.2 Sequencing Effects 3-22

3.6.3 Use of Handout 3-23

3.6.4 Sliding Window 3-24

3.6.5 New Printout 3-24

4. REFERENCES 4-1

APPENDIX A -- STUDENT HANDOUT

APPENDIX B -- SAMPLE INSTRUCTIONAL SEQUENCES

x

Ilow

low -!A
1. OVERVIEW

1.1 Objectives of Phase I

This report describes the results of the first year's effort of a threeyear program to develop and evaluate a new Adaptive Computerized Training

System (ACTS). The overall effort combines the principles of artificial

intelligence, decision theory, and adaptive computer-assisted instruction

so as to result in improved training techniques for use with Army recruits

in the learning of electronic maintenance troubleshooting procedures.

In particular, ACTS is intended to focus on enhancing the acquisition of

decision-making skills which underlie successful electronic troubleshooting

performance. An important aspect of ACTS is the emphasis on realistic

simulation of maintenance problems during training so as to increase the

potential for transfer of training to field situations. ACTS' design

incorporates an adaptive computer program: (1) to track students' diagnostic and decision value structures for comparison to that of an expert,

(2) to develop appropriate individualized feedback, and (3) to structure

subsequent learning experiences.

Major objectives of Year 1 were:

(1) To develop new software which provides for simultaneous

use of ACTS by multiple students, which permits simultaneous

use of different circuits, which provides for interchangeable

4H circuit modules, and which is compatible with equipment and

computer systems available at the Fort Gordon Army Signal

School.

(2) To develop new courseware which includes a more powerful

decision model and improved instructional text, and which

2 ]-I
orovides for utility driven performance feedback and

variable problem presentation modes.

(3) To develop a plan for ACTS transfer to a military training

environment, namely the Army Signal School at Fort Gordon,

Georgia.

(4) To conduct experimental work in order to evaluate the

effectiveness of the new ACTS.

1.2 Approacn

Computer-assisted instruction continues to represent a promising avenue

both for research on the basic instructional processes and for the development and integration of artificial intelligence techniques to enhance

individualized instructiooal processes in a number of areas, including

training of decision-making and problem-solving capabilities. ACTS is a

particularly important example of such training technology. Utilizing

adaptive, computer-based techniques, the ACTS permits individualized

trainifig in an electronic maintenance environment. The system has been

successfully implemented in the laboratory and subjected to limited

testing.

A major portion of the first year's effort consisted of expanding the

system to accommodate multiple tasks and multiple students, by developing multiple circuit capability and improving the ACTS' software organization. As a result of previous experimental findings, the utility models

of the origindl ACTS were replaced by more efficient multi-attribute

utility (MAU) models and the ACTS was supplemented with production rules.

3%'A In addition, the existing software was modified and expanded to provide

for simultaneous use of multiple students and multiple circuits, as well

1-2
aS for interchanygeaL circuit modules. Evaluation of the ACTS is being

documented, and a prelimiiiry study was conducted to evaluate the training effectiveness of the ACTS and the generalizability of the results

obtained with the ACTS. The software is compatiblq with equipment in

use at the Fort Gordon Army Signal School.

I.3 Accomp Iishments.

The ,ccomolisr.:,Ients of the first year tasks focus on four areas. The

following is a suwiiary or the work accomplished in each area.

1.3.1 Develoument of New Software.

Mew AC. Oe.siqrn. A rit-w version of the ACTS has been designed coded and

tested. The newvi ACFS contains the following main features.

Multiple Stu(ents. The new design allows two or more students to use

the system simultdfl2ously. The students can run on fully independent

instructional sequences, subject only to memory constraints. Additionally, an experimenter can coninunicate with the ACTS during system execution via the teietype I/0 channel.

,Multiple Circuits. Each set of circuit modules is circuit specific.

That is, any circuit is fully represented by the circuit simulation

model, together with the instructional text module for that circuit.

Consequently, any student using the ACTS may interact with it Indepen'-

dently of whatever circuit has been modeled, subject only to secondary

storaye limitations. New circuits can be modeled by creating a circuit

simulation model and instructional text module specific to that circuit,

using a fixed structure. A separate document (Perceptronics' Report

PDCMDM-1076-79-7, 1979) describes the necessary steps for modeling new

ci rcui ts.

, 1-3
Parameterized Circuit Model. Each circuit has a simulation model assbcciated with it, consisting of an overlay with a fixed structure. Creating

a new circuit model involves filling in the specific parameter values,

such as measurement outcomes and the number of faults for the new circuit.

This is done by utilizing an existing circuit model as a guide. The

process requires only that the new circuit be at a comparable level of

complexity to those already modeled. Since all circuit models have

standardized structure and parameters, the development of new circuits

for the ACTS is relatively simple.

Variable Mode Problem Presentation. Problems are characterized in terms

of their difficulty level. Since it is assumed that the sequencing of

problems may have an effect on performance, there are two modes in which

the problems ar.e presented to the students. In the first mode, the

problems are presented randomly, regardless of their difficulty level;

in the second mode, the presentation sequence is fixed, beginning with

all problems having simple-to-locate faults, continuing in a graduated

mariner with problems having intermediate levels of difficulty, and ending with problems having faults which are very difficult to locate.

1.3.2 Multi-Attribute Utility (MAU) Decision Model. A multi-attribute

utility (MAU) decision model, exhibiting efficient decision-making behavior for troubloeshooting electronic circuits, has been developed. This

multi-attribute model is superior to earlier models because it can capture

circuit troubleshooting behavior independent of circuit type, thus providing a more generalized behavioral representation than was the case in the

earlier version of the ACTS. The attributes used in the present system

include cost, and inforination gain measures. Cost is an estimate of the

time and materials required to take a particular measurement or replace a

module, and varies depending on the action involved. Fault information

gain refers to the proportion of faults that are expected to be eliminated

1-4
by a particular measurement or module replacement. Fault information gain

is an important attribute, in that some measurements or replacements are

more efficient than others in reducing the set of all possible faults.

Commercial information gain indicates the degree to which the remaining

possible faults will be "clustered" within commercial circuit components,

given a particular measurement. (Since it is easier to troubleshoot a

circuit when all remaining possible faults are clustered in a couple of

circuit components rather than scattered throughout, a measurement which

will permit greater proportional circuit module isolation, is more

efficient than one not having this capability.) The software is designed

in such a way that new attributes can be added, or any of the existing

attributes altered, if the need arises.

1.3.3 instructional Text. To develop instructional text, the circuit

module designer performs a conventional task analysis on the decisionmaking troubleshooting skills required for a given circuit. The result

is a sequence of skill-based objectives that must be translated into

teaching units. These units generally consist of text material, an

optional question-answer session, and a multiple-choice test, and are

designed to improve the circuit knowledge of the student. The student

is guided through these units until the required level of circuit knowledge necessary to begin troubleshooting is obtained.

Instructional text is also embedded in the troubleshooting unit in the

form of preliminary lecture siaterial, feedback messages, and material

provided by means of a "Help" option.

Utility Driven Performance Feedback. During the terminal troubleshooting

instruction, feedback is derived from the MAU model weights while the

student is being trained. Information concerning the degree of convergence, proximity to the expert model, and relative weighting difficulties

: 1-;

• " i l : *m 
can be assessed from the student and expert models. This information,

in turn, is used to encourage consistency, to teach the student to mimic

the expert, and to adjust the significance with which displayed attribute

information is viewed.

1.3.4 ACTS Transfer to Military Training Equipment. A plan for the

installation of the ACTS on an operational Army computer-based training

system was developed, and is documented in Perceptronics, PDIP-1076-79-6.

The document describes the plan for installation of the ACTS in the Computerized Training System at Fort Gordon, Georgia. It includes a study

of the current configuration and operation of the Training System, identifies expected problem areas, examines transfer alternatives, and presents a complete program plan to accomplish the ACTS transfer.

In addition, a site visit to Fort Gordon was made to determine the scope

of the existing training program and to perform a needs analysis, based

on the entry level of the students and the instructional objectives of

the U.S. Army Signal School at Fort Gordon.

1.3.5 Expeimental Study. A pilot experiment was performed, which

served two major purposes: the first was to fine tune the ACTS and the

methodology for performing the large-scale study designed for the following year and the second purpose was to obtain preliminary data to assess

the training value of the ACTS.

As a result of this study, several improvements were made to the software,

including the design of a new print-out routine for obtaining performance

measures. Despite some inadequacies in the system, the data obtained from

three college students suggest that the ACTS does indeed train the higherorder decision-making skills necessary to troubleshoot electronic circuits.

Details of the experimental method, and the results obtained, are described

in Chapter 3.

1-6

.717TV-S-
1.3.6 Directions for the Future. While initial results are particularly

promising, the ACTS still remains to be tested in the full-blown operational training environment. It is anticipated that such a study will

be undertaken in the near future which will include, among other things,

an assessment of: the transfer of skills which occurs with the troubleshooting of real, rather than simulated, equipment and the long-term retention of skills. It is hoped that field studies will bear out our

conviction that ACTS can make a significant practical contribution to

the training of electronics maintenance personnel. It should not be

overlooked that the basic approach outlined above may have implications

for improving the quality of human decision performance on related tasks,

although the applicability of ACTS outside the electronics maintenance

training area remains to be explored.

1-7

ww." - -- ,V--
2. THE ADAPTIVE COMPUTERIZED TRAINING SYSTEM

2.1 Overview

The Adaptive Computerized Training System (ACTS) focuses on iriproving and

sharpening higher-order cognitive skills in electronics troubleshooting.

The application of decision models to training is reviewed prior to presentation of the features of the ACTS.

Although maintenance tasks rely heavily on a technician's knowledge and

training regarding the maintained systems, such tasks can be viewed primarily as decision tasks. If the technician possesses sufficient knowledge of system parts and functions, he applies it by making a series of

decisions about which symptoms to look for, whether to repair or replace

a malfunctioning part, and so on. ACTS is used in electronics maintenance

training to address the quality of such decisions and the process of generating and choosing from among alternatives, rather than for the learning

of specific procedural sequences.

ACTS incorporates an adaptive computer program which learns the student's

diagnostic and decision value structure, compares it to that of an expert, and adapts the instructional sequence so as to eliminate discrepancies. An expected utility (EU) or a multi-attribute utility (MAU) model

is the basis of the student and instructor models which, together with

a task simulator, form the core of ACTS. Earlier versions of the system

used an expected values model (Freedy and Crooks, 1975; Crooks, Kuppin

and Freedy, 1971). The student model is dynamically adjusted using a

trainable network technique of pattern classification. The training content (instructions) and problem presentation sequence are generated with

heuristic algorithms. ACTS is implemented on an Interdata Model 70 minicomputer and uses interactive graphics terminals for man/machine communication.

2-1

w7-7
The present training system focuses on electronic troubleshooting. The

student's task is to troubleshoot a complex circuit by making various

test measurements, replacing the malfunctioning part, and making final

verification measurements. The model of the student evaluates the student's selection of measurements and replacement of circuit modules.

Troubleshooting provides an excellent application for the ACTS methodology

because it is heavily dependent on judgment and probabilistic inference.

In addition, troubleshooting is of great practical importance in numerous

commercial and military systems, and it lends itself to economical implementation for training purposes.

Work to date has produced an operational system which demonstrates the

feasibility of applying artificial intelligence techniques to computerassisted instruction in a minicomputer-based training system. Experimental evaluations of ACTS have demonstrated that the adaptive decision

model accurately learns the utilities of an expert technician and that

students can effectively use the simulated troubleshooting task.

Additionally, instructions based on utilities can further improve the

decision performance of students; however, feedback of optimum choices

immediately following the student's choice also seems necessary.

2.2 Background: CAI and Decision Making

2.2.1 Individualized Instruction. A central theme in the field of

educational technology is the creation of methods which allow individualized instruction. Training specialists and educational theorists recognize the importance of focusing on the individual student if significant

advances in the efficiency and effectiveness of instruction are to be

made (Crawford and Ragsdale, 1969; Glaser, 1965). Bloom (1968) has

advocated the concept of mastery learning, in which instruction is de2-2

I]
signed and managed so that all students reach a given level of achievement, albeit at different rates.

The principles now included under the rubric of programmed instruction

(PI), which grew out of pioneering work by Pressey, Skinner, and others,

have facilitated the practical implementation of mastery learning techniques. Such principles, also claimed as advantages of PI, include:

student-paced progression, immediate knowledge-of-results, individualized

instructional sequencing, use of explicit performance objectives, diagnostic assessment, and the division of instruction into small discrete steps.

These principles formed the basis for the multiplicity of programmed textbooks, teachiing machines, and the early CAI systems seen in the 1960's.

2.2.2 Adaptive Instruction. It has been recognized for more than a

decade that true individualized instruction must include some form of

adaptation to the individual student (Smallwood, 1962). However, while

most researchers recognize the need to adapt instruction to individual

differences, adaptation is usually made on the basis of response history.

That is, the great majority of adaptive programs are made adaptive by

the logic branching structure of the programs.

Central to the problem of adaptive CAI is the utilization of suitable

criteria for optimizing learning effectiveness and the construction of

.1 decision rules for selecting instructional options. The development of

adequate decision rules is very difficult in conventional adaptive CAI

systems because a student's knowledge and skill level appears to be

structured and fallible, woen viewed in the context of CAl.

Sophisticated optimization techniques for maximizing learning effectiveness have been used in several very elegant and highly adaptive CAI programs (Atkinson, 1972; Smallwood, 1971). However, these techniques have

2-3

Li 7
only been used for simple learning situations, which usually involve lowerorder cognitive skills such as memorizing lists of vocabulary words. This

is because the optimization methods (developed from control theory) require

a precisely stated learning model which predicts student response to alternate instructional options. As skills become more complex, it is less

likely that simple mathematical learning models can be found.

A promising approach to adaptive CAI is the application of Artificial

Intelligence (AI) techniques. Al techniques and theory, traditionally,

have been concerned with the intellectually demanding tasks of problem

solving and goal-directed decision making. These techniques are uniquely

suitable for applications where unstructured environments are involved

(Nilsson, 1965; Slagle, 1971). Natural language understanding and the

heuristic programming approach to pattern recognition have been used in

CAI systems which are based on information structure representations of

the subject matter (Carbonell, 1970; Hartley and Sleeman, 1973; Hoffman

and Blount, *:J74; Brown, Burton, and Bell, 1974). These systems utilize

network analysis of the structures to generate instructional sequences,

thus, the term "generative CAL."

Techniques of adaptive pattern classification can also be used to provide

individualized instruction. Given a model of the student's behavior, the

pattern classifier adaptively adjusts parameters of the model until the

model accurately predicts the student's performance. The model parameters

then provide the basis for generating instructions and feedback. For the

present decision training system, the parameters of an adaptive decision

model are used as the basis for training the student in a decision task.

2.2.3 Adaptive Decision Modeling. Adaptive models of decision making

attempt to learn the decision process of the human operators by (1) successive observation of their- actions, and (2) establishing an interim relation2-4
ship between the input data set and the output decisions (the model).

Learning in this context refers to a training process for adjusting

model parameters according to a criterion function. The object is to

imorove model performance as a function of experience or to match the

model characteristics to that of the operator.

There are two areas of research which attempt to establish useful adaptive decision models. The first, derived from behavioral decision research, is termed bootstrapping (Dawes, 1970; Goldberg, 1970). This

procedure uses a statistical regression process to fit the parameters of

the decision model to the decision maker's previous judgments. However,

the bootstrapping technique is applied off-line to decisions which have

been observed earlier.

A second approach to adaptive decision modeling involves trainable decision and classification networks. This technique is used as the basis

of the ACTS since it provides the capability to adjust model parameters

on-line and to change model performance accordingly. Two types of models

have been used in the ACTS, an Expected Utility Model (EU) and a MultiAttribute Utility Model (MAU). The technique centers around adjustment

of the EU or MAU model decision making. The decision network follows the

decisions of the decision maker and adjusts its parameters to make it behave like the operator.

The dynamic value estimation technique, developed by Perceptronics in the

context of a decision aiding task (Crooks, Kuppin and Freedy, 1977), is

based on the principle of a trainable multi-category pattern classifier.

The value estimator observes the operator's choices among R possible

decision options available to him, viewing his decision making as a pro-

*cess of classifying patterns of event probabilities. The value estimator

then attempts to classify the event probability patterns by means of an

.4

2-5

, . . . .... . .. ... *, . .
expected utility evaluation, or discriminant functions. These classifications are compared with the operator's decisions and an adaptive

error-correction training algorithm is used to adjust pattern weights,

which correspond to utilities, whenever the classifications are incorrect.

Thus, the utility estimator "tracks" the operator's decision making and

"learns" his values.

2.2.4 Decision Models in Maintenance Training. A maintenance technician makes a number of decisions while servicing the systems under his

responsibility. He must decide whether the system is performing within

tolerable limits, what symptoms of trouble to consider, what information

to gather in troubleshooting, what test equipment to use, and so on.

For these types of decisions, the technician must be trained to know

the alternatives available to him, to estimate the odds on the outcomes

of these alternatives, and to assign a value to each alternative. For

example, in auto maintenance, the mechanic is trained to adjust the distributor with a "feeler" guage or a dwell tachometer. He learns how

accurately he is able to set the dwell angle with either instrument.

The decision to choose one instrument or the other is influenced not

only by the odds of setting the angle correctly, but also by the technician's stakes of values for each alternative. The feeler gauge may

be preferred if it is right next to the mechanic in his tool box.

Decision training in maintenance should thus focus the student's attention

on (1) listing the alternatives that he must consider, (2) estimating the

odds of the various outcomes, and (3) evaluating the desirability of the

outcomes. The adaptive MAU decision model in the ACTS provides a method

for instructing the student in these activities. The student is not

trained to make a specific sequence of decisions. Rather, the parameters

of the MAU model are used as a standard reference to generate instructions

about how to evaluate the decision alternatives. In the ACTS, adaptive

L-6

2A. vx ~~A,-
sequential decision training is implemented within the context of electronic circuit troubleshooting. The student's task is to find a circuit

fault by making continued measurements until the device is repaired.

However, the same principles can be applied to many other types of

decision-making tasks.

The training given in the circuit fault diagnosis and repair task is

based on the assumption that the student has a good basic background

in electronics but that his experience with troubleshooting is limited.

This might be the case with a student who has recently completed advanced

military electronics training but has not yet performed troubleshooting

tasks in his first permanent duty assignment. This skill lavel can be

assessed either in terms of previous training received or in terms of

performance on an entering test of electronics and troubleshooting knowledge. It is assumed that the prerequisite laws of electricity, circuit

component behavior, circuit sub-systems, circuit diagrams, use of test

equipment, and the like, have already been learned.

2.3 ACTS System Description

The ACTS is an interactive computer program that models and simulates the

four functional units of training: (1) the task being trained, (2) the

student, (3) the instructor, and (4) the instructional logic. The organization of these four units in ACTS is illustrated in Figure 2-1.

2.3.1 Task Simulator. In ACTS, the student's decision task involves

troubleshooting an electronic device. The troubleshooting task centers

on a model of an electronic circuit in which faults can be simulated.

The circuits currently used are a modular version of the Heathkit IP-28

*regulated power supply and the U.S. Army A9000 power supply. The simulated circuits have 10 and 11 functional modules, respectively, which can

2-7

hoc "&OEM,
EVALUATION INSTRUCTIONSUETSIUAO

INSTRUCTIONAL LOGIC

ACTS ORGAIZATIO FNCTIONL 

MOE

2-8RUTO

DEISO MODEL
be replaced and 32 and 23 measurements, respectively, which can be used

to isolate faults. The operation of each power supply is simulated by

the computer program, using a table-driven simulation of the fault system. The program simulates the results of checking symptoms, taking

measurements, and replacing modules.

Training in the present system occurs with certain restrictions on the

extent of circuit simulation. The student interacts with a terminal

which contains a display of the simulated circuit; thus he cannot make

such troubleshooting observations as smelling faulty capacitors, looking

for burned resistors, or touching overheated semiconductors. In addition, the measurement results are presented in a semi-interpreted form

thigh, normal, low,, rather than as absolute readings, (e.g., 3.6 volts,

1.25nA), so tat the student need not refer to a table of normal circuit

levels. Although these inodifications involve an abstraction of the

troublesnooting task, it is assumed that they do not affect the critical

decisicn ;;aking js)ects of the troubleshooting task.

The ci' ilru; , ,esigned jt to meet several objectives. In

.i~o to rdrc ,,, ,rv ,oneit for observing troubleshooting

eni,,',r, toe i a,, ,Vtfs tne results of the student's choice of

:iternatives ,' 1, , tle results of measurements. Finally, the

,,rcar iuOd. I> ,,u ; o t, siimoil, t the essential characteristics of

le i ir- 'I': K f I , r'< t . Thus, the outcomes of the measure-

,ents r . ". , cing the fact that, in practice, fault

locations are uncer,;r, ,,r the troubleshooter.

2.,.2 jtidi:,1t 'I ,n . u ,I. The student decision model is a mathematical decision lodel uised in the ACTS to model the decision behavior

" of the trainee and nis i'nstructor. The student decision model provides

a method of describing or defining the student's behavior. The ACTS

then uses the model t inmer toe current state of the student's knowledge.

4A >9

V
The decision model not only describes the initial state of the student's

knowledge but it also tracks changes in the student's performance, adapting the model parameters to describe the student's improvements and

errors. From this model of the student's behavior, the ACTS gives instructions to improve the student's decision making.

A multi-attribute utility (MAU) decision model is used to represeht the

student. The MAU model is both a descriptive and normative model of

decision making which assumes that a "rational" decision maker selects

the alternative= with the greatest expected value. The multi-attribute

utility model is an improvement over the expected utility (EU) model

that was the basis of decision models in the original system. This

approach was selected on the basis of earlier experimental work done

with the ACTS system. The MAU model has been applied to related areas

in adaptive decision modeling and information acquisition tasks (Steeb,

Chen, and Freedy, 1977; Samet, Weltman, and Davis, 1977) and was found

to be more effective than the EU approach used in the original ACT system.

The improved MAU model has several advantages over the EU model: Fewer

utilities are needed to model a particular circuit troubleshooting

strategy; it converges faster, is more general, and is easily transferable across different circuits. It is also easier to establish a new

model for different circuits and thus, it is more compatible with the

operational training environment.

A unique aspect of the multi-attribute model is that utilities are

assigned to general attributes of troubleshooting actions, rather than

to specific outcomes only. According to the model, decision making

within the context of electronic troubleshooting nvolves three basic

factors: (1) Fault information gain, (2) commercial information gain,

and (3) cost. The expected value of an action is then the sum of these

2-10
factors weighted by specific utilities. The attributes and model are

presented below:

Fault Information Gain:

Ai 1 Pij (F-Fij)/F

Commercial Information Gain:

A. = 7 Pij (M-Mij)/M

Cost:

A i3 = Ci

MAU:

MAU i U A

- K iK

Where

Pij = Probability that the j'th outcome will occur if the i'th

alternative is chosen.

F = Current number of possible faults.

M = Current number of possible faulty modules.

Fij = Number of possible faults given current possible faults

and the j'th outcome for action i.

Mij = Number of possible faulty modules given current possible

faults and the j'th outcome for action i.

C. = Cost of i'th action.

AiK = k'th attribute for action i.

I UK = Utility for k'th attribute.

MAU = Expected utility of action i.

Given the available alternatives, attribute levels and utilities, the

optimum choice is determined according to the maximum expected utility

2-11
principle by calculating the expected utility for each possible alternative and then selecting that alternative with the-greatest MAU.

ACTS uses the MAU model not only as the description of the student's

decision making but also as the basis for estimating changes in his

knowledge as inferred from his decision behavior. A technique of artificial intelligence, known as the learning network approach to pattern

classification, is used to estimate the student's utilities in the EU

model (Crooks, Kuppin and Freedy, 1977). The utility estimator observes

the student's choices among the possible decision alternatives, viewing

his decision making as a process of classifying patterns of event probabilities. The utility estimator then attempts to classify the event

probability patterns by means of a multi-attribute discriminant function.

These classifications are compared with the student's choices and an

adaptive error-correction training algorithm is used to adjust pattern

weights, which correspond to utilities, whenever the classifications are

incorrect. This utility estimator operates concurrently in real time

as the student performs troubleshooting operations; thus, the MAU model

continuously tracks the student's decision performance as it changes

during the course of training.

2.3.3 Instructor Decision Model. The second decision model in ACTS

is an MAU model of an expert decision maker's performance. This model

is used (1) as a standard against which the utilities of the student

model are compared, and (2) as a source of help in directing the student's

activities and in suggesting alternatives. The instructor model has the

same mathematical form as the student model, except that the utilities

are preset and remain constant throughout a session. The utilities of

%;' this model are adaptively estimated prior to the training session by

tracking the performance of an expert technician as he indicates simulated

faults or they are set based on expectations of expert troubleshooting behavior.

2-12

#A ..... J
The ACTS includes an algorithm for calculating the conditional probabilities of action outcomes. Conditional probabilities are of the

form:

The probability of obtaining a particular measurement outcome, given the previous measurement outcome history, and

the measurement.

These conditional probabilities are obtained by the ACTS algorithm from

the a pri fault probabilities, PK, by the following formula:

P.- = L PK ? PK

i K Oij KcS

Where S is the current set of faults, Oij is the subset of S for which

the outcome of action i is the j'th outcome. The a prtori probabilities

are obtained from an expert technician during the development of the task

fault model.

2.3.4 Instructional Logic. The fourth major functional unit of the

ACTS computer program is the instructional logic which selects the instruction and aiding information for the student. The instructional

logic checks for convergence of the student's utilities, compares the

student's utilities with those of the expert, and compares the student's

expended cost with that of the expert for the same problem. These three

condition checks are then used to select or modify the following messages:

Your choices indicate that you are inconsistent in your

troubleshooting strategy. Before making a choice, consider

carefully the uncertainty reduction, fault isolation, and

costs associated with each choice.

2-13

- - A , W 7n =
Congratulations. Your choices show that you are consistent

in your strategy for troubleshooting. However, there may

still be differences between your strategy and the expert's.

If so, the next page will describe these differences.

You appear to overemphasize: uncertainty reduction and

underemphasize: cost.

Congratulations. Your performance is identical to that of

the expert. You are now a qualified troubleshooter on the

IP28 circuit.

Congratulations on repairing the circuit. Your total cost

to debug the circuit was 190. The instructor's total cost

would have been 120.

Prior to the troubleshooting session, the student is assumed to have completed the preliminary lessons on the power supply involved. Consequently,

instructions in the troubleshooting unit are not focused on the type of

measurements to make or the functions of specific components or subcircuits. Rather, ACTS instruction is directed toward training an inexperienced technician to evaluate the utilities of the alternative

measurements he can make and to select those alternatives that are most

effective, given their relative costs.

In addition to the instructions that are displayed on the basis of the

student's decision performance, the ACT system also includes a HELP

option which the student can select as desired. The HELP option uses

the expert decision model to suggest which measurements to make, their

tradeoffs, and their relative overall values.

2-14
2.4 Hardware ConfigurationI

The hardware configuration for the ACTS is shown in Figure 2-2. It consists of four major components, namely an Interdata 70 minicomputer with

a 64K memory, a Tektronix 4024 CRT for each subject, a console communication device, and an experimenter communication device, which may be either

a CRT or a teletype.

The experimenter communication device is distinct from the console communication device because the structure of DOS does not allow the return

of status for a supervisor call (SVC) read proceed on the console device.

It is necessary to have this status return after a carriage return is

received so that the ACTS can be apprised of the completion of a command

input, independently, for any of the subjects or for the experimenter.

The experimenter's communication device is used to enter control parameters and to terminate student sessions. The Experimenter has control

of the system, both prior to the start of a teaching session and during

its performance. Prior to the start of a session, the experimenter may

enter certain system parameters to indicate specific options which are

to be included in the current session. These parameters and associated

options are summarized in Table 2-1. Additionally, the experimenter may

terminate a student at any point during the instructional program.

Some of the experimenter's options are self-explanatory; those that are

not are listed below:

Commercial designations are used only for precision of description.

Their use does not constitute endorsement by the Department of the

Army or the Army Research Institute.

2-15

I7-
OEXPERIMENTER

TY TERMINAL

OR CRT

INTERDATA 70 CONSOLE

MINI-COMPUTER

iI

STUDENT I STUDENT 2

FIGURE 2-2.

MAN-MACHINE CONFIGURATION

2-16
TABLE 2-1

EXPERIMENTER-SUPPLIED SYSTEM OPTIONS

PARAMETERS OPTIONS

I. Number of Students in Session (1 or 2)

II. (For Each Student)

1. Restart? (1-Yes, O-No)

(If restart, student's performance

file is loaded from disk)

2. Student's Name Up to 20 characters

3. Current Date Up to 20 characters

4. Circuit to be Used (0 or 2)

5. Starting Objective (2 or 3)

6. Starting MAU Model Weights for

Student 3 values

7. Simulation (1-gnome operator,

2-real student)

8. Simulated Subject Weights 3 values to drive

(Objective 3 only) gnome operator

9. Help Available? (1-Yes, O-No)

10. Problem Presentation Sequence 0 - Random

During Troubleshooting Objective 1 - Fixed

2-17

-,- ow
RESTART - If you wish to load the student's performance data from the

disk and continue from the point that the previous session left off,

the proper response here is "1" for "Yes." There is a single performance file available for storage on the disk; so, if you wish to pick

up a session where it was terminated, no other student sessions may be

run in the interim, since that would change the contents of the performance file.

CIRCUIT TO BE USED - The IP28 circuit is obtained by typing "1." The

A9000 circuit is obtained by typing "2".

STARTING OBJECTIVE - The troubleshooting units for the IP28 are coded as

unit "3." The single preliminary unit implemented for the IP28 is coded

as unit "2."

STARTING MAU MODEL WEIGHTS FOR STUDENT - The student utility model must

be given some set of starting weights. The weights are, respectively,

for fault information gain, commercial information gain, and cost. Type

in ' integer values, each one followed by a carriage return. A good

starting set of weights is: 1,1,1. The weights are normalized prior to

MAU calculation.

SIMULATION - Currently no simulated subject has been implemented; consequently, the only legitimate response here is the "2" for "real

student."

SIMULATED SUBJECT WEIGHTS - This option will not be presented since the

simulated subject will not be selected for the simulation parameter.

HELP AVAILABLE - When a "1" for "Yes" is input at this point, all the

keys from the expert model will be made available.

2-18

-? :: ,, . i - ., ' , r - - - - - -T ... -z-* . "'!, . . # ' i
PROBLEM PRESENTATION SEQUENCE - The types of problem sequencing currently

available are: random - problems are selected according to a uniformly

distributed psuedo-random number sequence, and fixed - problems are presented in the order of the difficulty level of the corresponding fault.

NOTE: In Table 2-1 the parameters 1, 6, 7, 8 and 10 provide

information utilized in only the troubleshooting instructional

unit. This unit is associated with objective XII in Table 2-2.

However, the code for this unit used in the current implementation of ACTS is 3 as indicated in the prior discussion of starting objective. The reason for this is that only one of the

preliminary units was implemented, namely the unit for objective II in Table 2-2, and a transition from this unit to the

troubleshooting unit must take place by counting up I from the

current unit.

If all students on the system during a session are terminated, the session will terminate. Each student has exclusive access to his or her

own terminal, through which messages are sent or received.

Messages made available to the student are of two kinds: (1) strings of

English language text, and (2) circuit diagrams or modifications to circuit diagrams (e.g., the brightening of a module to indicate that its

replacement has been accomplished).

2.5 Instructional Approach

*

2.5.1 Training Procedure. Training on the ACTS is provided through a

system of phased instructional presentations. A series of units on the

given power supply is presented to the student. The material begins with

the most basic information about power supplies and terminates with the

2-19

--. -
troubleshooting unit, which consists of a number of circuit fault problems. For all instruction prior to the troubleshooting unit, the

procedure is to present text material to the student, allow him to ask

questions and receive answers, and then give the student a test. If

he passes the test he is advanced to the next unit; otherwise, he repeats

the current unit. When the student has completed all the preliminary

units, he begins the troubleshooting phase of instruction. Table 2-2

presents the instructional objectives for both the preliminary and

troubleshooting phases of instruction.

Each troubleshooting problem consists of a single circuit fault which

the student must locate and replace. On the display is shown a schematic diagram of an electronic circuit, plus printed messages which indicate possible actions and give information. The student selects his

responses and types them in on a keyboard. The student can select from

among a number of activities to isolate the fault in the displayed power

supply circuit. The student can choose to take a voltage or current

measurement, replace any circuit module, or request help. Following a

student's command to perform these activities, the ACTS program displays

the results of the simulated activity and then indicates the next allowable activities.

Interspersed among the fault problems, the ACTS presents the instructions

which describe recommended circuit measurements and the conditions during

which they should be chosen. After the instructions have been displayed,

the fault problems are resumed. However, the student can request to see

these instructions at the appropriate time by selecting the appropriate

command on the display screen.

Appendix B presents samples of the instructional sequence which characterize the preliminary and troubleshooting phases of instruction.

2-20
TABLE 2-2

INSTRUCTIONAL PHASES WITH OBJECTIVES (1P28)

I. Given a description of a general power supply and the IP28, the

student will state that the IP28 is a regulated power supply as

opposed to an un-regulated power supply and specify the distinction.

II. Given an IP28 module and the macro-assembly of which it is a part,

the student will correctly specify its function within the given

macro-assembly.

III. Given any of the four operating conditions for the IP28, the student will specify which circuit modules are affected functionally

by this condition and what the effect is.

IV. Given any module in the IP20, and a failure mode for that module,

the student will correctly specify the functional consequences of

that failure mode for a specified operating condition.

V. Given any module in the IP28 and a failure mode for that module,

the student will correctly indicate the value of any voltage measurement anywhere in the system as high, low, normal, and unstable

based on the circuit operating condition.

VI. Given initial circuit symptoms, the student shall report accurately

'4 one of the following:

(1) Circuit is functional and needs no adjustment.

(2) Circuit is malfunctioning.

2-2 2-2
TABLE 2-2 (CONTINUED)

VII. Given circuit symptoms with or without a history of pre-obtained

measurement results, the student shall accurately select the most

likely failure modes.

VIII. Given a set of possible failure modes, and a set of symptoms and

measurement outcomes, the student will select those measurements

most likely to test those failure modes within operationally defined

tolerances based on the expert's MAU model.

IX. Given a set of measurements and module replacement candidates and

a history of symptoms and measurement results, the student will

rank them according to expected information gain, topological

isolation potential, and cost.

X. Given the attribute levels for a series of measurements or module

replacements the student will select a measurement or module

replacement to perform differing from the expert's top MAU by a

value less than some deviation tolerance.

XI. Given a symptom and measurement result history and the knowledge

that one or more circuit modules has just been replaced, the student will make appropriate measurements to verify the result of

the module replacements.

XII. The student will troubleshoot a non-functional circuit having one

or two faults, replacing the failing modules in reverse order of

dependency when malfunctions are causally related. Attribute

levels will be displayed for him.

2-22

FV
2.5.2 Consideration and Help. When the circuit is displayed, a malfunction is signaled by displaying overt symptoms in a table of symptoms

and measurement outcomes. The student is then told that he will next be

expected to input some action candidates for consideration. He may also

ask for help at this point. Provided that the 'HELP' option is allowed,

a help request will provide the student with the expert's considerations,

as shown in Figure 2-3. After looking at these, the student may request

help again--in which case certain tradeoff information for the expert's

considerations will be displayed. This information includes the cost of

each action, all outcomes and their probabilities for each action, and

the fraction of faults to be eliminated by each outcome of each action.

The student next chooses his candidates for consideration. These candidates may be measurements and/or module replacements. The system then

displays for him the value of each attribute for each of his considerations. At this point help nay again be requested if the 'HELP' option

is set to aid tne student in choosing an action from amongst the considerations. The student may also choose immediately without help.

2.5.3 Action Selection and Help. If help is requested, the tradeoffs

of the final considerations are then displayed. The message is the same

as that used earlier to display the tradeoffs for the expert's considerations. Help nay then be requested again, in which case, an expert

ranking of the final considerations is presented. The attribute levels

of the considerations are then re-displayed.

The student may then choose 'none of the above' in which case he will

be asked for new considerations or he may type a choice from the list

of considerations. If his choice is a measurement, its outcome is displayed in the symptom/outcome table. If his choice is to replace a

module, the part of the display depicting the module is enhanced on the

2-23

l - -,, .- .. L' - =' . I - ,., .. .. 2-.. .. . .- . ... .... i3
[TrRANS- REcr. F, SER IES CIJRR.

F:ORME:R F: 1 Ilr. REG. SENSE

COM

CORR

I LOAK)

COMMON

REFERENCEl COMMON

VOL. CUR RES (INnERP THESE (A1RCHNMSTANCH, TimE INSTRUCTOR WOUILD CONSIDEFR TUlE ______ -- - FOLLOWlING FOOR ACI'FONS:

OIITI' t,

TI' I 1,MIWV1P TI' 2 rPAIP(Vf? TI' I rrsrwTR

TI' 4 N TP'9!WUP'

ri' 6 T-O tONTINN11E. iPRESS, "RrTIIRN",

TIV 7

i'A

FIGURE 2-3.

EXPERT CONSIDERATIONS DISPLAYED

IN RESPONSE TO THE HELP OPTION

2 -24
screen. If the chosen action did not repair the circuit, the cycle

repeats with a request for new considerations. If the chosen action

correctly replaced the faulty module, the overt malfunction symptoms

are corrected on the screen and the system enters an evaluation phase.

2.5.4 Evaluation (Feedback) Phase. When the evaluation phase begins,

the student is first congratulated on having repaired the circuit and

given his total expenditure to compare with what it would have cost the

expert. If his utility model has converged, indicating that he is using

the displayed attribute information in a consistent manner, he is told

that he is now consistent; otherwise, he is told to weight the attribute

information more carefully. If his utilities differ significantly from

the expert's, he is told which ones are high and which ones are low;

otherwise, he is congratulated as an expert and instruction stops. Providing that he has not yet converged to the expert's utilities, the

system advances to the next circuit fault problem and again presents

malfunction symptoms.

2.6 Software

Two Perceptronics' documents are available which describe the ACTS software. The first, PFSS-1076-79-8, is a description of the software specification for the ACTS. The specifications are accurate for the configuration of the ACTS software as of the time the first experimental study

was conducted. As a result of the study, some changes were made which

will be described in Chapter 3, but the changes do not fundamentally

4alter the specifications as described in PFSS-1076-79-8. This document

includes a dtscription of the function as well as of the structure of

the software, including the various files needed to run the system. Also,

programs for instructional objectives are described.

2
The second document, PDCMDM-1076-79-7, is a manual for developing new

circuit modules for the ACTS. Generalized techniques and step by step

procedures are included to aid circuit model developers in creating

instructional material, circuit-specific parameters, and software necessary to fully utilize the ACTS. The manual describes how a new circuit

module is created and what instructional material is needed to familiarize the student with the characteristics of the specific electronic

circuit. The circuit-specific parameters needed for implementing the

new module in software are explained, as well as the steps necessary

for integrating the new module with the rest of the system software.

Procedures for creating circuit diagrams on the graphic display, representing circuit specific parameters in data tables, and procedures for

generating circuit specific production rules are described in detail.

The manual emphasizes and describes the special considerations required

for integrating the instructor's and the programmer's tasks when developing new circuit modules and associated lessons.

i

2-26

T 7
3. RESEARCH ISSUES

5.1 Ubjectives

Although the major objective of Year I was to develop and implement the

software for the new ACTS as described in the previous section, an

equally important consideration was to run a pilot study with actual

subjects. The pilot study was needed to yield data concerning the subused to tcst the system so that appropriate modifications could be made

for tthe full-scale study designed for Year 2.

SubjeLtS

Three students from the Electrical Engineering Department at California

State Universi-y, Northridge, were recruited as subjects. The only recruiting requirements were that the subjects have Junior or Senior

siznaing, or equivalent experience, and that they have a good working

knowlcdye of English. The purposes of these requirements were, at

first, to minimize the need for extensive preliminary training so that

thu SUcjtctS' time could be primarily devoted to obtaining data on the

triublesnooting objective, and second, to avoid possible confounding effects duc to difficulties in understanding the instructional text associuted with the ACTS.

)omie oackground information on the subjects was obtained by means of a

preliminary questionnaire. Tne subjects were between 19 and 23 years of

uge, and they all hdd some experience in troubleshooting electronic circuits, as well as in interdcting with computer-based systems. One feMale and two iale subjects signed up for the experiment, which required

two sessions, each lasting between 4 and 5 hours. They were paid at the

A'-7

I .. ... .. . .. . .
base rate of $5.00 per hour and, as an incentive, were told that they

could earn additional pay for good performance at a rate of up to

$4.00 an hour. Good performance was determined after the data were

collected by comparing the three subjects' results with each other.

The subjects were paid by mail, following the completion of their two

sessions.

I ni sruct iOfnS

When t te subjects arrived for their first session, they were told that

tney were to participate in a study designed to train effective troubtshooting of an electronic circuit. They were seated at the console

and were given a handout which contained an introduction to the task and

to concepts relevant to power supplies in general, and to the IP28 in

particular. in addition to this introduction, a troubleshooting guide

was included in the handout in which measurement outcomes were defined;

also d table of the probability of occurrence of module faults was

given, as well as a table specifying the correspondence between faults

and measurement outcomes. An additional table listed the responses

recognized by the ACTS and their corresponding measurement or replacement

costs. ThQ handout is reproduced in Appendix A. The subjects in the

present study felt that the handouts were quite clear, and they reported

hdVing n problems in understanding the concepts that were presented.

The subjects were run individually, with the experimienter sitting at a

table nearby. Mfter having read the handouts, the subjects completed

the pyrt-limindry objective presented on the CRT, as discussed in the predious chapter. Any question was answered by pcraphrasing the written

instrucLions.

;-

7.2
.. 4 Sequenc i ng

uurrently, the sequence in which problems are presented to the student

can be in one of two modes, random or fixed. The fixed sequence is a

function of the level of difficulty of the problems, with easy problems

being presented at the beginning of the instructional sequence and

becoming increasingly more difficult as the sequence progresses. The

major criterion for determining the level of difficulty of a problem was

the logical complexity required for solving it. The minimum number of

measurements required for isolating a fault was not a criterion of difficulty. For example, the faults of problems #7 and #8 can both be isolated by understanding the relationships between the initial (given)

symptoms and the functioning of the modules, but it is rated as medium

in ditficulty bucause it assumes a nigh level of understanding of the

aocular functioning and interactions within the circuit.

Level of aifficulty was defined as follows:

£I) Easy problems--simple fault isolation.

Lj ,ieuilum problems--fdult isolation requires sequencing of

measurements and understanding of modular functioning and

interactions within the circuit.

.30 Difficult problems--fault isolation is in a feedback loop,

requiring multiple measurements and a high level of understanding of the circuit action.

The problem, numbers, which were not shown to the subjects, reflect incrasiny levels of difficulty, as defined above. Thus, for the fixed

isequence, subjects were given problems #1 through #14 sequentially; for

;-3 .AL
the random sequence, the problems were simply selected randomly. In the

present study, the random sequence was preprogrammed and was not randomized anew for each subject; this was a temporary simplification that

will be changed for the Year 2 study.

,1.5 Results and Analysis

During the troubleshooting part of the study, the students were allowed

to refer to the tables in the handout which provided information concerning the types of measurements that are permissible and the characteristic outcomes for any given fault.

The stuuents' responses were collected dnd printed out at the end of

each problem. In uddition, the experimenter noted the number of times

Lhl stuaent asked for help ana the amount of time needed to solve each

problew. Information concerning the attribute levels of the students'

cunsiderations was also recorded by the experimenter.

Luring tihe first session, two of the students were given the fixed

presentation sequence, and one student the random presentation sequence.

Tinls urger was reversed for the second session.

5.5.1 General Results. Table 3-1 summarizes the data on several performance measures. The subjects' performances are reported individually

for Sessions I ana 2. The presentation order of the problems can be inferred from the problem number: consecutive problem numbers represent

the fixed sequence, an non-consecutive problem numbers the random sequence.

The first column of Table 3-1 identifies the subject, the session (I or

2) and the prublem number. Columns 2 through 6 represent summarized in3-4

,-. - i - ,.:;; i- ,- -'" . ... l .• - ' ' -"S 
TABLE 3-1

SUMMARY OF SUBJECTS' PERFORMANCE

NUMBER OF

SUBJECT PROBLEM MEASUREMENT NUMBER UTILITY COST TIME

# NUMBER DIFFERENCES1 HELS DIFFERENCES? DIFFERENCES (MIN) COMMENTS

Is 10 +1 9 .48 0 45

E 1 +4 2 7 .17 +12 45 CONVERGE s 1I 0 0 .17 0 20

01 113 0 0 .13 -2 15

s 1 0 0 .17 +4 27

E -1 0 .15 -10 12 CONVERGED

3 0 0 .15 0 15

2 (1 +1 22 .47 +70 120 E . s 2 0 20 .47 -2 45 MAXIMUM

(3 0 17 .47 +4 45 TIME

10 0 2 .47 0 9

4 +1 5 .17 +8 13 SYSTEM' s 1 -1 1 .47 -8 4 WENT DOWN .3 0 4 .47 98 14

3 1+1 12 .47 *70 90

2 -1 8 .47 -6 25

E 3 0 3 .36 +4 12 MAXIMUM s 4 +i 4 .36 +8 8 TIME

5 +3 2 .27 -12 10

6 0 4 .27 0 10

7 0 4 .51 0 7

Oo 0 3 .47 0 16

E 11CONVERGED 4 +4 2 .26 +30 21

s 2 .26 0 6

13 O2 0 1 .14 -2 23 1 0 0 0 .13 0 5

1DIFFERENCES BETWEEN EXPERT AND STUDENTS 2 EAN OF ABSOLUTE VALUES FOR 3 ATTRIBUTES

3-5

IA
formation on each subject's performance. The last column is used for

comments. "cunvergeo" means that the subject's utilities came to match

those of the expert within a certain tolerance limit. In the present

stuoy, converyence dutomatically stopped the session because the subject

was assumed to have become an expert. Subject i converged in both sessions and Subject 3 converged in the second session. "Maximum time"

means that the session wds ended because the 5 hours allotted to it were

used up. This happened for both Subjects 2 and 3 in the first session.

uring the second session with Subject 2, the system went down and the

session was ended for that reason.

.5.z Expert's Model. Of the five performance measures reported,

thrte of them represent comparisons between the subject's performance

und t ,dt of the expert. The expert's performance--namely the sequence

of imesuruments and module replacements--was determined theoretically by

applying tne expert's model to the attributes under consideration, namely tne reduction in uncertainty, the expected fault isolation, and the

cosL of the action. Initially, the utility levels for the expert were

set in such a way that the attribute of cost was given a slightly higher

weignt than the other two attributes, decrease in uncertainty and fault

isulation. Given these assigned attribute weights, the expert's model

was simply jpplita to the system to determine the best action sequence

for each fdult. This sequence is shown in Table 3-2 for each fault.

Tie number of the fault corresponds to the fault numbers identified with

the MCTS programs: they are only used for identification and were not

shown to the suojects. The sequences of measurements and module replacements shown in Table 5-2 represent the best expected performance in

the long run, although they may not be optimal for any particular fault.

That is, in any particular case, it is possible for a subject to isolate

d t.ult more cheaply than the expert, or by using fewer measurements,

3-u

'4U
TABLE 3-2

OPTIMAL DECISION SEQUENCES (EXPERT)

FAULT COST SEQUENCE

1 120 TP4REPO TP9DCVR TP3ACVR TP1ACVR TRA

2 92 TP4REPO TP9DCVR TP3ACVR TP1ACVR REC

3 116 TP4REPO TP90CVR TP3ACVR TP40CVR SOU

4 90 TP4REPO VOL

5 90 TP4REPO LIM

6 58 TP500VR STA

7 50 SEN

8 130 SEN LIM

9 116 TP4REPO TP9DCVR TP3ACVR TP4DCVR SER

10 88 TP50CVR VOL

11 58 TP5DCVR STA

12 88 TP4REPO TP9DCVR TP8DCVR REG

13 88 TP4REPO TP9DCVR TP8DCVR REF

14 186 TP4REPO TP9DCVR TP8DCVR REF TRA

-3-

LA
TABLE 3-2 (CONTINUED)

EXPLANATION OF ACTION CODES:

Any three letter sequence corresponds to a module replacement. The full

meanings of each module replacement abbreviation are given below:

TRA - Transformer

REC - Rectifier

SOU - Current Source

SER - Series Regulator

SEN - Current Sense

LIM - .Crrent Limiter

VOL - Voltage Limiter

REF - Reference Rectifier and Filter

REG - Reference Regulator

STA - Output Stage

(See A-12, Figure 2)

The seven letter strings refer to measurements. The first three

characters refer to the major test point (See A-12, Figure 2). The

next two characters can be any of the following possibilities:

AC - Alternating Current

DC - Direct Current

RE - Resistance

The final two characters can be any of the following possibilities:

VR - Voltage Regulated

CR - Current Regulated

PO - Power Off

Thus, TP4REPO means a measurement with major test point 4, resistance

check, in power off mode.

3-8

. , -.-,.-•. . ,, - - ;:i -., . -- , . *..* *.. .. ..-. , ., -m ::: -
but in the long run, based on the multi-attribute utility model, the

expert's model optimizes all troubleshooting sequences.

3.5.) Number of Measurements. As can be seen from Table 3-2, the opilum number of measurements required to isolate a fault varies widely

across problems. In some cases, a fdult can be isolated in a single

measurement, wnile in others, as many as five measurements are needed

minimally. It would be meaningless, therefore, to list the number of

measurements made by each subject on a given problem; rather, the performance measure reflects the difference between the number of measurenents made by the subject ano the optimum number suggested by the

expert's model.

The difference Oetween the subjects' and the expert's number of measuremants is snuwn in column 2 of Table 3-1. Positive numbers mean that the

subject made more measurements than the expert, zero means that the

number of measurements was the same (but not necessarily that the measuruments tnemselves were identical), and negative numbers mean that the

subjects found a solution requiring fewer measurements than the expert's

model. In this case, while the subject's solution may appear more efficient, it does not imply that such a solution follows the rules of optimal decision making.

j.5.4 Help Option. Tne help option provides the expert's considerations for any given action and ranks those considerations from best to

worst, based on the multi-attribute utility model. By continually using

the help option, therefore, a subject could conceivably isolate a fault

dnd replace the correct module in the most efficient manner without

A* learning the aecision-making procedures inherent in the ACTS. According

to the built-in criteria, a subject could then become an expert at

troubleshooting electronic circuits simply by following the expert's

3-9

C7
consolerations, unless the subject's utll ities are differentially

:ralmo as a tunctzion of using help. Thus, whenever help is used, the

training of lhe subject's model is deactivated, implying that the

subject's performance does not reflect any learning.

AS discussed in the previous chapter, help could be obtained at several

points in the course of 1solating a fault. For each problem solved, the

experilmenter counted the number of times the help option was used by the

subjects, and tnis count is reported in column 3 of Table 3-1. The

NUPIDer oT tLimeS help was used vdries a great deal over subjects, but it

does Seem tu reflect sQme learning in that the amount of help used decredsUs over trials for all subjects. This may be a result of a decrease in neeo for nel , but it may also be a function of the instructions. Al*hougn the subJucts were uryed to use the help option whenever

necessary, tne instructions did specify that help should be used as littl] as Possible so hat their performance would reflect the course of

learning the decision-moking procedures of the ACTS.

Tnue dat also reflect a certain correspondence between the amount of

ninlp used and -lie dilfficulty each subject experienced with the system.

uoect i, who converged rapidly in both sessions, only used the help

option tor the first two problems, while Subjects 2 and 3 used it

Lhrounout the first session. In all cases, the amount of help used

during the Second session wus noticeably less than during the first session. An auuiltional reason for the decrease in the use of help is that

the handout proviueu the subjects with sufficient information for them

to make reasonable selections without the use of help. That is, the

tables gave thdmm ll the Options necessary to effectively troubleshoot

3-10

...
J.t.b Utility Differences. Changes over time in the differences

beCwen Lhe subjects' utilities ano chose of the expert reflect the

adaptdtion of the subjects' behavior to correspond more closely to that

of the expert. This comes about as a result of the feedback provided at

the end of edch problem which informs the subjects when they appear to

wtiyn one or another of the attributes oo heavily or too little.

Thanginy their behavior in response to this feedback causes their utilities to gradually match tnost of the expert, producing convergence of

the two models.

The differences between the subjects' utilities and those of the expert

ire shown in column 4 of Table 3-1 and in Figures 3-1 to 3-4. Column 4

of Table 3-i an d Figure 3-1 show the mean of those differences, while

Figures 3- to 3-4 show the changes in the utility differences for the

indivicuol aiLribuLes. Since some of the utilities are negative, tlie

mean was calculaLLd by taking the absolute differences between subjects

aflU eXpcr2S.

Tht utility differences on individual attributes appear almost erratic

from problem to problem, but when the mean of the differences is used,

Lnu prugrtssion of the subjects' behavior is more apparent.

Although all performance measures suggest that some learning is taking

plact:, utlilty differences probably reflect convergence of the two

mo'oels, and hence learning, better than any of the other performance

measures. This learning is readily reflected in Figure 3-1.

From both sets of douta, the individual and the mean differences, it is

clear that Subject I became very rapidly familiar with the system, a

* result which is dISO reflected in the amount of help used and the time

P taken per problemn. A similar result was obtained for Subject 3: all

i3-11
0 :RANDOM SEQUENCE x :FIXED SEQUENCE

: SUBJECT H!

SUBJECT 12 ........ . SUBJECT 13

.70

.60

.50

.40 "' I

.30 \". I

.20

.10

f0 I I I I l ! I I , g l I

1 2 3 4 5 6 7 8 9 10 11 12

PROBLEM ORDER

i*

FIGURE 3-1.

MEAN OF ABSOLUTE UTILITY DIFFERENCES

3-12

.117
SUBJECTS

02#3

/L03 ...... ..

+.60

+.50

*-. -- Q.

+.40 \. ........ 0

+.30

+.20 ....... .0

+.10

0

-. 10-

- .20

-.30

-. 40--

-. 50

• -. 80

1 2 3 4 5 6 7 8 9 10 11 12

FIGURE 3-2.

UTILITY DIFFERENCES FOR UNCERTAINTY REDUCTION

$ 3-13

I
#.92 SUBJECTS 99

+.60

-.50

- .80

1 2 3 4 5 6 7 8 g 10 11 12

FIGURE 3-3. I UTILITY DIFFERENCES FOR FAULT ISOLATION

3-14
SUBJECTS

02

360 . ....

+.0 - . .. . 0

+.0 -

L :. :.

. 5 6 "

-. 20 --. :

*3 1

*. //:

-.4o - .

:" I :

: I . ' :: -.50 : I : " I , .. .,

: I .:

- I I

8 9 10 11 12

FIGURE 3-4.

F UTILITY DIFFERENCES FOR COST

~3-15

i ;A zm ,

it....
perfonidnce mtasures improved over time, even during the first session,

incluoing his decision-making performance, even though he only converged

during the second session.

While convergence of the utilities appears to be reflected in similar

improvements on the other measures, the obverse is not necessarily true.

Improvements can be observed on some measures without coresponding convergence of the utilities. For example, the amount of help used and the

time taken per problem aid improve over trials for Subject 2, but his

utilities remaineo quite stable across all problems, suggesting that

this subject was not learning to match the expert's model and that he

was not making effective decisions.

The dbove seems to suggest that several types of learning take place

during an ACTS session, a consideration which will be further investigated ir Year 2.

Decreases in utility differences mean that subjects are learning to

troubleshoot the circuit efficiently with respect to their decisionmaking behavior, and are correlated with the concept of convergence.

Fur uuJect 1, the mean difference drops below .20 on the second problem

ilready and remians there for all the other problems; thus, Subject 1

converged to the expert's model almost immediately, matching her decision beniviur to tht of the expert. It is equally clear from Figure

J-1 tha: JubcLt 2 did not converge: the mean difference dropped once to

less lnan .2U, but tnln :ncreased again; Subject 2 therefore, did not

ledrn the appropriate decision behavior. The curve for Subject 3 appears -u tund towards convergence in the first session, but then the

difference increases again on problem 7; during the second session, on

the Other hand, the mean utility difference decreases steadily, and subJect 3 dld dchieve convergence.

3-16

F-117 - - ~~WO-,-- S
3.5.6 Time Per Problem. 6f the five hours allotted to each session,

approximately one hour was used up for the instruction and the prelimminary objective. Even though there is a real time clock associated

WiLh the [Irterdata 70, software considerations, mostly a lack of suffiClelnt computer memory, precluded the use of it. As mentioned, therefore, tne experimenter clocked the amount of time needed for each problem. Tlis provides a perfectly adequate measure, given the range of

times needed to isolate a fault.

Tiiae per problem varied widely across subjects, but decreased dramatically over problems for all subjects, the greatest decrease being observed between the first and second problem. A decrease over the two

sessions is also clearly evident, as shown in column 6 of Table 3-1 and

in Table 3-3 which gives the mean time per problem in the two sessions

for each subject. The actual number of minutes spent on each problem is

shuwn in Figure 3-b for each subject. The two sessions are indicated by

a break in tliu curves for each subject. Since the number of problems

solved virieo across subjects, the breaks do not occur at the same place

fur each Subject.

Figure >-5 clearly shows the sharp decrease in the amount of time needed

per problem, especially over the first two problems for Subjects 1 and

.. Subject 2 required so much time for each problem in Session 1 that

he was only able to solve three problems in the allotted time (approximdtely four hours). However, much saving can be observed at the beginning )f Session 2 (from 45 minutes to 10 minutes).

t91 These decreases in tLime and savings over the two sessions are primarily I,

attributable to increases in familiarity with the system, including the

consoles, the types of allowable measurements, and the responses to be

made. This is evident from the sharp drop observed after problem 1. It

3-17
TABLE 3-3

AVERAGE TIME PER PROBLEM

NUMBER OF NUMBER OF

SUBJECT SESSION PROBLEMS SESSION PROBLEMS

NUMBER 1SOLVED 2 SOLVED

1 27min 4 15min 3

2 70min 3 10min 4

3 23min 7 14min 5

3-18
* RANDOM

FIXED

120J SUBJECT 01

SUBJECT #3

.... SUBJECT #3

90 4 1

85

80

75

70

65

V) 60

* ~ 55

z~50

45

40

35

30

25 ;

20 **

is * **. x.....

10

1 2 3 4 5 6 7 8 9 10 11 12

PROBLEM ORDER

FIGURE 3-5.

NUMBER OF MINUTES SPENT ON EACH PROBLEM

3-19

him
Is not attributable to the difficulty level of the problems, since the

difficulty level for Subjects 2 and 3 increased over time, and was ranoom for Subject i. It is clear that for the Year 2 study, a few problems must be presented to all subjects to familiarize them with the

operation of the system if meaningful evaluations are to be obtained

from the various experimental groups.

Number of Problems Solved. Column 7 of Table 3-1 gives the number of

problems solved by each subject. As with the other measures, this

varied a great deal over subjects. Two factors contributed to this.

First, t)here was a time limitation as discussed earlier. Thus Subject

L, whose mean time for Session 1 was 70 minutes per problem, was only

able to solve three problems during that time, in contrast to Subject 3

,no solved seven problems. Second, the parameters of the program were

such thut wlhn a subject's utilities converged - thie subject was declarea an expert, and the program stopped. This is what happened in the

case of Subject I who learned very rapidly to make efficient and consistent decisions, and thus become an expert within a few trials.

Stopping the program when a subject becomes an expert may provide an efflclent way of using the ACTS in an actual training situation. For experimental and evaluation purposes, however, it is necessary to have a

great deal of data that are comparable across subjects. For this reasol, Inis option will be changed in the Year 2 study. Even if subjects

u becoiie experts, they will continue to solve problems until the number

uf problems specified in the experimental design have been run.

, . iebriefing Questionnaire. The subjects were given a debriefing

quesLiunnaire at the end of the first session, as shown in Table 3-4.

in general, all subjects felt that they had learned to optimize the

types of ii;easuremenLs to make in order to isolate the fault as effi3-20

JA.
TABLE 3-4

DEBRIEFING QUESTIONNAIRE

1. Do you feel you learned anything? _____ If so, what? ______

If not, what would you have liked to learn? _____________

2. Were you confused about anything? _ ___What? __________

3. Did you find the "Help" option useful? ______________

4. In general, did you agree with the expert's suggestions? ______

5. Would you have liked more help? ___ ________________

If yes, for what actions?

In what format?____________________________

6. Please make any comments or suggestions you feel may be appropriate.

We appreciate your contribution to, and participation in, this experiment. Thank you.

-. ~~~~~3-21..... .. . ~7
citntly us possible. However, all subjects also answered positively to

question i2, asking if they were confused about anything. One subject

uic not understand what the expert's choices were based on, another subject felt that the "computer Was inconsistent with its input," and the

tLhiru subject wondered about the purpose of the modules and their relationship to each other. They all felt the "Help" option was useful and,

ini general, agreed with the expert's choices. Only one subject (Subject

L) old not want help; the other two would have liked more help in understanoing the information provided by the expert, perhaps in the form of

an example in the handout, and an explanation of how the expert's

choices were arrived at. These problems are the result of the abbreviatea use of preliminary objectives and will be easily remedied in the

future.

3.o Discussion

5.6.1 Learning Effects. Overall, the results appear promising in

terms of the training potential of the ACTS. A great deal of learning

appears to have taken place on all performance measures and much of this

learnirng transferred from one session to the next. This is encouraging,

urISldtUlrn TLhal the two sessions were separated by at least one week

fur all subjects. Even Subject 2, whose utilities did not converge to

hose of the expert, snows transfer of learning on the amount of help

used ano the time taken per problem. Subjects 1 and 3 exhibited learninj and transfer on all performance measures.

From the figures, it is clear that a great deal of the learning occurs

in the initial trials, and this is probably more likely to be related to

the processes Of familiarization with the equipment and the task, than

L O the dCqUIsition of decision-making strategies. However, the figures

also suggest that this familiarization process does not take more than

3-22

i"

~w -wvs~ - -- 
one or two trials. In order to eliminate these effects in the Year 2

study therefore, the first session will begin with the presentation of

one or two randomly selected problems which will serve to familiarize

the student with the mechanics of using the ACTS, the feedback format,

and the various types of help.

. Sequencing Effects. The question of different effects attribuLable to the sequencing of problems cannot be answered at this time,

since the oifferences in the subjects' abilities were so large and so

few subjects were available. Effects of sequencing will depend a great

aeal on the accuracy with which problem difficulty can be identified.

From Lhe results, it apears that some problems which were identified as

somewnd difficult, were in fact easy for the subjects to solve. There

aru at least two confounding factors which impinge on the interpretation

of the problem cifficulty. One will be discussed below under "Use of

hrldout', tile other is concerned with the level of the students' abili-

.y: for college-level students, having some background in electronics,

problems iuentified as being of medium difficulty may in fact be as simple s tile easy problems, since difficulty is related to the logical

complexity in the circuit, and college students have had much practice

in dealing with logical complexity. This may not be true for high

school students who may have had less experience in dealing with logical

coInp I t1 -y.

,n order to avoid this problem in the Year 2 study, a base rate for

problem djifficulty will be established using performance on the random

suquence. ,M problem's level of difficulty, therefore, will be empirically rather than logically determined. Although this violates the

stanoard practice of random assignment of subjects to groups, the benefits of this approdch outweigh the dangers of partially violating the

rdnuoo assignment directive.

3-23
'..3 Use of Handout. The first factor mentioned above, which would

prevent evaluating sequence effects, has to do with the use of the hanaouts, and especially, the availability of Table 2 of this handout,

showing the match between module faults and measurement set. This table

shows exactly wnat types of measurements should be made, given different

types of initial outcomes, so that a bright student can in fact use this

table exclusively, without ever using the help option. All the students

in tne pilot study oiscovered this quite rapidly.

The use of Such a handout is acceptable as a training device, but there

are at least two considerations to keep in mind. The first is that the

, CTS only incorporates single faults and the characteristic measurements

obtained are fairly straight forward. In an actual troubleshooting environment, however, there is no guarantee that a faulty circuit will

nave a single fault; a breakdown could be the result of a combination of

faulty modules. Ubviously, in such a case, the measurements would not

be as straightforw'ard as those listed in Table 2 of the handout. Measurument outcomes in the ACTS are given as high, normal, low, or zero,

out in an actual environment, Ine measurement outcomes are numbers which

fldve to be evaludted. For example, an outcome that is slightly above

norml cUula in fact be normal or high. This type of decision cannot be

trained by the current version of the ACTS, and its possible effect in

evdluuting transfer to actual equipment will have to be kept in mind.

The second consideration in the use of handouts concerns the types of

learniny that can take place during an ACTS session. On the one hand,

4 one may emphasize the cognitive aspects of training to troubleshoot a

circuit, in this case, it is advisable to use as many cognitive aids

such as handouts and charts) as possible and to emphasize background

knowledge. On the other hand, the ACTS can also be viewed as a more

mechanistic training device where responses become differentially rein3-24

Q ... i .
forced through a conditioning process. That is, it is quite conceivable

that the appropriate responses, those which correctly utilize the attribute information provided, will become trained simply as a result of repeated trials with fairly immediate feedback. In this case, the use of

cognitive aids would not be beneficial. Thus, whether handouts or other

aids are used in conjunction wit' the ACTS must be determined in relation to the cognitive abilities of the students using the ACTS and to

tie training objectives of the school.

For the Year 2 study, a combination of these two extremes has been selected.

The handout will be made available to the subjects during initial training,

for approximately three problems, to give the subjects an opportunity to

understand the relationship between measurements and module faults. Further training, however, will be done without the handout.

3.6.4 Sliding Window. Aside from the improvement already discussed,

a change in the sliding window was made. The sliding window refers to

the number of problems over which the training weights are calculated.

The sliding window of size N consists of training information for the last

N choices selected by the student during decision cycles where help was

not invoked. Decision cycles involving help disallow any student model

training and thus are not involved in constructing the window. Two

statistics are available from the window: (a) total number of times

training occurred out of N, and (b) sum of the absolute values of all

adjustments to the model for these N decisions.

In the pilot study, the sliding window parameter was 3, so that the utility

weights were calculated for problems 1-3, 2-4, and so forth. This tends

to produce very rapid convergence, with some probability of spurious

results. It was therefore decided to make the sliding window dynamic,

3-25

S - ,.-.. . ..- -...
which means that the parameter can be changed according to current needs.

For demonstration purposes, for example, when rapid convergence is desirable, the size of the window can be made small, but during an experiment

or for training, the parameter can be accordingly increased.

3.6.5 New Performance Report Printout. On the basis of the pilot

studies performed, the performance report has been completely revised,

primarily to obtain more detail of the subjects behavior. The new report

is shown in Table 3-5. Table 3-6 explicates the report. For each action,

the student's considerations, attribute levels and expert's choices will

be printed out; the actual action selected and its outcome will be shown,

as well as ongoing changes in the student's utilities. This differs from

the pilot study in that information is obtained after each action, rather

than just at the end of each problem. Overall, this report provides a

performance record which traces every decision make by the ACTS student.

i'3-

!7 3-26
TABLE 3-5

PRINT-OUT REQUIREMENTS FOR

ACTS80 TROUBLESHOOTING OBJECTIVE

STUDENT: AGE: DATE:

CIRCUIT: FAULT:

HELP AVAILABLE: FEEDBACK AVAILABLE:

SLIDING WINDOW:

STUDENT INPUT SEQUENCE:

STUDENT'S ATTRIBUTE EXPERT'S CHOICES

CONSIDERATIONS LEVELS (WHEN "HELP" IS USED)

ACTION SELECTED: UTILITY RATIOS:

ACTION OUTCOME: SLIDING WINDOW PERFORM4ACE:

SUBSEQUENT UTILITIES: CLOSENESS TO EXPERT:

UTILITY DIFFERENCES: CONVERGENCE:

TYPE OF HELP:

(PRINT AFTER EACH ACTION)

STUDENT WEIGHTS:

STUDENT COST:

EXPERT COST:

TIME IN MINUTES: I

2

"TIME" , TO BE FILLED OUT BY EXPERIMENTER

-EXTRA SPACE FOR OTHER COMMENTS

-1

3-27
TABLE 3-6

REPORT CONTENT DEFINITION

The information content for each label in the ACTSBO Troubleshooting

Printout is described below:

STUDENT Student's name

AGE Student's age

DATE Date of experiment

CIRCUIT - !P-28 power supply

2- A9000 power suoply

FAULT FaUlt 'o. 'or current problem

HELP AVAILABLE 0 - no

I - yes

FEEDBACX AVAILABLE 3 - no

1 - yes

SLIDING WINDOW The no., n, of most recent action choices used to

check for consistency in student model

STUDENT'S CONSIDERATIONS The names of :he actions that the student considered

ATTRIBUTE LEVELS The ittribute levels or the student's considerations

EXPERT'S ZHOICES The expert's considerations by name

ACTION SELECTED The name of the action thosen by the student

ACTION OUTCOME The outcome of the action oerformed (e.g., -L"

or 'ow)

SUBSEQUENT UTILITIES The student's moael values after the action

UT:L2TY DIFFERENCES The Jiiereices between the exoerts and the

stucent's model values

UTTL:TY RATIOS The at'os of the student's utilities

SL:DING dINDOW The amount of 'raining occurring over the most

PERFORMANCE recent n dec sions

CLJEENESS TO EXPERT 7he cosine of the angle between the expert's and

student's elgnt vectors

'ONVERGENCE Yes Small amount of training of student's model

over 5lioinq window

No - Larger amount of 'raining of student's model

over slhiinq 4indow

TYPE OF HELP - Expert's considerations requested 2 - Exoert's tradeoffs for expert's considerations

'equested

l3 - Expert's tradeoffs for student's considerations

reauested 4 - Expert's ranking of student's considerations

V reauested

STUDENT 4EIGHTS Student's model weights

STUDENT COST Student's cost for current problem

EXPERT COST Expert s would-be cost for current problem

4,

*17
4. REFERENCES

Atkinson, R.C., October, 1972, Ingredients for a Theory of Instruction,

American Psychologist, 27(10):921-931.

Bloom, B.S., 1968, Learning for Mastery, Evaluation Comment, (UCLA

Center for the Study of Evaluation), 1(2).

Brown, J.S., Burton, R.R. and Bell, A.G., March, 1974, SOPHIE: A Sophisticated Instructional Environment for Teaching Electronic Troubleshooting. Cambridge, MA: Bolt Beranek and Newman, Inc., Technical

Report BBN No. 2790.

Carbonell, J.R., 1970, Al in CAI: An Artificial-Intelligence Approach

to Computer-Assisted Instruction, IEEE Transactions on Man-Machine

Systems, MMS-II(4), 190-202.

Crawford, D.G. and Ragsdale, R.G., July, 1969, Individualed QuasiInstructional Systems for the 70's, Working paper, Ontario Institute

for Studies in Education, University of Toronto.

Crooks, W.H., Kuppin, M.A. and Freedy, A., January, 1977, Application

of Adaptive Decision Aiding Systems to Computer-Assisted Instruction:

Adaptive Computerized Training System (ACTS), Woodland Hills, CA:

Perceptronics, Inc., Technical Report PATR-l028-77-1.

Dawes, R.M., 1970, Graduate Admission: A Case Study, Oregon Research

Institute, Technical Report 10 (1).

Freedy, A. and Crooks, W.H., April 7-10, 1975, Use of an Adaptive

Decision Model in Computer-Assisted Training of Electronic Troubleshooting.

Proceedings of the Conference on New Concepts in Maintenance Training

Research, Orlando, Florida.

Glaser, R., 1965, Psychology and Instructional Technology, In R. Glaser

(ed.) Training Research in Education, New York: Wiley.

4 Goldberg, L.R., 1970, Man vs. Model of Man: A Rationale Plus Some

Evidence for a iethod of Improving Upon Clinical Inferences, Psychological Bulletin, 73: 422-432.

Hdrtley, J.R. and Sleeman, D.H., 1973, Towards More Intelligent Teaching

Systems, International Journal of Man-Machine Studies, 5, 215-236.

4-I

- TWA ~~~~. ~ ~ ~ , 7 7 . .*, ''~r---'m..
Koffman, E.B. and Blount, S.E., 1974, A Modular System for Generative

CAI in Machine-Language Programming, IEEE Transactions on Systems, Man

and Cybernetics, SMC-4(4), 335-343.

Nilsson, N.J., 1965, Learning Machines, New York: McGraw Hill.

Slagle, J.R., 1971, Artificial Intelligence: The Heuristic Programming

Approach, New York: McGraw Hill.

Smallwood, R.D., 1962, A Decision Structure for Teaching Machines,

Cambridge, MA: MIT Press.

Smallwood, R.D., 1971, The Analysis of Economic Teaching Strategies for

a Simple Learning Model, J. of Math. Psychol., 8:285-301.

,1

4-2

Ii' ,| m a , , : , .ow":
APPENDIX A

STUDENT HANDOUT

Introduction

You are a subject in an experimental program designed to help develop

computer-assisted instruction for teaching electronic troubleshooting.

The system enables you to get actual troubleshooting experience in a

simulated environment. You will work with a computer terminal which has

a keyboard and a CRT display with graphics capability. The goal of the

current experimental series is to evaluate the teaching effectiveness of

the system under various experimental configurations. Specifically, we're

interested in how much and how fast you learn as you work with the system.

On-line measures will be used to evaluate your learning progress and to

adjust further instruction to your learning needs.

You will be working with the Adaptive Computerized Training System (ACTS).

The ACTS is adaptive in that it learns the troubleshooting behavior of the

student. For this reason it is important that you be as consistent as

possible in your approach to troubleshooting. The more consistent you

are, the faster ACTS will be able to learn your particular values and

give instructional feedback which is responsive to your learning needs.

Included here-in is an introduction to power supplies, which you should

study in order to learn the background material required for your on-line

* troubleshooting practice. Following the introduction will be a troubleshooting guide, which is to be used for reference during troubleshootinq.

A-i
INTRODUCTION TO POWER SUPPLIES

A power supply is a source of electrical energy. It supplies the energy

needed by various circuits and electrical devices to function. These circuits and electrical devices, being users of electrical energy, are known

as loads. The purpose of a power supply is to provide electrical energy

or power for use by a load.

A power supply may be either self-contained, such as a battery, or may

transform one form of electrical energy into other, more useful forms.

In this discussion, we are concerned with the latter.

Electrical energy exists in many different forms and levels. Two primary

forms are alternating current, or A.C., and direct current, or D.C. A.C.

is characterized by a constantly changing wave of electrical energy; whereas D.C. is characterized by a constant, non-changing level of electrical

energy. The magnitude or level of an A.C. or D.C. signal may be of any

numerical value, an example being a high level (100), or a low level (10).

Often, the energy requirement of form and magnitude of a load cannot be

met by the available source of power. An example of this would be a load

requiring a low level D.C. with an available source of power of high level

A.C. A power supply may be used to convert the high level A.C. to the low

level D.C. required by the load. Such a power supply would have an input

to which the source of power would be applied, and an output through which

the converted D.C. would be supplied to the load. A power supply may be

used to match the requirements of a load with the characteristics of an

available power source.

A-

~A-2
INTRODUCTION TO THE IP-28 POWER SUPPLY

The IP-28 power supply is designed to perform specific operations under

specific conditions. Understanding these operations and conditions will

help in understanding the IP-28 power supply.

The IP-28 converts a single A.C. input into a single D.C. output. All

power used for operation of the IP-28 comes from the single A.C. input.

All power used by the load comes from the single D.C. output.

Certain characteristics of the IP-28's D.C. output are maintained, or

regulated, at a constant level. This regulation is achieved in the IP-28

by monitoring the output characteristics, and acting to correct any changes.

The output characteristic of voltage, or level of electrical energy, and

the output characteristic of current, or flow of electrical energy, are

both regulated in this way. Because of this, the output of the IP-28 is

said to be current and voltage regulated.

The level at which the output voltage and current is regulated is adjustable.

Both voltage and current may be adjusted to regulate at high or low. The

value of the load dictates whether voltage or current regulation is taking

place. Should the load drain excessive current from the output at the

regulated voltage value, the current would be regulated. Otherwise, the

output voltage would be regulated.

FUNCTIONAL BLOCKS

Within the IP-28, three major functional blocks, the D.C. power source,

the reference source, and the regulator (see Figure 1) operate and

interact to convert power from the A.C. input into an adjustable D.C. output. Each functional block performs specific operations, which we will now

consider.

A- 3
0

CZl

LA- -
The D.C. power source converts the A.C. input power to D.C. The D.C. output of the D.C. power source consists of the common line and the connection

to the regulator.

The reference source provides an adjustable D.C. output. This adjustable

D.C. output is connected to the regulator and an output line. The reference

source converts A.C. power carried by the two lines connected to the D.C.

power source to the adjustable D.C. output.

The output of the regulator is current or voltage regulated D.C. and is

connected to the output of the IP-28. The regulator draws D.C. power from

the output of the D.C. power source and passes to the output of the IP-28

the proper D.C. voltage or current level. The voltage level between the

line from the reference source to the regulator and common, called the

feedback voltage, is used by the regulator to regulate the output voltage.

The feedback voltage level is the result of the difference between the

output voltage level and the reference source D.C. output voltage level.

Changes in the output voltage or the reference source output voltage will

cause the regulator to make an opposite change in the output voltage.

The regulated output voltage level can be changed by adjusting the reference

source output voltage. The regulated output voltage level can be kept

constant if changes in load conditions do not result in an excessive output

current level.

74

) D.C. POWER SOURCE MODULES

Each of the functional blocks of the IP-28 can be divided into modules,

or small functional units. The subject of this discussion is the modular

makeup of the D.C. power source and how these modules operate and interact

to perform as the D.C. power source functional block.

A-5

-!- ' - . ....... ...... . -. 
The D.C. power source, as a major functional unit of the IP-28 power

supply, performs the task of converting the A.C. input power to D.C. The

D.C. output of the D.C. power source provides D.C. power to be used by

the regulator and to be passed to the output.

The function of the D.C. power source is performed by two modules: the

transformer and the rectifier and filter.

The transformer, drawing upon the A.C. input, produces an A.C. output

whose voltage and configuration is compatible with the rectifier and

filter module. The output of the transformer consists of three lines as

displayed in the diagram, Figure 1. The center line is the common line,

and the A.C. voltaqe between the common center line and the two outer

lines provides A.C. voltage to the rectifier and filter.

The rectifier and filter convert the A.C. power from the transformer's

output to D.C. The output of the rectifier and filter consists of two

lines, one connected directly to the regulator and another connected to

the regulator and the output common terminal. It is through these connections that the D.C. power output of the rectifier and filter is

delivered to the regulator.

REFERENCE SOURCE MODULES

The reference source performs the task of providing an adjustable D.C.

voltage needed by the regulator for voltage regulation. Modules contained

within the reference source work together to perform this task.

The reference supply is made up of the following modules:

A-6

Z1I7
Transformer

Reference Rectifier and Filter (REF RECT & FILTER)

Reference Regulator (REF REG)

Output Stage

Refer to the diagram (Figure 1) and recognize their placement and interconnections.

The reference source shares the transformer with the D.C. power source.

The output of the transformer connected to the reference rectifier and

filter is A.C. and is separate from the other transformer output. The

transformer performs for the reference supply the task of converting the

A.C. input voltage to an A.C. voltage needed by the reference rectifier

and filter.

The reference rectifier and filter converts the A.C. from the transformer

to D.C. The output of the reference rectifier and filter, consisting of

the line connected to the reference regulator and the reference common

(REF COM) line, supplies D.C. to the reference regulator.

The reference regulator draws upon the D.C. from the reference rectifier

and filter and produces a constant D.C. voltage output needed by the output stage.

The line connecting the reference regulator to the reference common line,

as it is also with the reference rectifier and filter, is a common input

and output line to that module.

The output stage uses the constant D.C. voltage from the reference regulator

to produce an adjustable D.C. voltage output. Within the output stage is

a means to adjust the D.C. voltage between the line connected from the

output stage to the output line, and the line connected to the regulator.

A-7

F J -- -.
In summary, the transformer converts the A.C. input voltage; the reference

rectifier and filter changes the converted A.C. to D.C.; the reference

regulator stabilizes the D.C.; and the output stage provides a means for

adjusting the D.C. These actions perform the task required of the

reference source.

REGULATION MODULES

The regulator passes the proper amount of D.C. voltage or current from the

D.C. power source to the output. The regulator has two modes of operation,

either voltage regulation or current regulation.

1. VOLTAGE REGULATION MODULES

The modules contained within the regulator which perform voltage regulation

are:

Current source (CURR SOURCE)

Series regulator (SERIES REG)

Voltage limiter (VOLT LIMITER)

Refer to the diagram, (Figure 1) and recognize their placement and interconnection.

The current source provides the proper flow of current to the series regulator and to the voltage limiter to insure their operation. The current

source draws current from the D.C. power source and outputs a constant and

proper amount of current to the series regulator and voltage limiter.

The series regulator is placed between the output and the D.C. power source

so that the power to the output can be controlled. The output voltage

level of the series regulator is controlled by, and proportional to, the

A-8
voltage level between the line connected to the base of the series regulator and common, called the control voltage or control signal.

The voltage limiter conducts different amounts of current from the current

source. The more current conducted by the voltage limiter, the lower the

control voltage.

The amount of current conducted by the voltage limiter is controlled by

the feedback voltage level between the feedback signal connection to the

voltage limiter and common. The higher the feedback voltage the higher

the current conducted by the voltage limiter and the lower the control

voltage.

2. CURRENT REGULATION MODULES

The regulator performs both current and voltage regulation. In addition

to the current source, the series regulator, and the voltage limiter, the

regulator contains the current sense and current limiter modules.

Current regulation or limiting is the result of the current limiter module

causing a drop in the control signal voltage across the current sense.

This voltage drop across the current sense is directly proportional to the

output current. When the output current is at a certain level, the voltaae

drop across the current sense will be high enough to cause the current

limiter to reduce the control signal. The output current level at which

this occurs, called the current limit level, can be changed by means of

an adjustment within the current sense module. When the output current is

less than the current limit level, the voltage drop across the current

sense will not cause the current limiter to reduce the control voltage.

A-9

....
When the output current is below the current limit level, the current

limiter has no effect on output voltage regulation as performed by the

current source, the series regulator, and the voltage limiter. If the

output current is greater than the current limit level, the current limiter

will lower the control signal voltage by conducting current from the

current source through the load to common. The resulting drop in the control signal voltage will cause the series regulator to lower the output

voltage. The decrease in output voltage will lower the output current

until it is at the current limit level.

i.

I

* A-10

6mo.

... . - m.r-..,- - w '..b- --.- I

---- aI
TROUBLESHOOTING GUIDE

The display will be as shown in Figure 2. Each module is abbreviated with

the three letters brightened on your display and underlined in Figure 2.

When you must refer to a module (as when you replace a module), it is

necessary to use the three letter abbreviations. The tables which follow

also use these abbreviations. Notice that various test points are represented on the diagram (e.g., TP1, TP2, etc.). It is necessary to use these

three character test point labels to take circuit measurements. The table

in the lower left portion of the display (see Figure 2) lists the measurements with three columns corresponding to the three types of measurements:

voltage, current and resistance. The outcome of each measurement you take

is listed in the appropriate place in the table. Measurement results are

presented in a semi-interpreted form to save you the trouble of consulting

a table of normal values. Thus all measurements have one of the following

outcomes.

Normal (N or blank)

Too High (H)

Too Low (L)

Zero

Table I lists each fault and its probability of occurrence. A good troubleshooter takes these probabilities into account while making troubleshooting

decisions. Table I uses the module abbreviations menticned earlier as

well as the following abbreviations to refer to failures.

4&.

OPN = Open

SHlT = Short

OUT : Output

TRAN Transistor

RES = Resistor

A-11

w -T - . ,m 77 -T A-4: , .~ T" . '
-00

S II

I< I

A12
TABLE 1

MODULE FAULT PROBABILITIES

MODULE FAULTS PROBABILITY OF

MODULE FAILURE ________

LIM 0PN TRAN .11

LIM SHUT TRAN .09

SER OPN RES .05

RTA 0PN RES .06

STA SHT RES .05

TR OPN OUT .03

TRA 0 ROU .0

VOL OPN RS .04

VOL SHT .09

A- 13
Table 2 gives the correspondence between faults and measurement outcomes.

The abbreviations used in this table were explained earlier. The information presented in Table 2 represents a substantial amount of the information used by the ACTS' expert model to troubleshoot the circuit. You

should find it useful when considering measurements.

Table 3 gives the list of recognized responses and their corresponding

measurement cost or replacement cost. (When a module is replaced it is

brightened on the display as you will see the first time you replace a

module.)

A- 14

-!A -
I ~

Iii

I ~

-s

I I

Ill II I I 3 I Mi

I MiMiMi

01 ~ j Mi Mi ~I a I Mi - 03'b- ~ I oJ I

~I...SIMi Mi ~ Mi Mi

,Ii Mi 'I! Mi wC.Mi,. - - ZMiiMi. - ~.1, I

Mi

I I

I-) ii;

C\.j I - IIIIII C

LU I ' I I I -

I; ;I ii

I 3

MIMi p MIMI i~Mi~

II I I 0

Mi

I I II -- ~, Mi I *IJ r.. 0 MI ~iI~MiMitI,

~~l

0

I ** 'I

~ -~! I, 1;: 0 I ~i'iMi!MiIMi * .S]Mi)..iIMij C !i' z~

* ~ ~I I -2 I I

* II I

9- ~ I-s. 2-

I II MiMi

'9- I I Mi

Mi HIM j..i jM '-0.

' 'Mi ~I

~ 0.

.~;zIzIz~I ~ 2

Mi.S~'M~ 013 - ~ ~'~I '3=

'~

1~

A-15

'S.
TABLE 3

LIST OF RECOGNIZED RESPONSES AND THEIR COST:

TP1ACVR $a TP1ACCR $4 TP1REPO $10 TRA $98

TP2ACVR $4 TP2ACCR $4 TP2REPO $10 REC $70

TP3ACVR $4 TP3DCCR $4 TP3REPO $10 SOU $90

TP4DCVR $8 TP4DCCR $8 TP4REPO $10 VOL $80

TP5DCVR $8 TP5DCCR $~3 TP5REPO $10 LIM $80

TP6DCVR $4 TP6DCCR $4 TP6REPO $10 STA $50

TP7ACVR $4 TP7ACCR $4 TP7REPO $10 SER $90

TP8DCVR $4 TP8DCCR $4 TP8REPO $10 SEN $50

TP9DCVR $4 TP9DCCR $4 TP9REPO $10 REG $70

TPADCVR $8 TPADCCR $8 TPAREPO $10 REF $70

A- 16
ATTRIBUTES

There are three attributes which should be used as criteria for selecting

an action:

1. Decrease in uncertainty. This is the decimal fraction (i.e.,

a number between 0 and 1) representing the proportion of faults

expected to be eliminated by the considered action.

2. Expected faulty module isolation. This is the decimal fraction

(i.e., a number between 0 and 1) representing the proportion of

faulty modules expected to be eliminated by the current action.

3. Cost. This is the cost of the action.

A-17

aL!AL
APPENDIX B

SAMPLE INSTRUCTIONAL SEQUENCES

2.7 PRELIMINARY LECTURE

A lecture covering the overall functional requirements of the IP28 Power

Supply is displayed. Also displayed is a diagram relating input and output connections.

A.C. [P28 POWER SUPPLY D.C. LOAD

I NP11T 

OUTPUT

INTRODUCTION TO THE IP28 POWER SUPPLY

THE IP28 POWER SUPPLY IS DESIGNED TO PERFORM SPECIFIC

OPERATIONS UNDER SPECIFIC CONDITIONS. AN UNDERSTANDING OF

THESE OPERATIONS AND CONDITIONS WILL HELP YOU TO UNDERSTAND

THE IPZ8 POWER SUPPLY. THE 1P28 CONVERTS A SINGLE A.C INPUT

INTO A SINGLE D.C. OUTPUT. ALL POWER USED TO OPERATE THE IP28

COMES FROM TIE SINGLE A.C INPUT. AFTER THE IP28 POWER SUPPLY

CONVERTS TIlE A.C INPUT INTO D.C., 'rHiE D.C. OUTPUT PROVIDES

ALL THE POWER REQUIRED BY THE LOAD.

TIIE IP28 POWER SUPPLY CONVERTS A SINGLE A.C. INPUT INTO A

SINGLE D.C. OUTPUT REQUIRED BY THE LOAD.

PRESS RETURN TO CONTINUE

I4t

B-1

*1'
Following understanding of the material, the student responds by pressing

the return key, and the lecture units continue.

IA.C. IP28 POWER SUPPLY D.C. LOAD

INPIUT OUTPUT

. . -J

TilE OUTPUT OF THE P28 IS REGULATED. CERTAIN CHARACTERISTICS

OF THE D.C. OUTPUT ARE MAINTAINED, OR REGULATED, TO MAINTAIN A

CONSTANT LEVEL. THIS REGULATION IS ACHIEVED BY MONITORING TIlE

OUTPUT CHARACTERISTICS, AND ACTING TO CORRECT CHANGES. OUTPUT

CHARACTERISTICS OF VOLTAGE (LEVEL OF ELECTRICAL ENERGY) AND

THE CURRENT (FLOW OF ELECTRICAL ENERGY) ARE BOTH REGULATED,

THUS, THE OUTPUT OF T4E 1P28 IS SAID TO BE CURRENT AND VOLTAGE

REGULATED.

THE OUTPUT OF THE IP2R IS REGULATED TO MAINTAIN CONSTANT

VOLTAGE AND CURRENT LEVELS.

PRESS RETURN TO CONTINUE

B-2

AA-."*.
IA.C. IPZ8 POWER SUPPLY D.C. LOAD

INPUT 

OUTPUT

TIHE LEVELS AT WHICH OUTPUT VOLTAGE AND CURRENT ARE REGULATED

CAN BE ADJUSTED. HIGH OR LOW LEVELS MAY BE SELECTED TO

REGULATE BOTH TIlE VOLTAGE AND TIIE CURRENT. TIE VALUE REQUIRED

BY THE LOAD DETERMINES WHETHER VOLTAGE OR CURRENT REGULATION

WILL OCCUR. SHOULD THE LOAD DRAW EXCESSIVE CURRENT FROM THE

OUTPUT, AT THE REGULATED VOLTAGE VALUE, THE CURRENT WOULD BE

REGULATED. OTHERWISE, THE OUTPUT VOLTAGE WOULD BE REGULATED.

REGULATION OF OUTPUT VOLTAGE OR CURRENT ARE ADJUSTABLE AND

DETERMINED BY LOAD REQUIREMENTS.

PRESS RETURN TO CONTINUE

I, B-3
At the end of several units, which comprise the entire instructional

objective, the student's understanding of the concepts discussed in the

lecture is evaluated. The student may select one of the several options

displayed. In this example, the student requests additional information

on concept 42.

'>

A.C. [I2R POWER SUPPLY D.C. LOAD

I NPI'IT OUTPUT

J

YOU 'JTI0ILI1 NOW UINI)ERSTAND TIHE [01.,LOW I NG C(ONCEPTS

(I) I', INPIFS AND OlTIPlITS.

I2 CHARACTERIST ICS OF VOLTAGE AND (CIIRRENI'.

%'0V.1\;l . NID CURRENT CIRCUIT CONNE(TIONS,

1) I'28 RE III.Ar iON,

Si ,IIRUMET OR V(i. rA(E RE(I11ATI oN

1 IllS T'POIN1' S:VERAL OPTIONS ARE AVAIL.ABILE TO YOU,

I. IF 'O1 FUEL. YO1 IINIERS[AN) [A(CIl 01 THE CONCEPTS ABOVE,

rYPE "RETIRN" SNl YOU WI LI. BE G T V : A sIoRT TE.T ON TillS

IATER TAT..

2. II: YOu ARI NOT ' IIRE ABOU!T A C(()N(EPT PRESENTED IN TIlE

LEC'FTuRE, rYPE ill: LETTER "II" I:OL.LOWED BY TIlE NUMBER OF THE

CON:EI'T (I.;., CIIARACTER IST ICS 01' Vol.FA(;E ANT) CTIRRENT) AND

"RETURN" II Is ENTRY WII ['ROVIE)T YOll MORE INFORMATION ABOUT

ITIE CONF PF YOU1 SE, LE T.

I F Y y)1 WIII1 I. IKE TO REREAi) THIS ILECTIRE, TrPE "H" AND

"RTIRN.

FOLLOWINC YOuR RIEVII V Or TIUE IECTIIRE ANT/OR ADD ITIONAL

%AiER 1L.\0 YPEREI1R TRN :TEST TlIF maTII.AL

The student responds to the options by typing "H2", followed by "RETURN."

B-4
Additional lecture material covering concept number 2 is displayed.

First, a statement succinctly stating the concept is displayed, followed

by a detailed example.

A.C. IPZR POWER SUPPLY D.C. LOAD

INPUT OUTPUT

2. ELECTRICAL ENERGY CAN BE EXPRESSED IN TERN'S OF VOLTAGE

AND CURRENT. VOLTAGE DESCRIJES THE MAGNITUDE OF ELECTRICAL

ENERGY AND THE ('IIRRENT, THE FLOW OF ELECTRICAL ENERGY.

THE DIFFERENCE BETWEEN VOLTAGE AND CURRENT IS ANALOGOUS TO

THE WATER BEHIND A HYDRO-ELECTRICAL DAMf AND THE FLOW OF

RELEASED WATER THROUGH THE TURBINES. IN TIlS EXAMPLE, THE

WATER BEHIND THE DAM IS LIKENED TO VOLTAGE, THE GREATER TItE

WATER LEVEL GEHIND THE DAM THE GREATER ITS POTENTIAL FOR

GENERATING ELECTRICAL POWER. THE AMOUNT OF WATER FLOWING

THROCH THE TI)RBINES IS LIKENED TO CURRENT. THUS THE

COMBINATION OF THE HEIGHT 'OF THE WATER (VOLTAGE) AND THE

AMINT FLOWING TIIROIUGI THE TURBINES 1CIFRRENT) PRODUCES

ELECTRICAL POWER,

PRESS CARRIAGE RETURN TO CONTINUE.

B-5

$ ~- -.IPs - -- . . .-
The student is again asked to evaluate his understanding of the concepts.

The options are again the same. This time, the student requests to

receive the test material by typing "RETURN."

A.C. IPZ8 POWER SUPPLY D.C. LOAD

INPUTOUTPUT IN PUT

YOII SHOULD NOW UNDERSTAND THE FOLLOWING CONCEPTS:

(1) IP2. INPUTS AND OUTPUTS,

(2) CHARACTERISTICS OF VOLTAGE AND CURRENT.

(3) VOLTAGE AND CURRENT CIRCUIT CONNECTIONS,

(4) IP28 REGULATION.

(5) CURRENT OR VOLTAGE REGULATION.

AT TIllS POINT SEVERAL OPTIONS ARE AVAILABLE TO YOU,

I. IF YOU FEEL YOU UNDERSTAND EACH (i) TIE CONCEPTS ABOVE,

TYPE "RETU1RN" AND YOU WILL BE GIVEN A SHORT TEST ON THIS

MATERIAL.

2. IF YOU ARE NOT SURF ABOUT A CONCEPT PRESENTED IN THE

LECTURE. rYPE TIIE LETTER "N" FOILOWED BY THE NUMBER OF THE

CONCEPT (E.G., CHARACTERISTICS OF VOLTAGE AND CURRENT) AND

"RETURN" THIS ENTRY WIL.L PROVIDE YOU MORE INFORMATION ABOUT

THE CONCEPT YOU S.ELECT.

3. IF Y01l1 WOTl.D . IKE TO REREAD THIS LECTURE, TYPE "H" AND

"RETURN. "

FOLLOIVNG YOUR REVIEW OF TIlE LECTURE AND/OR ADDITIONAL

MATIR IAI., TYPE "RETIIJN" TO RECEIVE TIIE TST nN T MAT A -

II

6 .-.

IMi _ -.- _ - _ _ -" - " " • T:..,-: .4.,
The first question is presented to the student.

WHICH Or TUE FOLLOWING IS FALSE:

A. THE IP28 HAS A SINGLE A.C. INPUT AND A SINGLE

D.C. OUTPUT.

B. THE INPUT OF TIHE POWER SUPPLY IS WHERE THE

EXTERNAL POWER SOURCE IS APPLIED.

C. TIHE TOTAL NUMBER OF LINES OR CONNECTIONS GOING

INTO AND OUT OF TUE IP28 IS FOUR.

D. THE LOAD DRAWS A.C. POWER FROM THE OUTPUT.

E. NONE OF THE ABOVE.

PLEASE TYPE THE APPROPRIATE LETTER KEY, FOLLOWED BY A CARRIAGE

RETURN, TO GIVE YOUR ANSWER.

The student responds by typing the answer on the keyboard, in this

case, by pressing "D" followed by "RETURN."

B- 7

* '.,,. , . T'.. .~ ...
The answer the student selected was correct. The tested concepts are

restated.

WHICII OF TIlE FOLLOWING IS FALSE:

A. THE IP28 HAS A SINGLE A.C. INPUT AND A SINGLE

D.C. OUTPUT.

B. THE INPUT OF THE POWER SUPPLY IS WHERE THE

EXTERNAL POWER SOURCE IS APPLIED.

C. THE TOTAL NUMBER OF LINES OR CONNECTIONS GOING

INTO AND OUT OF THE IP28 IS FOUR.

D. THE LOAD DRAWS A.C. POWER FROM THE OUTPUT.

E. NONE OF THE ABOVE.

PLEASE TYPE THE APPROPRIATE LETTER KEY, FOLLOWED BY A CARRIAGE

RETURN, TO GIVE YOUR ANSWER,

D.

CORRECT. TIE LOAD DRAWS D.C. POWER FROtl THE OUTPUT, NOT A.C.

THEREFORE, THE FOLLOWING ARE TRUE STATEfENTS:

A. THE IP28 HAS A SINGLE A.C. INPUT AND SINGLE

D.C. OUTPUT.

R. THE INPUT OF THE POWER SUPPLY IS WHERE THE

EXTERNAL POWER SOURCE IS APPLIED.

c. THE TOTAL NUMBER OF LINES GOING INTO AND OUT FROM

THE IP28 IS FOUR.

D. THE LOAD DRAWS D.C. POWER FROM THE IP28 OUTPUT.

PRESS CARRIAGE RETURN TO CONTINUE.

Following recognition of the correct answer, the student responds by

pressing "RETURN," and the second question is presented to the student.

AA

8-8
r THE OUTPUT OF THE IP28 IS NOT;

A. VOLTAGE REGULATED.

B . CURRENT REGULATED.

C. CONNECTED TO THE LOAD.

D. FIXED OR NON-ADJUSTABLE.

E. MADE UP OF TWO LINES.

PLEASE TYPE TIlE APPROPRIATE LETTER KEY, FOLLOWED BY A CARRIAGE

RETURN, TO GIVE YOUR ANSWER.

B-9

LA
The student responds by typing the answer on the keyboard. Assuming

the student responds by pressing "C" followed by "RETURN," the following

display will be presented.

THE OUTPUT OF THE 1P28 IS NOT:

A. VOLTAGE REGULATED.

B. CURRENT REGULATED.

C. CONNECTED TO THE LOAD.

D. FIXED OR NON-ADJUSTABLE.

E. MADE UP OF TWO LINES.

PLEASE TYPE THE APPROPRIATE LETTER KEY, FOLLOWED BY A CARRIAGE

RETURN, TO GIVE YOUR ANSWER.

C.

WRONG. THE 1I11B SUPPLIES D.C. POWER THROUGH TIlE D.C. OUTPUT TO

THE LOAD. THE LOAD DRAWS POWER FROMMIiE IP28'S D.C. OUTPUT.

THEREFORE, IN ORDER FOR TIlE IP28 AND THE LOAD TO WORK AS

INTENDED, TILE OUTPUT OF TUE IP28 IS CONNECTED TO TIE LOAD.

PRESS CARRIAGE RETURN TO CONTINUE.

uB

4--
The answer selected by the student was incorrect. An explanation is

provided explaining why the answer was incorrect. After the student

responds by pressing "RETURN," the third question is presented to the

student.

WHICH OF THE FOLLOWING MUST ALWAYS CHANGE THE OUTPUT VOLTAGE

OF THE IP28:

A. LOWERING THE VALUE OF THE LOAD, CAUSING LESS

POWER TO RE USED.

B. INCREASING 'TiIE VALUE OF THE LOAD, CAUSING MORE

POWER TO BE USED.

C. INCREASING THE CURRENT REGULATION VALUE.

D. LOWERING TiE CURRENT REGULATION VALUE TO ZERO

AND HENCE CAUSING CURRENT REGULATION.

E. INCREASING THE VOLTAGE REGULATION VALUE DURING.

CURRENT REGULATION.

PLEASE TYPE THE APPROPRIATE LETTER KEY, FOLLOWED BY A CARRIAGE

RETURN, TO GIVE YOUR ANSWER.

so,

B-11

-- A-. . ...... .,-
The student responds by pressing "0" followed by "RETURN." Since the

answer the student selected was correct, the correct concept is restated.

F WiICH OF THE FOLLOWING MUST ALWAYS CIIANGE THE OUTPUT VOLTAGE OF THIE IP28:

A. LOWERING THE VALUE OF THE LOAD, CAUSING LESS

POWER TO BE USED.

B. INCREASING THE VALUE OF THE LOAD, CAUSING MORE

POWER TO BE USED.

C. INCREASING THE CURRENT REGULATION VALUE.

1. LOWERING THE CURRENT REGULATION VALUE TO ZERO

AND HENCE CAUSING CURRENT REGULATION.

E. INCREASING THE VOLTAGE REGULATION VALUE DURING.

CURRENT REGULATION.

PLEASE TYPE THE APPROPRIATE LETTER KEY, FOLLOWED BY A CARRIAGE

RETURN, TO GIVE YOUR ANSWER.

D.

CORRECT. ANY ACTION CAUSING THE OUTPUT OF THE IP28 TO CHANGE

FROM A VOLTAGE REGULATION MODE TO A CURRENT REGULATION MODE

WILL CAUSE THE OUTPUT VOLTAGE TO CHANGE. SUCH ACTIONS MAY BE

INCREASING TIUE POWER CONSUMPTION OF THE LOAD AND LOWERING THE

CURRENT REGULATION VALUE.

PRESS CARRIAGE RETURN TO CONTINUE.

B-12

! 7M&
This lecture represents the format used for preliminary units. Other

preliminary objectives may be added as the need arises. During these

lectures, the students are taught and tested in an iterative fashion

until they are familiar with the material and can be advanced to the

troubleshooting objective.

2.3 Troubleshooting Lecture and Presentation

For the troubleshooting objective, the introductory material is presented

in a similar format, namely in short, easily understood paragraphs, which

include an explanation of the troubleshooting task.

i B-13.-
TROUBLESHOOTING INTRODUCTION

YOU WILL NOW HAVE AN OPPORTUNITY TO TRY YOUR HAND AT TROUBLESHOOTING THE IP-28 POWER SUPPLY. TIHE CIRCUIT WILL BE DIS- PLAYED FOR YOU ON THE SCREEN, A SINGLE FAULT WILL BE PRESENT

SOMEWHERE IN THE CIRCUIT, AND THE SYMPTOMS OF TIHE FAULT WILL

BE DISPLAYED BELOW TUE CIRCUIT, YOUR JOB WILL BE TO MAKE ADDITIONAL TESTS UNTIL YOU CAN LOCATE THE FAULTY MODULE AND REPLACE IT.

PRESS "RETURN" TO CONTINUE.

IZ

-t

B-14

''' '
The explanation of the task continues over several frames.

TWO TYPES OF ACTIONS ARE POSSIBLE FOR DIAGNOSING THE FAULT;

1) MEASUREMENTS AND 2) MODULE REPLACEMENTS. EACH OF THESE

ACTIONS WILL RESULT IN DIFFERENT COSTS AND DIFFERENT INFORMATION CONCERNING THE FAULTS. THE OBJECTIVE WILL BE TO EXPEND THE LEAST COSTS TO DIAGNOSE AND FIX THE FAULT.

PRESS "RETURN" TO CONTINUE.

TO GO BACK TO THE PREVIOUS PAGE, PRESS "B" AND THEN "RETURN".

-.

I -1

J]al
SELECTION OF AN ACTION WILL INVOLVE TWO STAGES: (1) DETERMINATION OF FOUR USEFUL ACTIONS THAT COULD BE MADE, AND (2) SELECTION OF ONE OF TIHE FOUR ACTIONS. AFTER TAKING THE

ACTION, THE OUTCOME WILL BE SHOWN ON TIHE CIRCUIT. YOU WILL

CONTINUE TO LIST POSSIBLE ACTIONS AND MAKE CHOICES UNTIL THE

FAULT IS DIAGNOSED AND THE BAD MODULE REPLACED.

PRESS "RETURN" TO CONTINIUE.

TO GO BACK TO THE PREVIOUS PAGE, PRESS "B" AND THEN "RETURN",

B-16

*
IT WAS MENTIONED IN YOUR EARLIER SESSION THAT YOU SHOULD

CHOOSE ACTIONS ON THE BASIS OF THREE FACTORS - UNCERTAINTY

REDUCTION, FAULT ISOLATION, AND COST, THE FIRST OF THESE,

UNCERTAINTY REDUCTION, IS TIlE NUMBER OF POSSIBLE FAULTS THAT

ARE EXPECTED TO BE ELIMINATED BY THIS MEASUREMENT. THE SECOND

FACTOR, FAULT ISOLATION, IS HOW MUCH THE ACTION "CLUSTERS" THE

REMAINING FAULTS INTO FEWER MODULES, THE FINAL FACTOR, COST,

REPRESENTS THE TIME AND MATERIALS REQUIRED BY THE ACTION.

A MODULE REPLACEMENT HAS A HIGH COST; A VOLTAGE, CURRENT,

OR RESISTANCE MEASUREMENT HAS A LOW COST.

PRESS "RETURN" TO CONTINUE,

TO GO BACK TO THE PREVIOUS PAGE, PRESS "B" AND THEN "RETURN",

B-17

T"-A1
TO AID YOU IN THE LEARNING PROCESS, SEVERAL FORMS OF FEEDBACK ARE AVAILABLE TO YOU. rF YOU ARE UNSURE

AS TO WHAT ACTIONS TO CONSIDER, YOU CAN HAVE THE SYSTEM DIS- PLAY A SET OF POSSIBLE ACTIONS AND THE REASONS WHY THEY ARE GOOD. IF YOU ARE UNCERTAIN AS TO WHAT OUTCOMES MAY OCCUR,

YOU CAN REQUEST A DISPLAY OF THE POSSIBLE OUTCOMES. YOUR

STRATEGY OF SELECTING MEASUREMENTS WILL BE ANALYZED AND CORRECTIVE FEEDBACK GIVEN.

PRESS "RETURN" TO BEGIN TROUBLESHOOTING.

- ---- "r,,

7 6. . .... ... . ..- - - ' - 1

I7
Following the introduction to the troubleshooting task, the system generates a modular diagram of a circuit and a fault table. Shown here, is

the diagram of the IP28 power supply, which consists of 10 modules,

divided into three major functional blocks and 10 test points. Circuit

measurements taken at these points can be of three types: voltage,

current, or resistance. Explanations of the modules, their functions,

and the types of measurements that can be made are given to the students

in the form of a handout and troubleshooting guide. During the course

of the troubleshooting objective, this guide may be referred to or not

at the student's discretion.

,4

4 B-19

L.A
I.,T3T UTU

TRANS-A- RECT. SER IES CURR.

ITFR14R FLT R F REG SENSE

TPR TP4 

TP 4 :.

TP 22

TP7 7P

TP 8

TPA

P 8-2

TP

-- T A ,
Each troubleshooting problem consists of a simple fault, which the

student must locate and correct by replacing the faulty module. The

initial output symptoms are displayed in the table.

TRA?1S- -- RECT. 4 SERIES CURR.

FORMER FILTER REG. SENSE

TP 24

TP4 3UR

TI 4I IT

Ti'. vo~.

ITi 6IIT

Ti'CO7

TCOMMO

Ti' 9P P

TP

T8-21

TP.
To begin troubleshooting, the student is asked to select appropriate

actions. The student may select help, if desired, but has not chosen

to do so in the present example.

TRANS- -- RECT. 4 ~ SERIES CURR.

r rORM FLT REF R EGC . SENSE40

IO UT LIIT L

TP 2 RES RTR. OTU

OTP L TP I IFOUEL AEOUDTO NE POSSIBLE ORFU T ACTIONS AE"I , THEN TP 2 PRESS "RETURN".

TP 3

TP A

B-22
When asked to select four measurements or replacements, students may

refer to the list of available actions in the troubleshooting guide, or

they may type in selections without reference to the guide. If choices

are made which are not legitimate actions (e.g., TP5ACVR, instead of

TP5DCVR), the choices are rejected by the system, and new choices need

to be typed in. In the following example, the student types in acceptable measurements and replacements.

I RANS -- RECT. 4 SE.RIES CURR.

(') RC SO 

rP*

* 6

- -- - - RECCIE T .HEUTYOU BLNANTYEAARAG RETUR 

It' :I1 I ICh$ VPIPV T SRTGEP TPA

REERNC COMO

TP9I

TI' A

'A

IT

B-23

&MAIN-,r'
The student's four considerations are listed, along with their attribute

levels, and the student is asked to select one of the actions, using the

listed attribute levels as a guide. The result of the selected measurement is indicated in the table. In the following example, the student

selected "TP4REPO," and the outcome of this measurement was normal.

If the attribute levels for all four considerations appear unsatifactory

to the student, the option is available for rejecting all four considerations

and selecting new ones. In this case, the student would type "N" and then

press "RETURN" and four new actions could be selected.

TP T3 11 OUTPUT

TRANS- RECT. F, SERIES CURR.

FORME FIiiEE F"".' REG -- SIENS.EI

S TP4 UITR

A.-- TPS * I

INPUTI

LOA

COMMON

VOL CUR RES LISTED BELOW ARE TIlE FOUR CONSIDERED ACTIONS AND THEIR

. .. .. -.. rATTR I BUTE L'VELS.

OUTP L I.

TP 1 DECREASE IN FAULT

TP Z ACTION UNCERTAINTY ISOLATION COST

TP 3

TP 4 N 1. TP9DCVR +.40 +.33 +4.00

TP 5 2. TPSDCCR +.40 +.33 +8.00

TP 6 3. TP4REPo 4.41 +.43 10.00

TP 7 4. TRA +.28 +.19 +98.00

TP 8

TP 9 TO SELECT AN ACTION, TYPE ITS NAME, THEN PRESS "RETURN."

TP A TP4REPO

IFa B-24
Measurements continue to be taken in this fashion until the fault has

been isolated. If the student is unsure what to do, "Help" is available

at several points of the troubleshooting phase. In the following

example, the student is again asked to select four actions, but this

time, the student responds by requesting help.

TPI tP3 T P OUTPUT

TRANS- RECT. SURIES CUR ---

FORMER FILTER EG. SENSE

. CUR 

~REFE.RE;NCE COM$MON

V'Ol.q CUr lIES| YOul MII(l SEIECT FOURi %CTr0NS IVII ICfl WVOILD RE IlSEPIIL IN

OUTP.. )" ."- ~ l' INOEITIFY I.'u; A.ND/OR RIEPIR INC TIlE I'%UIT.

I TPT 

TI' 4 N ]FOR HIELP ABOluT POSS IBI.F \IINS TO TAKE " THIEN

TI' S PPFSS "RETIIRN".

TI' TI' 7I

TI' 9

. . VO T P

TI' A

It

..

C B-25

I:~~~~~:. P11 'RE.STG 
The instructor's considerations are displayed in an effort to help the

student make selections.

TilT3TP6 OUTPUT TRAN4S- RECT. F, S:R I ES CURR. ----

- -- -- F LTE IOREO. S(ENSEAC 

OUTI'SOURCE

TI'TP N TENIC4 

Ip 5T

TA. C. TOCVTOLRSSRT.RN". TI' 7I~

TI'~rF SIG TG

TI' 9P

TI' A

RuEECECMO

VO U E NE HS ICIITNES H NTUTRWUDCNIE H

FOLWN nIRATOS

OTB- 26L

TP I . . S . I-
The reasons for the instructor's choices are displayed.

OUT; LP TP3 MU

TI'R LE RG SAES

TI' AN CT A O O STCOE S P B C IN. C STI' 3 1) TP9RCVR L '.2 4/

TI'~TP .6 FT /

TI7 7P N * 7 3/

VOLCU RE TREINSTRTRN OR qFNA THEO ACTIONS BCUEOFTEFL

- ~ ~ LWN ~ RAOS ~ ~ B 27-- 
T RA M S RECT. 4 EIS Ck.-

OMCR FIT ER A R.ELS TEGO CTO SENSERDBTH XET

L P4Th I. rul TS

Ti ACTCONOT. PR SLI. CS

Ti ES J) i TPCI Lrr T.32 OUTPUT

71' 8PESS ' EFRNCTO COMNTE

Tl' A /

TP4I -3 /

TP54 PDCR L+3 /

8- 26
Further help can be obtained at this point, which would provide a list

of the four actions, ranked in terms of their multi-attribute utilities.

If help is not selected, the program requests the student to list four

considerations, which need not be the same as those of the expert, and

then select one of them for implementation. After having taken several

measurements, the student is acain asked to select four possible actions.

TP6 OUTPUT

I gI

FO CRE FTY E FIR FS RE G. SN/RRPAEN TS SP IEO. T

CUR L 7LN-RF 

TP I l"H1 !&4

TI * T I

A. CUR RVSTPOFu M~ LENT N/RRPAEET O. SEART

TP I4

TimpS LII[

TP' 9 N

TIP A

V

i B- 29

F I.- LT REG."ST G. TPA
The student is asked to select an action, using the listed attribute

levels as a guide. The student selects replacement of the faulty module

by typing "TRA" followed by "RETURN."

F rTRANS~~ -*ET UR,----

OIILIMILEL

TI'. IUT L ERAEI ATp

TI'T11 N . TTfCR.4 141.3 4

TV 5 .R DCREG STAGE TPA ..

TI' 6 3. TPRr REC COMMON3 *1.0

TI' 3 L

TV 9 2. TO SELCACTN T'TSN.4 T+.E3 +RSS" 8TR.00

Tr A TRA

B- 30
The module is enhanced on the display to indicate that it has been replaced.

If the replaced module correctly repairs the circuit, the output measurements in the fault table will be changed from low readings to normal

readings, as indicated below. The student also will be given feedback

to the effect that the circuit has been repaired, and the student's cost

involved in troubleshooting the circuit will be compared to that of the

instructor.

VOL .... RT N I TOI TO 3 R TP E OU T OT

-m TO t)F.'I TEC RLC * IA se S 1110

A ~R .C . C T 4:~ ~::iii! I V . E-- C'T, U" It

TP I L :::::: T HE I '4SR I ITR; TOA | CSTAG WOL TPAVERE 1

CQ UR.ENEOMN

TI'4 C

OUTSOURCE

TA 3 PR.SS RETRN" TO CONTINUE. VLL 

TP'

TI' S N

TP' 6 TI' 7 !

TIP 9 N

TP A

B- 31

CF. " 0. '
Comments are displayed to the student stressing considerations to improve

performance.

YRN HCT. 8 S:RIES T CURUTPU

VOLCURRES Y~RE COCSIDCT HTYUAEICNITN NYU - - TRILESIOTc SREG. B E SE AIGACOE, OSDR OVP N C AELL TCUR NERANYREUTOFAL SLAIN N

Ti'Z

TA 7.VLT P

ITI "'8 LMT

TI' 9 N)

TCOMMA

T17T P

II4.F fz:

01 r

1 rFR RG TG P

--- TRULSOTN STRAEGY R MAIN A CHO.E CNSIE
Display of comments to the student continues. The student will now begin

another troubleshooting problem.

1*~ RECT. 4 SERIES CURR

VOL R' CUR YO APE RD T OV RE P 6

- - - -* uNCE~RTANYRDJCXN4FUL SLTO

TI. C. NVRS RE N OLOTINU P

P7 AP TP

.- .., - .. .. E*RE C M O
At the conclusion of each problem, the student's performance data are

printed out on a teletype and a new problem is presented until the

session ends, whether as a result of completing a certain number of

problems, or because the student has become an expert troubleshooter.

An example of an actual print-out for the first two problems of a

session is shown in Table 2-3.

.4

B-34

-- !SEE 4. 
FIGURE 1

ACTS 79 TROUBLESHOOTING OBJECTIVE

PERFORMANCE REPORT (CURRENT FAULT)

STUDENT: AGE: DATE:

CIRCUIT: 1 FAULT: 1

HELP AVAILABLE: YES FEEDBACK AVAILABLE: YES

STUDENT INPUT SEQUENCE: ACTION-OUTCOMES

TP5DCVR L

TP4REPO N

TP3ACVR L

TPIACVR L

TRA F

STUDENT WEIGHTS: +.45 +.88 +.07

EXPERT WEIGHTS: +.70 +.70 0

CONVERGENCE ACHIEVED: NO CLOSE TO EXPERT: NO

UTILITY RATIOS: STUDENT: +5.76 +11.32

UTILITY DIFFERENCES: +.25 -.18 -.08

SLIDING WINDOW PERFORMANCE:

TRAININGS: 1/20 TOTAL ADJUSTMENT:

SIMU. SUBJ. DRIVING WEIGHTS: +1.00 +1.00 +1.00

PROBLEM COST:+O.1240E+03

TOTAL COST: +0.1240E+03

ACTS 79 TROUBLESHOOTING OBJECTIVE

PERFORMANCE REPORT (CURRENT FAULT)

STUDENT: AGE: DATE:

CIRCUIT: 1 FAULT: 2

HELP AVAILABLE: YES FEEDBACK AVAILABLE: YES

STUDENT INPUT SEQUENCE: ACTION-OUTCOMES

TP9DCVR N

TP3ACVR L

TPIACVR N

STUDENT WEIGHTS: +.45 +.89 -.02

EXPERT WEIGHTS: +.70 +.70 0

CONVERGENCE ACHIEVED: NO CLOSE TO EXPERT: NO

UTILITY RATIOS: STUDENT: -19.22 -37.74

UTILITY DIFFERENCES: +.25 -.18 +.01.

0 OW PERFORMANCE:

TRAININGS: 1/20 TOTAL ADJUSTMENT:

SIMU. SUBJ. DRIVING WEIGHTS: +1.00 +1.00 +1.00

)PROBLEM COST: +82.00

TOTAL COST: +82.00

B-35

"~ ~ ~ ~ ~ ~ "-L-7 ~ ~ ~ ~ ~ ,,-"" ~ t ~~ ~~~. : : i., -- : :",, ...... 'w
DISTRIBUTION

*~~ A I I I 

-i t, I_ l, ! A A I I " A I ( 'A I - W ,

*. , j A "'t ,itAk( r, -,Jul jI~L UIJ,

1, 11 1 A IT )(-c :..A IJ Il.AL OL VLL 10i"1 l U I V WCMS1430i. tq k42( 1 idli 4oN tiL

i IA dlI) t A 1'41 L )

- 4 I ('AT A fl rilv I. L L L 14lm' I

I H~j - A I': tAI'M I - I 1,

I iS~ A~v I AI iifl , -, I )*, . (. o m L Al It J'-A -

I I CkWA(PC10 4$ Al \4: AM'ILL-~ A-N'

I,!-owht2.\NHItol' J'1 r A jI4 141 i~uhF S A I IN: jJUt I IW

1' Ak 4-my rthI t. *' . dVi 11' AHMY

,I I [NI, AiN I w I)I v I i S.v ou jo t 1. .Lr 411I A*IeN--lr -

I ,tj~ omIf-jY Ai~L- 114I: #~ PAt~ 04M N c1 j ;

z 1 k~c A, )Q I Cr I - a L I --Nu Y

INA iAL I WA I , I .,)~ ~.I PrO NJ kLN .1 N: Iti-.- I AL L l~b-Ay

I P~ilI IAAI~y 0CCI'PA I I ),JAL 1t-V tL Ul"Mi -N I Ul *& i . AJ-~~)~ t- t MA '40' I

*I A I I [A"Y VCCfIPA I I .II)AL IHrVtLII114h'Jl I V V Id i, A i t. .- Oki -4414&.A -ALI',

1 '4 11$ IrAW.Y (JCUPA I rk,4AL I)t VtLLL)MP 4 I V a I It uillt.-'4u-,* T?. tM~ il

I PI i INf AWT 4ii [Li , I ol

I "' I)A yA~I\4 I'IP.4 , tAIJAutmt, '.1 II.

I H11,1 A AT IN: OA >--JI.-,

1 l1? 41) L'SAHL0lM H, '1--olvL t d~

1 16 1 tit , JAM I Ii PiAl-e 1 ,014 9 I'd 40e I j

I J l1 (_CI k * AI t- 1 4t a1 N L N e i 1 . # ii u

I 'I'dJ IAt4CIM MAHI iNr -_ OHH L I " L WN ul t.

I)Of- 'Ak I -At N1 01- 1 -it w.My U-) AHMY INI LL-i- I..0 *.iLCJWi' II',CJANU f 

I I4., )A rpiiltF . Mt II iILI) A( I i V I I It', UN

I IIS% Mjc' ,ILI- 441,k I r kL-AuIllSSb~ LIMI',Aljd .I IN: UR:%[-NTv

I I ISN I F JCt CU AM I

us mJ ~It.IrHy L ujIil( I ol o.. :-jilN.IUN 0 VL 4 o L uU AL CIPPOW tT UN I TY

'A JAL ( IV I I AN '-h -14NILL Ltu "id t,~il--4N L$4 ii '

L. I ti I ,UI. *)II . i&

il Ak AY I -A I j L t ] tI

rS, ~NU ~ ~~~ N[. ~~~ r 1 14t INUI VI IJAL I NAIil41N'.l tVAI J'AT IkN

I H(-)Ao IiC.-UP'n I (IjV IUUJL JIAINII'sU

I 4(-)Ao I)C'm.PS I - Ai. u6ItLH CIU 44li

I 1,4140A9 I)CSLO'- MA 1i 41 tIjljtLt mANA LMLN I

I HI )A I ( I TUU( )J-Icl:

1 IICUEr W3 A T I N: ;I- I - t.A -L 1 tjMAN l' ALI Ok

I lSA"$AMvNIII AIL/ P-)Z t

*1i I SfP A'.oS~/ I aC agAt- .Y AI)VISuH

I NgT1--nNIV SLr4I'344- UN AlNM"L4 POPLtb 4 IU

I -,AA 1..UA) OtiiLN I tUN 3tLiLNCEIA sNU rLCm..4ot.objy

* I oIr o~i NAVAIL NL,t ANCH

I AV ,RL/1 Hi

I A t'i1L/I RL

I Al~ 1 1 0,Ct P40MAI- NOMU140 t L Ad A IN: At tii / tbi

*.I t to Al4HL /Nil

-. I NAvY PJ-wflINtL t. 101 ) ULLFVILH VIHLCIJ4 m- PHU(jtRAiwq

t IJAVY t W>ONNIL *-ANJI 1) CLIVILH /

e~ (P OF NAVAL Nt'it at-Crl P'tr:,UNNiJ L AN'U IMI11111b RE!3EARC14 I't4I(,AMb

c-1
I flH: UF ilsAVAL Pt tAHCti i~.uItLIO!. P['os * IRAININi PSCH PktJus

I !,' Lfl' ',AVAL, Rf,4.AkCm PHJtLI o II~C, NVIHUNMLNTAL PHY!..IULUi.Y

1 'r4 AL i-USI-PACL 'ii':LAL HN)Lt LA!' A1114; ((;OUL L51)

i NjA AL 4efiUSPfl~t M:_IJCAL HNILt LALV AIk:WUNt HANULH RESFAWC,1

1 4o1-t AIJ 01,t IVAL it.)U~ L bL I lil IIF I C Ai)v I SUN (tPLkS-JN)

I tuA AL itkI-,ALL ('1ICAL llbLlr LAt3 AUi JSPAcF PbIYCOLOUY DEPANWIAtJI

W-- lk!,A~ "YI fl ANALV bI ACILVIIY ATTi: /,dAA-rCA

A'iuta -. IL.t'. k. "sI uuo( Ctlbt-, PSY~irloiI uIAL HSCH i ip

k I A, 1!A1, A 14 T CtNULUUW LAtA A] IN: oAvI)L-AS

P) rU!- I. I Y ji. -' ~ T H U41)1 LUMU Ali I:uNumE-T14

I A TI~. t. HIFI L y-.~ A N'a.L Y AI L I VIY I ,iN: UHASI-Ni

m A 0:#IC tiomp r4 t ra.I 1 t " ~Ijo tiANL,

IH . 11L LI t - C -)L IME)-1 4MUNSIUMIlSt IALTIC4J I .HN I CAL Ut C

I St.~ A~rHClt lit-s t- A I 11,: AMS IL-tHL-1

W,,,~ AH! I I L I t.S I L: A I I N: S I LAL-L -mli

t ol !)i-iLNS!-! L ArtW 'AU, I IS Ii '.t . I %t'L ibN LAocii, A(3L CL.N

I ofi wRATW lIV 0- 10tl0lY lJ I0

WI LY 1Lf C IWUNIL 4vA-o AH. I AD LtiTlt- , 1tILLI M~b OFNC VtL *S.t t

1 015' 'wbr i DF- Vt L + * ~IANLDANU I ZA Ulj9 Uts K

I iFFItL/Fbk (C~L;L)

I tj , N!j1lNC ktSL t4:" .iNti OLvELtWILNl LIJI"Mtil), CltF, HFHAV SLILNCb ULvq tJULJ SCI L~iH

I oA~ I)* A.'J r. it ~~ H ILITAY ASboI i-0 IN6i + !-'EI TLLHNOL

I -i 4,)A

I rvA IaL 1%1 kI- SY IL '1L L IM M'J, A I I A iN-oj i

I lj Cut rt E_. I L~ r l I t L 1_i14r1.tY

I I IS AAL L I HP At-Y

I HI) AN tELUIk t '> H-<L!1 Om,', (UMHPU L i JHAP y

I StVILLI- iL5F.AW%-_ UNI'(JAuTIUN

i IINkUmLLJ w ,tkvi- t-5 1jIlI Ul, Ttit ILAL Iri bC ULiPAHImENT OF PSYClIAT4'

I ki U -I 4N 0A St)Uk?( tA. ") Ht UO' ttiumkwk)

I '-I-iPHo Wt-LSt- 1kJ LitiHAHV

I HATTkLt- NI.POHI ) LItiNAi4y

j pa Ij) CtiH~.A I ILI ) A-IN; LltiNAHY U

I tilk).JIN1,LH IjNM-l AlIN- AIL-Pb-L H-LJ A' I 

i CE jTLW t0£- KNAvAl ANALYSiS)

I tI'AvAL H-LM1.IH N i(ti'J LinHAkY

*I NAtIAL t-H!:SONNLL k INl) U LLh LIHt<AHY AM.i': CUUL. 4201L

I AI-4 FULWCt HUIAN H;_7 WNCt:: LAb ATTN: 0r"-wL/Ul

I .- w. Ff. rIUAC'Jl. x.T1N: IlLk~lI W-I DIV

i w-,. ALAULMY 01, -ALTr! zLIL LLS IiMSJN iMNHANY 0iOCUoMFNTb)

I -C OUL uI- liY')it -t si',u LtIlA'TiLl, AIft,4 At, IT/LSCM

j FRIC t-4(LoSSJ , k' LI-tHNLL -Al_ A.JUIIUINS LIH-NAIJIAN

I JIE0Al)AIlNI Ut- (i-, '"AVY ImAININO3 ANALYS)Is ANO LVALUATION OwtI l"IA! ItNlL C1 1 11-i )I- i rLALIN SIAT AtI ICS /

I UIS NAVY CitF I b''waokN I NLLMNLM Llb(AwY

I IL-) U~miNInN LtNTVziNSITY ltL~tUWHANCL ASN SSMI!NT LAtiONATORY

*I IJSJ COAAMANII tANU li;NkAL 3-1u-F- ('vLLLtit A 1IN: L I dRA1Y I US TNASNUNAII') ~IUI USA ItiANbv It(i krqFU ANO Nct CEN

I NM~litC PHUO.NAM -"'%JAOLi I-Ut HUMA.N ItkHt-JHnW4~Ct

I I lSi AU-11N1t-N I1 LiNICAL kLt)LAItl bkAti.;tt I HkAHY

I its.k F-IFLU) A.4TY 41! /

I ('AT CLPAHIN6H-Ut fION .WNIAL H-wtLlh 141 PAWALAWN tlLofl,

1 U iF to AAb CLN F JiN CUM.IUNICAl JUN kSC'1

I INT [iit -Ok U tI ::t AN.ALfSLS

I IIS.j INalNufJ St3':PF~ Cl-.'ITLt UFVkL SYSIE;45 rNU * I)LVICFS L)INkCTORAIE

I 4-4WiL T L0ttd0 1 io Jf- C (H)~

I P0 ilUt- LIN IV Lt.ol (4f ISYL?IULUtI4AL SLkt.N0-S

I 101. 1 10-4 IfIjYy ti .I sN N A4) 1 LUMMANJ .%TN: Uku'm-Z,

C-2
1 0)4 0US AHMY Rt IKNANU hoiL RLStu11(H * LVIILUAfIoN oIR

I tMii 'AN a'LS0 R NCL liA14~t1L il t-LN. Z)N vIl.uU

I iiS kFA iuLPI U I 0- ANIJ flLt SoLI

I IfS MiLVIAHY AC-t4Y LitiHAHNY

iI I1 INtLLLIl6ENL. ZtN ANio tSLH 41IN: sT~moI LAUNANY

1'S I N I L LI(6t N L f.N A NJ) SLH ILtPl UF ON.'INU SENbOWS

~A I N t L.(I H pS 11",4 111 Ei

fAIjAL QAl.Y Ct~~- /

I IIS CO(sT lbLAPJ lN C N ul IN: i-iULATIJN1441 SVL!S UI FCk

I ISIAVN(- ANIn . kiJCEH Al IN: x I U-LS

I k15 AHMY AVN INU LlirWAHY AITN: LIIILF LIHIARIAN

I IiSl A1w O'HN ,L ChlUL Al IN: A ISA-01

I oyS'AvNr A ITN< .- U

*I w3 MilL iAHv ACA-t~l UAj.i~tIf)Hk) III NSI AljT TONAL N5C~l

I H-tO. USA SLPUI4 ANT1 rAJUH Al.AULMY Al Ia N A nNINU H WSUN(iS, C:tNIt

10 1,ISi INPANIRY r~u;,NJ ATl.-Y AT~ti-jb-IS-r1

i uS ) INT LLI 6-'1L [l-N ANin S-Lti I-L)JUL A IIJNAI AU V bUH

I II. OW0NANCE Lt. Al-U1 s(91 AT I N: A I L- I LiA-.

it' I AH,4(IH '-Ctj(flt. nikIN; il-TJ

I ISA Ak-ttk C(1. It.- tiIN~CfORUNATL oI* CUMHAI i~tvLLUIPMt.~j1

I NAvjAL 'SnAI SLH Ml IN: I)1ULLLY KNkJx LIBRARY (CIF 14?4)

1~ kp. JN(IJ5N1 14 IL'010uUL OLPII I Y AS)S I . ( OMMANUANT LOUCl(A* TLCHNI)LU -Y

I i IS n,oAL S( MUlL AtI'jU I-i. IONU01I AFT; ATLI1-LT

* ~~ IIS5~~ATL.'Al ii ')Cti At li-: AT 31-i-UT-[M- I

i tuS tI I4 I A~lY I'Ll 17 .Ctl1IL uITJ: LIn-ZANY

I II A'I~u.0100, rVAL viHMN\,r1, UlHLCIJAT U INSTRUCTION

I CHJIF r)F NAVAL ii.'l T Iuj A1'4L) 1rj'

j1S~ tSIAI-I ~'Cuj a ly uLv ANOL II4 i)Jv

I t

4

lJ AIL/Kp-To Ih 1\ )fU O~Mb qlLvIL~liLNI

1 1 3 Is 1111L TI' I . f- I L I I iAHY ASSb j~nI ANL A TIN: Al 5'- TU-TA

j I I' ARMY ANPM, b, t5,1),, 01IHLLTUk,,iL UF TKAININ,

I kIS,% 4± All olftt NS 'IAUL u 1N: j~, II -D)I

I IIS~ UNt ~~- SCLlOU~L ulk (IUHAII UW JWAINN.I ULVELOPMENTS

I i's c~ll'I 'A 4HU M o MlIN: (uuLl LJUNSI-LUN (WICK SLIMAK)

I I),~ It 10\lIA i ' LH-UUL UloqpLIU( OF TwA'ININ6

i IS', [NI. AoNiny S-t L 1n,-AHY /

i ui INI-ANIN-Y AtUJ TIN: AIS~i-I-V

I [IS IlHMYI JN A f..ITHy )( tUUI A IN; AiStI-(.J

I I1 1i Ni, ANTYI I t5L 0,)l AIIN: Al S.i-UUF

I (IS INt:ANlWY S L.-J J I Al I N A I:3--LV

I IS 111 1AioY I Ju I -- 'I5Lti0UL/TI.jANU. CI'til.w Al TN: ATLN-Prb

* ~I 'S ( -I III !,~Y '0i Lj C-lL NINNJt.J ULN. COMB~AT VLO'4N

II Il~ M I II AMI4 V-) P14k L SLI~iIUL, I~ali\.LNLU CUNL ILR TWAlNIN.i VLL0PM'-NI

*I tIS *4IAII I I A- PU ICI )L i IL / FklT4AL CENTf 04 ATTN: AliN-ACL

I iS%. IN-1l I IL Th U, AomINsitiiAT IU'.,j All r,: k iUu'4 IRA1NIN~G MANAbt4NT

i ijS iN, II U A 1(IMINI AIN4I .. ALAILMIr LIbiMANY

I US A wA CUI LhI- L £Al IN: L-10-4kY

I IIS I EN(.INLINSL4tW L jIlu 1.kY A-jU LIZAWNIAN 4LSUUW-Et) LFgTER

A PSIk ANm(IN S)C.-U IWlAuN.) Al 1, A L1BMANf

I IlS C0As I 6L AN) 4LAsIJI MY L I iNAt- Y

IIS t TPA'ORf)JAII'0\4 SLHOUL THAW3PlUxTsAI 10 SCH-IOL L164<APY

I 1N'ANIAAIONlkL -*r r Ci ~ltNtLS5 T-ou CLN - S. i ATTN: LIN8WARLA.

I ITS *HMy JNT1.LL 1,4 NLL CLr. I t. + SCMULL AV T NP: A I Sl-1

II IS AHMY IN1T LL I -r NlkL CL 4 t * SLJIUUL MFIT: ArSl-14M-M

i II AHfMY I N IF-L L1,l 4 \ t CL14I Lt, + LIIUOL A I I w: AlSI-fTl-SF-1P4

I US MMM TNt COIUNS umLA IIu,4 CLN ILH

I AUs~ h Lu j A LI t-qy S)Ctll)UL UA.4tLJURAlL fl- CUUNSL hLy * rAININI,

4 HwNTLbt4 LMtiASSY ikI I I rl UtF LNCL SAF - F

~CA IAUAAN JOINT ',IAI1-

C-3
I COI LI LiA Hw) H

I n')TrN1tN LMHiAS >r MILlItiNY AN() Alk ATFACtifi CA .AUI1N UJLANtL' *,'i IA 10 ATTN: LOieNSLLLUH. O'ftFNCL W ANt) 0

I wuYAL tLt*LAMN.S t4AWY MILIIMM'V Af1ACIHL

i c. Aubi LiCtS s CoqNwakLLIct AIIN:* PtHSONNEL SLLLCTION

I- (A ~il k I(r,.) P iSONN'LL -..bL IHSCh 0'4 1 I

J ANYI VFHS' )NN L *kt~tAPCtf tL4AOLI MMEN1

iAitV t PO-WNW~~IL -4tSl-AHCP- Lt~IAt$LI~nMLNT 1,61 SLLLNE [FtC roOiPIiNA11hI"LW ul t

"I I ikAHN' O Cfbyt-'tSSj LACIMI'dL A~t 14 l Iu

I ~~otS 1tCH-N1L.I )N~NIHAdiION CLN A I Ii'N: T LL-TIC

I ti 14WA14 UP- CO)NU-tSt UNJI I )OtUiAL1Ibo1 t dXIi ITINU PHOnJLC1

I H)1IOR. H ANI) U 'A9%kbA11Nt ATIN: uWCiA-LfN

I tS LiJV HIJMI- NI V, I HilN(, k4L L j.NAHY , "U-41 IC UOCUMFN ~I S flFAw IMINT

I l' l(VtW41tN1 P lII1t, Oi1- LI.,M(AHY AWg f.ATuTURY, LIH )IV (SLL)

I rIH- Ai,', LTiHAhy

GPO Sea.8es

C-4