The 5th COE Postdoctoral and Doctoral Researchers
Technical Presentation
NAIST, Aug. 24, 2004

List of Contents
Research BackgroundResearch Objectives
Theoretical Bases
Mental State and Attention
Modeling and Simulation for Cognitive Processor and Memories
Adaptive Aiding Examples
Summary
Future Work

Study on Adaptive Aiding for Plant Operation

Xiwei Liu, Hiroaki Kosaka, and Hirokazu Nishitani
Systems and Control Laboratory

Aug. 24, 2004

The 5th COE Technical Presentation

1

Research BackgroundAug. 24, 2004

2

Research Objectives

Plant Control System becomes complex.
Human errors easily occur because of limitations of human
information processing.
Some support system may actually be a burden or harmful toplant operation.
Computers should know human mental and physicalinformation for effective supports.

Aug. 24, 2004

The 5th COE Technical Presentation

The 5th COE Technical Presentation

Human modeling and simulation
To analyze human performance by using
computational models and simulation.
 To evaluate the effect of operation supports.
Prototype system for adaptive aiding

3Aug. 24, 2004

The 5th COE Technical Presentation

4

Model Human Processor
General Architecture of HCI Model

Long-Term Memory
Short-Term Memory

Human Model

Machine Model

Visual ImageStoreAuditory ImageStoreInteraction Model

PerceptualProcessors(Cacciabue, Piero, C., 1998)

PC

We use a boiler simulator as a machine model, and build both interaction modeland human model.

Aug. 24, 2004

The 5th COE Technical Presentation

CognitiveProcessor

5Aug. 24, 2004

MotorProcessor(Card et al., 1983)

The 5th COE Technical Presentation

61

Human Model

Framework of Adaptive Aiding

Perceptual processor, memory (long-term and short-term), cognitive
processor and motor processor constitute a human information-processing
system.
Perceptual processor captures visual and audio information from user panels.
Short-term memory codes, holds and transfers data for perceptual andcognitive processor.
Cognitive processor diagnoses failure cause for a plant control system basedon the information from memories, and informs the corresponding
countermeasures to motor processor.
Long-term memory stores declarative and procedural knowledge.
Motor processor executes procedures by manipulating devices on HMI.

Aug. 24, 2004

The 5th COE Technical Presentation

7

Mental State

Measurement
MentalWorkload

MotorProcessor

Human Errors and Mental
State Estimation

MentalState

CognitiveProcessor

Long-term
MemoryAttention

Short-term
MemoryPerceptual Icons, Data
Processor Alarms

Adaptive
Functions

Realtime
Data

HMI
Model OPC
ServerClick, Keypress
Eye Movement

Aug. 24, 2004

The 5th COE Technical Presentation

8“Attention” is used to present the cognitive limitation of human in cognitive
psychological research.
We use “selective attention” and “attention resources allocation” in thehuman model (Treisman, 1960, 1964; Deutsch, 1963; Kahneman, 1973).
During the coding stage of short-term memory, selective attention can keep
abnormal data but reject other normal ones.
The total amount of attention resources is limited and can be increased ordecreased by the mental state. The attention resources are allocated toperceptual, cognitive, and motor processors and memories for information
processing, and the assigned attention resource affects the performance ofeach component of the information processing system.

The 5th COE Technical Presentation

9Aug. 24, 2004

The 5th COE Technical Presentation

10Fault Diagnosis

Fault Diagnosis
A state vector Y includes all available process variables.
An operator monitors which variables are abnormal
according to experience and understanding of the plantsystem.
Y=[Y1, Y2, Y3, …Yk, …, Ym],
Where Yk=[yhk, ylk], (k=1, 2, 3, …, m)
m: number of available process variables.
Yk: a vector including two discrete variables torepresent abnormality.

Aug. 24, 2004

Human Behavior

Attention

Mental state can be categorized into three states:
confidence, inference, and confusion (Kurooka et al.,2001) .
Mental state is determined from mental workload in
this study.
Mental state affects the total amount of available
attention resources.

Aug. 24, 2004

Adaptive Aiding

The 5th COE Technical Presentation

11

Probability of the 2ndfailure cause P1,1P
 2 ,1 M
E= Pk ,1
 M Pm ,1P1, 2P2, 2L
LLLM
MM
Pk , 2 L Pk , jM
MPm , 2 LM
LL P1, n 
L P2, n 
MM 
L Pk , n MM L Pm , n Probability when the kth
variable is abnormal

E: evaluation matrix of failure probability (mental model),n: number of failure causes,
Pk,j: probability of jth failure cause when the current value of Yk has
an abnormal value.
Aug. 24, 2004

The 5th COE Technical Presentation122Mental Model

Fault Diagnosis

Matrix E for simulation

J=Y×E=[P1, P2, …Pj, …, Pn],
Pr =Max (Pj ),Ps = Ac×Pr ,If Ps> θc , the failure cause is identified,
Ac: cognitive attention level,
θc: cognitive threshold.

The 5th COE Technical Presentation

13Input

00.4

00.5

0.2

000

0

0.6

0.4

00.5

00

0.2

0.2

0.6

0.5

High

1.8

1.8

00.3

0.4

0.1

00000Low-1*

00.30000000.1

0Aug. 24, 2004

The 5th COE Technical Presentation

14

Short-term Memory
From perceptual
processor

Encoding
Attention

00.6

Low

Through this table, cognitive processor can detect two failure causes.

Maintenance Rehearsal

Sensory
Memory-1*

Down*Exclusive method

Modal Model of Memories

SensoryP203. P203. A201. A202. F204. F204. P205. F202. P204. R080. F203.
PVMV PV
PV
PV
MV PV PV PV CPV PV

FDF Capacity HighIDF Trip

Whether the judgment is correct depends on the mental model(matrix E) of an operator. The identified failure cause is called
acceptable result for the operator.
Aug. 24, 2004

Failure cause State

Working or
Short-term
Memory Retrieval

Long-term
memoryCode

HoldRetrieval To cognitive
processor

Code: Selectively stores data into chunks. Every variable has aintensity value.
Hold: All data decay exponentially with time.
Retrieval: A variable cannot be read out for diagnosis, If its
intensity less than memory threshold.

(Shiffrin, Atkinson, 1969)

Aug. 24, 2004

The 5th COE Technical Presentation

15

Aug. 24, 2004

16

Simulation Experiments

Memory Decay−α ( t − t0 )The 5th COE Technical Presentation

intensityµ = µ0e Astmµ: memory intensity,µ0µ0: initial intensity decided by the long-term
memory and perceptual processor,

θstm

α: loss factor,
t: current time,

Visualcoordinates

α2> α1
Userpanel

Used fordiagnosis

Memoryintensity

α1

Variables

time

α2

Unavailablefor diagnosis(Peterson & Peterson, 1959)

t0: time when the variable is firstly coded,
Astm: short-term memory’s attention level,

Aug. 24, 2004

The 5th COE Technical Presentation

Captured
items
within avisual field

All items
within avisual fieldθstm : short-term memory’s retrieval threshold
17Aug. 24, 2004

The 5th COE Technical Presentation

183

Simulation Experiments
Simulation Experiments
When an operator finds abnormal variables during a gaze, A judgment
is launched. Total number of judgments for an acceptable result is used
to evaluate cognitive performance. This table indicates that the smaller
the cognitive threshold is, the quicker a judgment is made, but there is
no guarantee of a correct judgment.


An incautious operator has a low cognitive threshold.
A conservative operator has a high cognitive threshold.

Effect of cognitive threshold on judgment*
When the probability ofa failure cause is bigger
than the cognitivethreshold, a fault
diagnosis is completed.

Cognitive threshold

1.0

1.5

2.0

2.5

3.0

3.5

Number of judgment

23

4

5

5

6

Last failure probability

1.4

1.6

2.3

3.4

3.4

3.9

*Experimental condition: memory retrieval threshold is 0; failure cause is FDF
Capacity Down

Aug. 24, 2004

The 5th COE Technical Presentation

19

Aug. 24, 2004

The 5th COE Technical Presentation

20

Simulation Experiments

Adaptive Aiding Examples for Fault Diagnosis

Effective rehearsal is helpful to hold data for a long while, i.e, to decrease theloss factor α. For operators with the same skill level (cognitive threshold), asmall α can complement human’s memory deficiency (big retrieval threshold).

(a) When some important process variables are abnormal, thesystem should provide timely alarm information. (for rehearsal)(b) If the mental state is confusion, the system should supply a listof likely failure causes and their probability (decreasing
cognitive threshold).
(c) If there are some abnormal variables on the panel, amemorandum function should record them (for rehearsal).
(d) A help function should be used to complement operator’s longterm memory (knowledge complement for mental model).

Effect of short-term memory retrieval threshold on judgment*
Retrieval threshold
Number ofjudgment

0.05

0.10

0.15

0.20

0.25

0.30

0.35

Condition

555555-**α =0.455555--

α =0.55555---

α =0.6*Experimental condition: cognitive threshold is 2.5; failure cause is
FDF Capacity Down**Failure cause cannot be detected.
Aug. 24, 2004

The 5th COE Technical Presentation

21Summary

The 5th COE Technical Presentation

22Future WorkA framework of cognitive informationprocessing model incorporating with mental
sate and attention was proposed.
Human’s cognitive and memory performance
was simulated quantitatively.
Adaptive aiding examples were proposed formonitoring and fault diagnosis.

Aug. 24, 2004

Aug. 24, 2004

The 5th COE Technical Presentation

To improve the fidelity of simulation.
To validate the influences of human mental
state and attention on human performance.
To evaluate the effect of adaptive aiding.
To design prototype adaptive aiding systems.

23Aug. 24, 2004

The 5th COE Technical Presentation

244