# Using Human Dynamics to Improve Operator Performance

Rui Antunes, Fernando V. Coito, Hermínio Duarte-Ramos

► To cite this version:

Rui Antunes, Fernando V. Coito, Hermínio Duarte-Ramos. Using Human Dynamics to Improve Operator Performance. First IFIP WG 5.5/SOCOLNET Doctoral Conference on Computing, Electrical and Industrial Systems (DoCEIS), Feb 2010, Costa de Caparica, Portugal. pp.391-398, 10.1007/978-3-642-11628-5_43 . hal-01060763

HAL Id: hal-01060763

https://inria.hal.science/hal-01060763

Submitted on 16 Nov 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Using Human Dynamics to Improve Operator Performance

Rui Antunes<fnref n="1" />,<fnref n="2" />, Fernando V. Coito<fnref n="1" /> and Hermínio Duarte-Ramos<fnref n="1" />

**Abstract.** Traditionally Man-Machine Interfaces (MMI) are concerned with the ergonomic aspects of the operation, often disregarding other aspects on how humans learn and use machines. The explicit use of the operator dynamics characterization for the definition of the Human-in-the-Loop control system may allow an improved performance for manual control systems. The proposed human model depends on the activity to be performed and the mechanical Man-Machine Interface. As a first approach for model development, a number of 1-D manual tracking experiments were evaluated, using an analog Joystick. A simple linear human model was obtained and used to design an improved closed-loop control structure. This paper describes practical aspects of an ongoing PhD work on cognitive control in Human-Machine systems.

**Keywords:** Human Dynamics, Man-Machine Interfaces, Human-in-the-Loop Control, Manual Tracking Systems.

## 1 Introduction

Our life is enhanced by mechatronics products, comprising Man-Machine Interfaces. However, ordinary machines are not usually designed to assist human to improve one's skill, and in many cases much time and effort are needed for an operator to be trained. One of the main reasons is because machines usually don't change regardless of the human skill, often requiring a long operator training stage.

An important goal is to create and develop intelligent mechatronics systems, capable of adapting themselves to the level of the skill/dexterity of the operators who use them, considering not only the ergonomic aspects of the operation, but also the way humans learn and use machines<fnref n="1" />. This brings along a new concept for manual control engineering on Human-Machine systems that inevitably have to consider human in the closed-loop. Recent demands in many areas, for more precision,

<footnote n="1">¹ Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa
2829-516 Caparica, Portugal
{rui.mantunes, fjvc, hdr}@fct.unl.pt</footnote>

<footnote n="2">² Escola Superior de Tecnologia de Setúbal do Instituto Politécnico de Setúbal
2910-761 Estefanilha, Setúbal, Portugal
rui.antunes@estsetubal.ips.pt</footnote>

<footnote n="1">¹ A Human-Machine mechatronics system that has the function to assist the human operator is usually called Human Adaptive Mechatronics (HAM).</footnote>
accuracy and safety (such as in medicine, biotechnology, robotics, transports,
entertainment, space, nanotechnology, ocean, disaster site and factory), led to a new
need for the development of HAM systems [1].

## 2 Contribution to Technological Innovation

The purpose of this paper is to present a contribution on human operator modeling for control applications purposes. The inclusion of the operator model on the development of Man-Machine interface devices leads to improved performance on manually controlled operations, as well as easier operation and a reduced training stage.

As a first approach for model development nonparametric system models are used. The models may be obtained from frequency, transient and spectral analysis. The scope of this document is centered on frequency analysis, which proved to be adequate for the development of operator models that, in spite of their simplicity, lead to a good performance. In the sequel, the modeling procedure will be described. The other approaches have also been evaluated but fall beyond the scope of this paper.

A real-time simulation experiment was developed for evaluating skill on pre-defined closed-loop Human-Machine systems, as a special tool to measure the overall performance. An effort performance measure is also proposed, based on human lazy strategies.

## 3 State-of-the-Art / Related Literature

It is a key idea that we need to model human behavior, and, so far, many models of
the human controller have been proposed.

In 1940's Tustin tried to introduce a human control model using a transfer function to model human action, proposing a linear servo control. In the 1960's Ragazzini modeled a human as a PID controller and showed that humans are time-variant systems having randomness, stating also that the differences among individuals should be addressed. In 1970 Kleinman et al. studied the dynamics of pilots. The transfer function of a pilot was considered as the cascade of the reaction lags/delays attributed to the neuromuscular human system. A method to compensate the time-delay, using a Smith predictor, was described. Anil proposed in 1976 that the human controller could be described both as time-delay and a Kalman filter.

Kawato introduced later in the 1990's the feedback error learning model, which assumes that human has inverse and forward models of the dynamics of the movements, and that the brain, by learning, tends to change human's model from feedback to feedforward. More recently, Wolpert and Kawato improved the feedback error learning model to a module selection and identification control (MOSAIC), by expanding the inverse model into a controller and the forward model into a predictor.

HAM research [2], [3] is being promoted now in some countries, mainly at Japan and UK. Latest developments include studying the brain activity at particular
Brodmann's areas using near-infrared spectroscopy [4], and the manipulation of human behavior by distorted dynamics vision [5].

# 4 Research Contribution and Innovation

Several methodologies can be carried out to obtain an LTI model. In this work, special focus is given on improved frequency analysis [6], for obtaining Human-Machine linear models from 1-D manual tracking experiences. However, two major points must be stressed over human operator modeling:

1) The operator behavior can not be fully captured through a simple dynamic model, or even a set of such models. Thus, the objective of the models developed is not to replicate human behavior, but only to capture enough information to compensate the drawbacks inherent to the human operator dynamics.

2) It is difficult, if not impossible, to experimentally obtain an open-loop operator model. In any experiment the operator closes the loop between sensing and acting. Hence, to obtain the operator "true" model it is necessary to extract it from the close loop data.

## 4.1 Frequency Analysis

Consider a one-dimensional input signal, to be tracked, $x(t)$, built from a sum of $N$ sinusoids at pre-defined multiple frequencies. Assume that the one-dimensional input normalized signal has duration $T$, and $y(t)$ is the correspondent LTI system output. Such signals may be written as:

$$x(t) = \sum_{k=1}^{N} x_k(t) = \sum_{k=1}^{N} a_k \sin(\omega_k t) \quad \max\{|x(t)|\} = 1, x(t=0)=0. \quad (1)$$

$$y(t) = \sum_{k=1}^{N} y_k(t) = \sum_{k=1}^{N} b_k \sin(\omega_k t + \varphi_k). \quad (2)$$

For each applied frequency, the I/O response may be obtained through the following scheme:

<figure><img src="image_2.png" /><figcaption>Fig. 1. Frequency analysis *block diagram* for each *k*-multiple frequency.</figcaption></figure>
By performing the integration along time $T$ (a multiple of the sinusoid period, $T = \frac{k2\pi}{\omega}$), leads to:

$$y_{C_k}(T) = \int_{0}^{T} b_k \sin(\omega_k t + \varphi_k) \cos \omega_k t dt \quad y_{C_k}(T) = \frac{b_k T}{2} \sin \varphi_k . \quad (3, 4)$$

$$y_{S_k}(T) = \int_{0}^{T} b_k \sin(\omega_k t + \varphi_k) \sin \omega_k t dt \quad y_{S_k}(T) = \frac{b_k T}{2} \cos \varphi_k . \quad (5, 6)$$

$$b_k = \frac{2}{T} \sqrt{y_{C_k}^2(T) + y_{S_k}^2(T)} \quad \text{and} \quad \varphi_k = \arctan \left( \frac{y_{C_k}(T)}{y_{S_k}(T)} \right) . \quad (7, 8)$$

which corresponds to the operator closed-loop frequency response.

For each multiple input frequency, a corresponding magnitude was settled to build a human feasible manual tracking input signal. Figure 2 shows the magnitude Bode plot of the input signal $x(t)$ created for the manual tracking experiments (made with a commercial Joystick).

<figure><img src="image_3.png" /><figcaption>Fig. 2. Input signal magnitude based on the N=30 frequencies sum, ranging 0.0083Hz to 10Hz.</figcaption></figure>
## 4.2 Modeling Human Behavior

One hundred tracking time-trials, with *T*=120 seconds duration each, were obtained for the same participant with no history of neurological deficits. At least, a minimum 10 minute rest was given between trials, that all, lasted for 3 weeks.

A sample of a trial for the input *x*(*t*) is presented *below*, in figure 4:

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** LabVIEW developed application for the manual tracking experiences (**left**). Manual tracking time-trial using Logitech's Extreme 3D Pro. 8-bit analog Joystick (**right**).</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** A 1-D manual tracking sample (first 20 seconds) at 100Hz sampling rate. The input signal is null at 0 and at 120 seconds (*t*=0, *T*).</figcaption></figure>

One hundred open-loop linear models for the same individual were obtained, from the closed-loop time-trials, by inverse manipulation. The ±2σ limits were calculated (assuming a normal unimodal symmetrical distribution, approx. 95 of the models fits inside). A 3 stable pole open-loop nominal model was proposed, based on magnitude:
<figure><img src="image_6.png" /><figcaption>Fig. 5. Proposed human model open-loop magnitude Bode plot, ranging between 0.02Hz and 7Hz, based on 100 1-D manual tracking experiments.</figcaption></figure>

$$H(s) = \frac{2060}{s^3 + 4.5s^2 + 527s + 679} \quad (9)$$

## 4.3 Controller Design Strategy

Three controllers are proposed to control an unstable $P(s)$ dynamics. The first (C1) is a classical phase-shift compensator. The second (C2) considers human as a static gain only, and the third controller (C3) is obtained from the human model:

<figure><img src="image_7.png" /><figcaption>Fig. 6. Block diagram for the closed-loop physical system to be controlled.</figcaption></figure>

$$C_1(s) = \frac{0.5s + 0.05}{s + 50} \qquad C_2(s) = \frac{C_1(s)}{ko} = \frac{679}{2060} \cdot \frac{0.5s + 0.05}{s + 50} \quad (10, 11)$$

$$C_3(s) = \frac{C_1(s)}{H_1(s)} = \frac{256000s^4 + 1.178e006s^3 + 1.35e008s^2 + 1.873e008s + 1.738e007}{2060s^4 + 597400s^3 + 6.427e007s^2 + 3.032e009s + 5.274e010} \quad (12)$$

Where $H_1(s)$ presents the same frequency behavior as (9), but includes an additional term (with unity static gain), in order to allow the implementation of $C_3(s)$.
# 5 Discussion of Results and Critical View

This section presents obtained experimental results, with C1, C2 and C3 controllers:

<figure><img src="image_8.png" /><figcaption>**Fig. 7.** *Human step response* for two closed-loop control systems (without controller, and with C3 developed controller), with *P(s)* dynamics, at 300Hz sample rate. *Input reference is 0.3925.*</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>**Fig. 8.** Detailed human step response for two closed-loop control systems (with C1 and C2 controllers), with P(s) dynamics, at 300Hz sample rate. Input reference is 0.3925.</figcaption></figure>
Manual step performance was measured on a double-integrator *P(s)* unstable process, considering the reference mean square error, for acuity, and the mean stabilized duration (related with human's lazy strategies, within ±0.0035 from input reference). From figures 7 and 8, *C3* controller clearly took the best performance.

The simulation results from table 1 show that the *C3* controller (obtained from the proposed linear human model and *C1*) widens the bandwidth of the Human-Machine system and raises the overall phase angle curve, improving the frequency response and the stability margins. *C2* controller, assuming human model as a simple static gain (*k*o) gives also higher stability than *C1* (which neglects *H(s)* in the closed-loop).

<table><caption>**Table 1.** Step response manual performance (in 60 seconds) and stability margins, for P(s).</caption><thead><tr><th></th><th>Controller C1</th><th>Controller C2</th><th>Controller C3</th></tr></thead><tbody><tr><td><strong>Performance:</strong></td><td></td><td></td><td></td></tr><tr><td>Mean square error</td><td>0.0045634</td><td>0.0091764</td><td>0.0016567</td></tr><tr><td>Mean stabilized duration (s)</td><td>1.5382</td><td>1.5542</td><td>5.5260</td></tr><tr><td>Number of stabilized sequences</td><td>24</td><td>20</td><td>5</td></tr><tr><td><strong>Stability:</strong></td><td></td><td></td><td></td></tr><tr><td>Gain Margin Frequency (Hz)</td><td>1.0618</td><td>1.0618</td><td>4.6104</td></tr><tr><td>Gain Margin (dB)</td><td>40.5313</td><td>50.1712</td><td>52.1004</td></tr><tr><td>Phase Margin Frequency (Hz)</td><td>0.0494</td><td>0.0202</td><td>0.0202</td></tr><tr><td>Phase Margin (°)</td><td>58.2328</td><td>45.9675</td><td>51.4105</td></tr></tbody></table>

# 6 Conclusions and Further Work

In this paper, some HAM research topics that take human factor into account were introduced for the control of Man-Machine systems. To prove the effectiveness of the proposed modeling method, a 1-D Human-Machine experimental SISO system was implemented and tested. Further work is to improve the controller robustness and to develop multi-model design strategies, and also to move on to 2-D systems.

# References

1. Harashima, F., Suzuki, S.: Human Adaptive Mechatronics - Interaction and Intelligence. In: 9th IEEE International Workshop on Advanced Motion Control, pp. 1--8. Istanbul (2006)

2. Suzuki, S., Watanabe, Y., Igarashi, H., Hidaka, K.: Human skill elucidation based on gaze analysis for dynamic manipulation. In: IEEE International Conference on Systems, Man and Cybernetics, pp. 2989--2994. Montreal (2007)

3. Suzuki, S., Harashima, F., Pan, Y., Furuta, K.: Skill analysis of human in machine operation. In: International Conference on Neural Networks and Brain, pp. 1556--1561. Beijing (2005)

4. Suzuki, S., Kobayashi, H., Harashima, F.: Brain monitoring analysis of skill on voluntary motion. In: International Conference on Control, Automation and Systems, pp. 1178--1182. Seoul (2007)

5. Kobayashi, H., Ohyama, Y., Hashimoto, H., She, J.-H.: Manipulation of Human Behavior by Distorted Dynamics Vision. In: International Joint Conference SICE-ICASE, pp. 4446--4450. Busan (2006)

6. Söderström, T., Stoica, P.: System Identification. Prentice Hall (1989)