# A Linear Approach towards Modeling Human Behavior

Rui Antunes, Fernando V. Coito, Hermínio Duarte-Ramos

► To cite this version:

Rui Antunes, Fernando V. Coito, Hermínio Duarte-Ramos. A Linear Approach towards Modeling Human Behavior. 2nd Doctoral Conference on Computing, Electrical and Industrial Systems (DoCEIS), Feb 2011, Costa de Caparica, Portugal. pp.305-314, 10.1007/978-3-642-19170-1_33. hal-01566603

HAL Id: hal-01566603

https://inria.hal.science/hal-01566603

Submitted on 21 Jul 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A Linear Approach towards Modeling Human Behavior

Rui Antunes¹,³, Fernando V. Coito¹,², and Hermínio Duarte-Ramos²

¹ UNINOVA, ² Departamento de Engenharia Electrotécnica, Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa
2829-516 Caparica, Portugal
{ fjvc, hdr } @fct.unl.pt

³ Escola Superior de Tecnologia de Setúbal do Instituto Politécnico de Setúbal
2910-761 Estefanilha, Setúbal, Portugal
rui.antunes@estsetubal.ips.pt

**Abstract.** The human operator is, no doubt, the most complex and variable element of a Mechatronics system. On simpler manual control tasks, a linear model may be used to capture the human dynamics, however experiences on human operator response during pursuit manual tracking tasks, show that the dynamics of the human operator appear to depend on the specific task that the subject is asked to perform. This means that a unique truly human model cannot be completely achieved. Rather, a different set of models, each for a certain class of task, seems to be needed. This ongoing PhD work introduces several approaches on the human operator dynamic characteristic modeling and identification procedures, which may be useful for developing improved "humetronic" systems, i.e. human-machine systems which may be able to adapt themselves to the skill level of humans, aiming, with reduced effort, to achieve for best performance and safety.

**Keywords:** Human Dynamics Modeling, Identification, Human-Machine Interfaces, Manual Tracking Systems, Machine Adaptation.

## 1 Introduction

Nowadays we rely on many different mechatronics equipments and gadgets for carrying out our way of life, and we are well aware of the limitations with which machines can efficiently perform controlled manual tasks, because unfortunately these machines usually do not change regardless of the human operator's skill.

Current research aimed to improve performance and safety on man-machine systems is increasing. In an ordinary human-machine interface the human operator controls the machine by performing various manual control tasks. While performing the activity the operator is also learning it, and its skill is increasing. Hence, it seems natural the need for designing human-oriented machines which, in a smart way, may improve the assistance and performance with its human users.
## 1.1 Automatic Control versus Human-in-the-Loop (HIL) Control

Automatic control applications usually do not consider that the human factor inevitably belongs into the resulting closed-loop, and in an automatic control process human is still often considered remissness.

Human-in-the-loop (HIL) control is a starting approach to integrate the human model into the already assortment of electrical and mechanical factors involved in a closed-loop man-machine control design.

<figure><img src="image_2.png" /><figcaption>Fig. 1. a) Human-machine system block diagram. Comparison between automatic b) and human-in-the-loop control c).</figcaption></figure>

## 1.2 The Human Adaptive Mechatronics (HAM) Assist Control Concept

An important goal today is to improve the design of next generation adaptive human-oriented machines, which will have the ability to intelligently cooperate with its human users. These machines should be able to actively adapt according to the skills of their operators, being capable to evaluate human's global performance. A human adaptive mechatronics assist control system [1], [2], [3] identifies the operator based on his acquired actions in the process, and an assist controller is then iteratively conceived to improve the operator's performance according to its skill.

<figure><img src="image_3.png" /><figcaption>Fig. 2. Human adaptive mechatronics assist control block diagram.</figcaption></figure>

Fig. 2 shows a HAM assist control application example [4], where the machine dynamics is settled by tuning its virtual model from previously estimation of human's skill.

## 1.3 Modeling Human Behavior

Human models are important to design a machine system to be controlled by an
operator. The model is not aimed to closely replicate the operator behavior, but to provide sufficient information for the design of closed-loop control system incorporating the human behavior. Thus, in spite the fact that humans are non-linear time-varying complex systems, linear models are used as a way to capture some of the relevant characteristics of the human operator. This allows the use of standard control design techniques, combined with supervisionary controls approaches to tackle non-linearity and time variations.

To obtain linear models a number of experiments is required. Such experiments must be designed so as to minimize undesired effects like operator learning and prediction of the task or action signal saturation.

The first studies in human modeling were inspired by the demand for pilot models during the Second World War, and since that time there had been made many experiments, especially in pursuit and compensatory manual tracking tasks. One of the earliest studies of the human operator acting as a linear servomechanism was done in 1947 by Tustin, who proposed that there might be a linear law that could represent most of the operator's behavior. More recently new identification theories, methods and tools became available, striving human modeling to achieve significant progress.

In this paper several approaches are introduced on the human dynamic modeling and identification methods, and experimental procedures, for developing new human-oriented machines, which aim for best performance and safety, reducing also operator's effort. Although it must be stressed that whatever approach is taken to the description of human behavior, it only will capture a fraction of it, as human operator comprises the complex element in a man-machine system.

## 2 Contribution to Sustainability

Human-oriented machines help to create more sustainable services and products, without compromising human and environmental safety, as we are all challenged to produce and recycle more with fewer resources.

Essentially, three necessary phases are required for designing HAM systems:

1) Estimate human control characteristics;

2) Quantify the overall skill;

3) Design the assist-control human-machine system.

The first phase can be considered both as a signal processing and a system identification problem, which implies developing theoretical and experimental procedures for obtaining the human controller from measured response data. On simpler manual control tasks a linear model can capture relatively well the human's behavior<fnref n="1" />. However, the model depends on the task and the process to be controlled, and simultaneously on many other factors, both personal and environmental, such as concentration, training, disturbances, comfort, workload, saturation, intermittency, fatigue, etc. The second phase for HAM design uses the experimental data obtained to quantify the overall operator skill, by comparing between an individual and its

<footnote n="1">¹ The Crossover and the Proportional Derivative are generalized low order linear human models, well described in the literature.</footnote>
ideal response. Comparing ideal and identified parameters can also provide more accurate skill quantification. Next, according to the skill level obtained, an assist-control system (which is able to change the "amount of assistance") is finally implemented, based on an ideal model and on the acquired skill data.

In this work special focus is given on the first step of the Human Adaptive Mechatronics project, i.e. on the methods and procedures that can be derived for estimating the human control characteristics. We propose that the systemic notion of task dependence must be employed to model a human-machine system. New schemes for deriving task-dependent characteristics are explored though nonparametric and parametric identification methods.

Different approaches to the human identification problem are presented: we started by transient analysis (impulse and step response), obtained from pursuit manual tracking experiments. A special method to estimate the operator's delay-time was investigated. Structured Auto Regressive with eXogenous (ARX) input-output data parametric models where employed to obtain human-machine linear models from manual 1-D tracking experiences. Finally, the improved frequency analysis method is described in detail for modeling operator dynamics. An experimental setup that allowed the implementation of a human operator control task has also been developed, implemented and tested. The experimental procedures deployed for estimating the human-machine dynamics had an important role in this work, and where also described in detail.

## 3 State-of-the-Art / Related Literature

There have been in the past many studies covering the human operator dynamics modeling, especially within aeronautical engineering and flight control systems. Since the late 1940's of many experiments of human pilots performing target tracking tasks where studied. Important theories have been first introduced by McRuer et al., for manual tracking tasks using random references. A quasi-linear model was developed for describing human behavior during compensatory tracking tasks. In the 1960s and the 1970s modeling the human controller made great progresses [5]. The McRuer's et al. linear crossover model [6] and the optimal control model of Baron and Kleinman (applied for compensatory tracking) where proposed. The Baron's optimal control model was further developed by Tomizuka for preview tracking. In 1974 Shinners proposed the use of ARMA (autoregressive moving-average) models, obtained from tracking tasks.

In the 1990s Kawato postulated that the human's control has feedback and feedforward structures, and through the process of learning he changes from feedback to feedforward (in the Feedback Error Learning model [7] the cerebellum acquires a model of the machine, as an inverse model in the feedforward path). Wolpert and Kawato improved the Feedback Error Learning model to a module selection and identification control (MOSAIC) [8], expanding the inverse model into a controller and the forward model into a predictor. Latest developments include using optimal control model (OCM) design techniques (by minimizing a certain performance index), assuming that the human operator performs the manual task in an optimal way. Particle swarm optimization (PSO) is a promising method for
obtaining the OCM parameters from experimental data. ARX models (linear approach) and adaptive-network-based fuzzy inference tools have been recently studied. A new hybrid fuzzy-ARX modeling method [9] has also been recently developed for predicting the human operator control actions.

# 4 Research Contribution and Innovation

This work introduces several methodologies on the human-machine dynamic modeling and identification procedures, which can be applied on the development of improved HAM systems. In general, for execution of complicated tasks the human response does not follow linear behavior. However, for simpler servo/regulator control tasks, a linear model can be employed to capture most of the operator's dynamic characteristics.

Linear models can be obtained by physical (mathematical) modeling, and through system identification techniques based on measured data. The first approach is rather involved and time consuming, lying far beyond the objectives in hand. Hence, this work is focused on the system identification problem, i.e., on how to estimate a human-machine system model from the observed input/output data. Parametric identification methods are techniques to estimate parameters for pre-defined model structures, by finding through numerical search the parameters values that minimize the differences between the model's output and the measured data. ARX parametric identification methods were used in this work to model the human-machine dynamics. Nonparametric identification methods allow obtaining the operator behavior without the need for a pre-defined parameterized model structure. We have employed also nonparametric methods in this work, by analyzing the human operator transient and frequency responses on manual tracking tasks.

## 4.1 ARX Model Estimation

Auto-Regressive with eXogenous terms (ARX) models are well suited for modeling linear systems, due to its high potential and simplicity. The most common ARX structure is the linear difference equation:

$$y(t) + a_1 y(t-1) + \dots + a_{na} y(t-na) = b_1 u(t-nk) + \dots + b_{nb} u(t-nk-nb+1) . \quad (1)$$

The present output $y(t)$ is related to a number of past inputs $u(t-k)$ and past outputs $y(t-k)$. $na$ equals the number of poles and $nb$ the number of zeros. $nk$ is the time delay inherent to the system. Typical estimation methods used for obtaining $a$ and $b$ parameters are the least-squares, the Kalman filter, and the instrumental variable method.

## 4.2 Transient Response Analysis

The dynamic properties of a linear model can be investigated by analyzing its transient response. For example, system time delays, time constants and static gain
can be computed through the obtained impulse and step responses. Also, the right previous selection of the delay nk in the ARX model structure may be crucial for obtaining good identification results. Finally, the identification of low order linear models can also be easily confirmed from transient (step and impulse) analysis experiments.

## 4.3 Time-delay Estimation

Time-delay [10] is an important human factor that affects operator performance. Although human is considered a nonlinear system, many studies (such as proposed by Ragazzini) support that in a simple motor task the operator behavior can be sufficiently identified by a linear model plus a finite time-delay (as a result of the neuromuscular and central nervous latencies, and also due to other human dependent and environmental factors). The time-delay parameter is inherent to the ARX method, through the sampling shift. Therefore, human operator time-delay estimation can be further confirmed throughout the ARX identification procedure.

## 4.4 The Improved Frequency Analysis Method

Another nonparametric identification method is based on the frequency response, i.e. on how a linear dynamic model would react to certain sinusoidal inputs. In a LTI system, if we let the input $u(t)$ be a sinusoid of a certain frequency, then the output $y(t)$ will also be a sinusoid of the same frequency. However, the amplitude and the phase may be different. If we consider a one-dimensional normalized input signal of duration $T$, to be tracked, $x(t)$, built from a sum of $N$ sinusoids at fixed multiple frequencies, the correspondent output $y(t)$ and the I/O response for each frequency may be obtained through the following diagram:

$$x(t) = x_0 + \sum_{k=1}^{N} a_k \sin(\omega_k t) \Rightarrow y(t) = y_0 + \sum_{k=1}^{N} b_k \sin(\omega_k t + \varphi_k) . \quad (2, 3)$$

<figure><img src="image_4.png" /><figcaption>Fig. 3. Frequency analysis block diagram for each k-multiple frequency.</figcaption></figure>

By performing the integration along time $T = \frac{k2\pi}{\omega}$ (a multiple of the sinusoid period), results in:

$$y_{C_k}(T) = \int_{0}^{T} b_k \sin(\omega_k t + \varphi_k) \cos \omega_k t dt \Leftrightarrow y_{C_k}(T) = \frac{b_k T}{2} \sin \varphi_k \quad (4,5)$$

$$y_{C_k}(T) = \int_{0}^{T} b_k \sin(\omega_k t + \varphi_k) \cos \omega_k t dt \Leftrightarrow y_{C_k}(T) = \frac{b_k T}{2} \sin \varphi_k \quad (6,7)$$

(8)
$$y_{S_k}(T) = \int_{0}^{T} b_k \sin(\omega_k t + \varphi_k) \sin\omega_k t dt \Leftrightarrow y_{S_k}(T) = \frac{b_k T}{2} \cos \varphi_k$$

$$b_k = \frac{2}{T} \sqrt{y_{C_k}^2(T) + y_{S_k}^2(T)} \quad \text{and} \quad \varphi_k = \arctan \left( \frac{y_{C_k}(T)}{y_{S_k}(T)} \right) \quad K_0 = \frac{y_0}{x_0}.$$

which corresponds to the resulting human-machine closed-loop frequency response and static gain $K_0$ (for a previous input offset $x_0$). From the closed-loop experimental data, an open-loop human-machine model can be obtained, by inverse manipulation.

# 5 Discussion of Results and Critical View

This section presents obtained results from the experiments that where conducted for estimating human-machine models, in 1-D pursuit manual tracking tasks.

## 5.1 Experimental Procedures

For the transient analysis, two sets of five tracking samples (each lasted 2 minutes, and with at least 6 steps/impulses) were performed for a same operator with no history of neurological disease. To ensure human memorization or fatigue does not influence results, a minimum 15 minute rest was imposed between trials. The impulse and step inputs were visually recognized through large led indicators. In concerning ARX modeling, a set of four tracking tasks where conducted, with $T=120$ seconds duration each, comprising a white noise low-pass ($H(s)$) filtered input signal.

<figure><img src="image_5.png" /><figcaption>Fig. 4. A manual tracking time-trial using Logitech's Extreme 3D Pro. 8-bit analog Joystick.</figcaption></figure>

<figure><img src="image_6.png" /></figure>

$$H(s) = \frac{25}{s^2 + 7.0711s + 25} \quad (9)$$

To obtain a human-machine linear model from the experimental data, a procedure was proposed. In system identification the adequate choice of the sampling rate is
important. Sampling to fast leads to poor modeling. However, the use of an incorrect value for pure time-delay can mask system behavior. So the procedure to choose both time-delay and sample rate is as follows:

1. The signals are oversampled, at 100 Hz.

2. The first 0.5 seconds of each test are discarded because it corresponds to the low-pass filter initial transient.

3. For each choice for time-delay the signals are decimated before an ARX model is computed. From the point of view of discrete time models this corresponds to non-integer delay.

4. Taking the quadratic deviation between the true output and its estimated value as a performance index, the best combination for (time-delay, sampling rate, model structure) is chosen.

5. Continuous time equivalents of the time-delay and ARX model are separately obtained, and combined into a single continuous time model.

Frequency analysis experimental procedures and results fall beyond the size of this paper, and were already described in detail in previous work [11], [12].

## 5.2 Experimental Results

Obtained step and impulse response data showed that pure time-delay is unrelated with the number of trials (or human experience). Hence, for the identification of the operator's behavior a simple compensation data shift can be included in the correspondent model. Saturation and nonlinearity characteristics can also be found at the maximum limits, and near the joystick's origin, and the impulse tests where made with a 0.2 offset, to avoid the mechanical nonlinearities near the origin.

<figure><img src="image_7.png" /><figcaption>Fig. 5. Step (left) and impulse (right) responses, at 100Hz sampling rate.</figcaption></figure>

<table><caption>**Table 1.** Pure time-delay (in seconds), on five manual tracking step and impulse tasks.</caption><thead><tr><th></th><th colspan="12">delay (s)</th></tr></thead><tbody><tr><th>Impulse</th><th>1 - 10</th><th>11 - 20</th><th>21 - 30</th><th>0.43</th><th>0.37</th><th>0.36</th><th>0.39</th><th>0.41</th><th>0.46</th><th>0.39</th><th>0.42</th><th>0.38</th><th>0.51</th></tr><tr><th></th><th>0.57</th><th>0.43</th><th>0.41</th><th>0.40</th><th>0.46</th><th>0.45</th><th>0.44</th><th>0.41</th><th>0.42</th><th>0.39</th><th>-</th><th>-</th><th>0.58</th></tr><tr><th></th><th>0.44</th><th>0.41</th><th>0.46</th><th>0.46</th><th>0.44</th><th>0.49</th><th>0.54</th><th>0.39</th><th>-</th><th>-</th><th>-</th><th>-</th><th>-</th></tr><tr><th>Step</th><th>1 - 10</th><th>11 - 20</th><th>21 - 30</th><th>0.43</th><th>0.42</th><th>0.45</th><th>0.45</th><th>0.46</th><th>0.4</th><th>0.4</th><th>0.42</th><th>0.42</th><th>0.42</th></tr><tr><th></th><th>0.41</th><th>0.39</th><th>0.44</th><th>0.46</th><th>0.47</th><th>0.44</th><th>0.42</th><th>0.37</th><th>0.43</th><th>0.49</th><th>0.41</th><th>0.41</th><th>0.41</th></tr></tbody></table>
As for ARX modeling, the best fit corresponds to the model:

$$Y(s) = e^{-0.28s} M(s) \qquad (10)$$

$$M(s) = \frac{61.42s^7 - 1.58e004s^6 - 3.483e005s^5 - 1.785e006s^4}{s^9 + 116s^8 + 5730s^7 + 1.691e005s^6 + 3.198e006s^5 + 4.197e007s^4} + \frac{3.243e007s^3 + 5.992e008s^2 + 3.584e009s + 4.209e009}{+3.656e008s^3 + 2.132e009s^2 + 7.016e009s + 6.316e009}$$

<figure><img src="image_8.png" /><figcaption>**Fig. 6.** ARX-980 model validation (left) on a manual tracking sample (first 20 seconds). Magnitude Bode plot for the human-machine ARX-980 model structure (right).</figcaption></figure>

The human-machine estimated time-delay is 280 ms, while the average time-delay obtained from the impulse and step analysis was, respectively, 440 ms and 437 ms.

Although impulse response analysis is a simple and fast open-loop experimental method, it has some disadvantages, due to augmented saturation, nonlinearities and human fatigue. Step response analysis confirmed the same time-delay values that were obtained in the impulse response experiments. These methods, along with ARX model estimation are still limited modeling techniques, which need additional validation. Frequency analysis can be used to obtain an accurate open-loop LTI model, from a sum of sinusoidal input signals. It is a closed-loop experimental method. Hence, to capture the relevant human-machine dynamic characteristics, the obtained models need to be converted to open-loop ones.

# 6 Conclusions and Further Work

This work investigated the linear modeling methods potential for identifying the human-machine controller characteristics, as required for the design of Human Adaptive Mechatronics (HAM) systems.

An experimental human-machine LabVIEW interface setup has been developed and implemented for collecting data from human operators, as they performed pursuit manual tracking tasks with an analog joystick. As confirmed in previous
work, these obtained models depend on the type and shape of the input tracking
signal (the same occurs with the estimated time-delay). It was also verified that the
time-delay characteristic (for a same individual) is almost constant regardless the
skill level or experience. Hence, pure time-delay can be detached for identification
purposes.

For future work, and to be even more applicable, the linear identification methods
described should be also accomplished from real-time collected task execution data.

References

1. Harashima, F., Suzuki, S.: Human Adaptive Mechatronics - Interaction and Intelligence. In: 9th IEEE Int. Workshop on Advanced Motion Control, pp. 1--8. Istanbul (2006)

2. Habib, M.: Human Adaptive and Friendly Mechatronics (HAFM). In: Proceedings of IEEE Int. Conf. on Mechatronics and Automation, pp. 61--65. Takamatsu (2008)

3. Kado, Y., Pan, Y., Furuta, K.: Control System for Skill Acquisition – Balancing Pendulum based on Human Adaptive Mechatronics. In: IEEE International Conference on Systems, Man, and Cybernetics, pp. 4040--4045. Taipei, Taiwan (2006)

4. Suzuki, S., Harashima, F.: Assist Control and its Tuning Method for Haptic System. In: 9th IEEE Int. Works. on Advanced Motion Control, pp. 374--379. Istanbul, Turkey (2006)

5. Gaines, B.: Linear and Nonlinear Models of the Human Controller. In: International Journal of Man-Machine Studies, 1, pp. 333--360. (1969)

6. McRuer, D., Graham, D., Krendel, E., Reisener, W.: Human Pilot Dynamics in Compensatory Systems. Technical report n.º AFFDL-TR-65-15, Air Force Flight Dynamics Laboratory, Wright-Patterson AFB. Ohio (1965)

7. Ito, M.: Internal model visualized. In: Nature, vol. 403, pp. 153--154. (2000)

8. Suzuki, S., Watanabe, Y., Igarashi, H., Hidaka, K.: Human skill elucidation based on gaze analysis for dynamic manipulation. In: IEEE International Conference on Systems, Man and Cybernetics, pp. 2989--2994. Montreal (2007)

9. Celik, O., Ertugrul, S.: Predictive human operator model to be utilized as a controller using linear, neuro-fuzzy and fuzzy-ARX modeling techniques. In: Engineering Applications of Artificial Intelligence, 23, pp. 595--603. Elsevier (2010)

10. Boer, E., Kenyon, R.: Estimation of Time-Varying Delay Time in Nonstationary Linear Systems: An Approach to Monitor Human Operator Adaptation in Manual Tracking Tasks. In: IEEE Transactions on Systems Man and Cybernetics, vol. 28, n.º 1, pp. 89--99. (1998)

11.Antunes, R., Coito F., Duarte-Ramos, H.: Using Human Dynamics to Improve Operator Performance. In: Emerging Trends in Technological Innovation, pp. 393--400. Springer Boston (2010)

12.Antunes, R., Coito F., Duarte-Ramos, H.: Human-Machine Control Model Approach to Enhance Operator Skills. In: Proceedings of IEEE International Conference on Mechanical and Electrical Technology, pp. 403--407. Singapore (2010)