# Tracking of Human Arm Based on MEMS Sensors

Yuxiang Zhang, Liuyi Ma, Tongda Zhang, Fuhou Xu

► To cite this version:

Yuxiang Zhang, Liuyi Ma, Tongda Zhang, Fuhou Xu. Tracking of Human Arm Based on MEMS Sensors. 4th Conference on Computer and Computing Technologies in Agriculture (CCTA), Oct 2010, Nanchang, China. pp.597-606, 10.1007/978-3-642-18336-2_73. hal-01562759

HAL Id: hal-01562759

https://inria.hal.science/hal-01562759

Submitted on 17 Jul 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Tracking of Human Arm Based on MEMS Sensors

Yuxiang Zhang¹, Liuyi Ma¹, Tongda Zhang², Fuhou Xu¹

¹203 office, Xi'an Research Inst. of Hi-Tech Hongqing Town,
Xi'an, 710025 P.R.China

²Department of Automation, Tsinghua University,
Beijing, 100084, P.R.China
yuxiangz@tom.com, cuiqin_wu@163.com,
ZT2008@qq.com, bainetsi@foxmail.com

**Abstract.** This paper studied the method for motion tracking of arm using triaxial accelerometer, triaxial gyroscope and electronic compass. The motion model of arm is established. The hardware of tracking system of arm is designed. The track method of arm gesture based on multi-sensors data fusion is analyzed. The compensation algorithm for motion accelerations is researched. The experimental results demonstrate that the motion acceleration compensation algorithm is validity, which can improve the dynamic measure precision of arm gesture angle.

**Keywords:** MEMS, movement track, gesture fusion, lever arm effect.

## 1 Introduction

The human body movement track technology has been widely used in the areas of virtual reality and game development. At present, the human body movement track technology is mainly on the basis of pattern recognition technology or on wearable equipment technology. The human body movement track technology based on pattern recognition only need simple equipment and do not affect the personnel movement, but it has following shortcomings: it is rigorous to the recognition environment and it has blind angle when not in the open space; distinguishes; The recognition is affected under circumstance of occlusion; The recognition algorithm is hard to reach high precision. The wearable equipment used in movement track technology includes mechanical equipment and electromagnetism equipment. The mechanical devices adopt the support installation, which will restrict user's activity. The electromagnetism equipment is easy to be disturbed by the magnetic field, and is not suitable for complex electromagnetism environment.

With the development of MEMS technology and semiconductor technology, the mini-accelerator sensors have been used in the human motion track. The movement track technology based on MEMS sensors has the advantages of frequency bandwidth, stability small size, low cost, long life and light weight. Many researches have concerned the technology. Hou W. s., Dai J.m. used triaxial accelerometer to track the human motion. Because the accelerometer is easy to be disturbed by motion
acceleration and can't detect the change of horizontal angle, the effect of their method is not good always. W.w. Feng [2] used both the magnetic sensors and accelerometer to track the motion of upper limb. Since the magnetic is week and easy to be disturbed by environment, so the measurement is not accurate. Ryan Aylward [3] applied both triaxial accelerometer and triaxial gyroscope to measure the attitude of dancers. Liu X.y., Zhou Z.y.[4] add the triaxial magnetic into micro inertial measurement element to detect the motion of lower limbs. However, when the limbs have motion acceleration, their method will produce a relative large error. To obtain high precision measurement result, the compensation of acceleration is necessary.

This paper studies the arm movement track technology based on the triaxial accelerometer, triaxial gyroscope and electronic compass, and researches the compensate algorithm for the accelerate component of arm movement. The arm movement track system is designed. The experiments show that the movement track effect is good.

## 2 The motion model of arm and the movement track principle

The movement of arm can be simplified to the rotation under elbow joint and shoulder joint. The simplified arm is a linkage mechanism with two joints (figure 1). Taking the shoulder joint as origin, the reference coordinate system $O_b$ - $X_b$ $Y_b$ $Z_b$ and linkage coordinate system $O_{p1}$ - $X_{p1}$ $Y_{p1}$ $Z_{p1}$ are established. The references coordinate system $O_b$ is parallel to the human body, and is constant in the process of arm movement. The linkage coordinate system $O_{p1}$ rotate with the big arm, and is described with pitch angle $\theta_1$, roll angle $\phi_1$, course angle $\psi_1$. The position of big arm is as following:

$$ \begin{bmatrix} x_b \\ y_b \\ z_b \end{bmatrix} = T_1^{-1} \begin{bmatrix} x_{p1} \\ y_{p1} \\ z_{p1} \end{bmatrix}. \qquad (1) $$

$$ T_1 = \begin{bmatrix} \cos\phi_1 \cos\psi_1 - \sin\phi_1 \sin\theta_1 \sin\psi_1 & -\cos\theta_1 \sin\psi_1 & \sin\phi_1 \cos\psi_1 + \cos\phi_1 \sin\theta_1 \sin\psi_1 \\ \cos\phi_1 \sin\psi_1 + \sin\phi_1 \sin\theta_1 \cos\psi_1 & \cos\theta_1 \cos\psi_1 & \sin\phi_1 \sin\psi_1 - \cos\phi_1 \sin\theta_1 \cos\psi_1 \\ -\sin\phi_1 \cos\theta_1 & \sin\theta_1 & \cos\phi_1 \cos\theta_1 \end{bmatrix}. \quad (2) $$
<figure><img src="image_2.png" /><figcaption>Fig. 1. The motion model of arm</figcaption></figure>

where, $T_1$ is the strapdown matrix between the coordinate system $O_{p1}$ and the coordinate system $O_b$. The position of big arm can be obtained by solving the $T_1$ real time.

Because the roll angle does not affect the position change of elbow joint, the roll angle of main arm can be considered as 0. The position of elbow joint can be expressed as:

$$ \begin{bmatrix} x_{op2} \\ y_{op2} \\ z_{op2} \end{bmatrix} = \begin{bmatrix} \cos \psi_1 & \sin \psi_1 & 0 \\ -\cos \theta_1 \sin \psi_1 & \cos \theta_1 \cos \psi_1 & \sin \theta_1 \\ \sin \theta_1 \sin \psi_1 & \sin \theta_1 \cos \psi_1 & \cos \theta_1 \end{bmatrix} \begin{bmatrix} 0 \\ l_1 \\ 0 \end{bmatrix} = l_1 \begin{bmatrix} \sin \psi_1 \\ \cos \theta_1 \cos \psi_1 \\ \sin \theta_1 \cos \psi_1 \end{bmatrix}. \quad (3) $$

where, $l_1$ is the length of big arm.

The linkage coordinate system $O_{p2} - X_{p2}Y_{p2}Z_{p2}$ is established at elbow joint $O_{p2}$. The linkage coordinate system $O_{p2}$ rotates with the forearm. The forearm can be described with pitch angle $\theta_2$, roll angle $\phi_2$, course angle $\psi_2$, the position of forearm can be expressed as:

$$ \begin{bmatrix} x_b \\ y_b \\ z_b \end{bmatrix} = T_2^{-1} \begin{bmatrix} x_{p2} \\ y_{p2} \\ z_{p2} \end{bmatrix} + \begin{bmatrix} x_{op2} \\ y_{op2} \\ z_{op2} \end{bmatrix} = T_2^{-1} \begin{bmatrix} x_{p2} \\ y_{p2} \\ z_{p2} \end{bmatrix} + l_1 \begin{bmatrix} \sin \psi_1 \\ \cos \theta_1 \cos \psi_1 \\ \sin \theta_1 \cos \psi_1 \end{bmatrix}. \quad (4) $$

where, is the $T_2$ is the strapdown matrix between the coordinate system $O_{p2}$ and the coordinate system $O_b$.
## 3 Hardware of arm movement track system

The arm movement track system consists of big arm gesture detection module, forearm gesture detection module and the host computer. The hardware frame is shown in figure 2. The big arm detection module and the forearm detection module are fixed on the big arm and forearm, respectively. The pitch angle, roll angle and the course angle data of big arm and forearm can be measured, and then sent to the host computer by wireless module. According to the motion models of arm, the position of arm can be obtained by the position solver. Hence, the movement track of arm can be realized.

<figure><img src="image_3.png" /><figcaption>Fig. 2. The arm movement track system hardware frame</figcaption></figure>

The arm gesture detection modules are consist of triaxial accelerometer, triaxial gyroscope, triaxial magnetic intensity sensors, data sample module, microprocessor and wireless transmit module. The hardware frame is shown in figure 3.

<figure><img src="image_4.png" /><figcaption>Fig. 3. The arm gesture detection module hardware frame</figcaption></figure>

The triaxial accelerometer ADXL330 is produced by AD corp, which has a measure range of ± 3g, with a sensitivity of 300mV/g. The gyroscope LPR530AI (with a range of ± 300 o/s) and LY530ALH (with a range of ± 300 o/s) are produced by ST corp. The electronic compass sensor HMC5843 (with a range of ± 4gauss) is produced by Honeywell corp.

The data sample module uses the ADS8344 (16 bit, 8 channels) which is produced by TI corp., it is used for the A/D translation of the triaxial accelerometer and the triaxial gyroscope. The microprocessor is STM32F103RBT6, which is used for the noise reduction, position solver and data fusion. The results from the microprocessor are sent to the host computer by wireless module RF24L01.

The gesture detection module is shown in figure 4.
<figure><img src="image_5.png" /><figcaption>Fig. 4. The arm gesture detection module</figcaption></figure>

# 4 The arm gesture movement track algorithm based on multi-sensors

## 4.1 Multi-sensors data fusion based on Kalman filter

The gesture fusion algorithm of arm consists of three parts. Firstly, according the angle velocity vector measured by the triaxial gyroscope, the gesture angle is solved by the quaternion rotation vector three samples method which has good compensate effect to noncommutativity error; secondly, according the observe values of gravity vector and geomagnetism vector from the triaxial accelerometer, triaxial geomagnetism sensors, the gesture angle and the course angle of arm can be obtained; finally, the arm gesture optimal estimation can be obtained by the data fusion of the gesture angle and the course angle using Kalman filter. Since the rotational angle is big and the change of gesture angle is quick, this paper adopts the closed loop correction indirect method Kalman filter to realize the data fusion of arm gesture [5].

$$ \bar{\omega}_{true} = \bar{\omega}_{m} - \bar{b}_{true} - \bar{n}_{w1} . \qquad (5) $$

$$ \bar{\omega}_{est} = \bar{\omega}_{m} - \bar{b}_{est} . \qquad (6-a) $$

$$ \bar{b} = -\frac{1}{\tau}\bar{b} - \bar{n}_{w2} . \qquad (6-b) $$

where, $\bar{n}_{w1}$ is the measurement noise of gyroscope, $\bar{n}_{w2}$ is the drift noise of gyroscope, $\tau$ is the drift time.

Take the error angle and the gyroscope drift as status variable $X = [\Delta\theta \ \Delta\varphi \ \Delta\psi \ \Delta b_x \ \Delta b_y \ \Delta b_z]^T$. The system equation is established as following:

$$ X = AX(t) + \Gamma W(t) . \qquad (7) $$
where,

$$A = \begin{bmatrix} [\bar{\omega} \times] & -I_{3 \times 3} \\ 0_{3 \times 3} & -\frac{1}{\tau}I_{3 \times 3} \end{bmatrix} \qquad (7-a)$$

$$\Gamma = \begin{bmatrix} -I_{3 \times 3} & 0_{3 \times 3} \\ 0_{3 \times 3} & I_{3 \times 3} \end{bmatrix}$$

$$W(t) = [\bar{n}_{w1} \quad \bar{n}_{w2}]^T .$$

$W(t)$ is white noise with zero-mean-value, and $E[W(t)W(t)^T] = Q$, $[\bar{\omega} \times]$ is the symmetric matrix of $\bar{\omega}$.

When the object is in non-accelerated status, we can get the error angle $\Delta\theta_m$ which is the difference between the observed values from the accelerometer or magnetic sensors and the calculation value from gyroscope. We can establish the Observation Equation

$$Z_K = HX_K + V_K, \qquad (8)$$

where, $H = [I_{3 \times 3} \quad 0_{3 \times 3}]$, $V_K$ is white noise with zero-mean-value, $E[V_K V_K^T] = R$.

According to Eqs.(7), (8), we can design the Kalman filter to estimate the error angle of arm gesture. By taking error angle into the arm gesture quaternion, we can realize the correction of calculation values from gyroscopes.

## 4.2 Lever arm effect and acceleration compensation of arm gesture detection module

When the object is in non-accelerated status, the output of accelerometers have no acceleration components, the accelerometer can be used as inclinometer, the output is the gesture angle. By using the Kalman algorithm proposed in section 4.1, we can realize the good estimation of gesture angle and the gyroscopes drift, and then correct the gesture angle and the gyroscope drift. In our movement track system, the accelerometer can't be installed at the joints of arm. Hence, when the arm is moving, a big deviation is produced due to the normal and tangential acceleration. The deviation will cause the gesture estimation error increased or even failure, which known as lever arm effect.

### 4.2.1 lever arm effect

Lever arm effect will produce if the inertial navigation system is not installed at the center of rolling motion, which can be described by the following model [6],
<figure><img src="image_6.png" /><figcaption>Fig. 5. The sketch of installation position</figcaption></figure>

Show as in figure 5, define the static coordinate system $O_i - X_i Y_i Z_i$ and the dynamic coordinate system $O_n - X_n Y_n Z_n$, and consider $O_n$ as the roll center. The accelerometer of inertial navigation system is installed on the fix point P. $\vec{r}_p$ is the position vector of point P in dynamic coordinate system, and the acceleration $\vec{\delta f}$ of point P in static coordinate system can be described as:

$$ \vec{\delta f} = \dot{\bar{\omega}}_{in} \times \vec{r}_p + \bar{\omega}_{in} \times (\bar{\omega}_{in} \times \vec{r}_p) . \qquad (9) $$

where, $\dot{\bar{\omega}}_{in} \times \vec{r}_p$ is the tangential acceleration, $\bar{\omega}_{in} \times (\bar{\omega}_{in} \times \vec{r}_p)$ is the normal acceleration. $\bar{\omega}_{in}$ is the angle velocity vector of the dynamic coordinate system relative to the static coordinate system, $\dot{\bar{\omega}}_{in}$ is the angle acceleration of the dynamic coordinate system relative to the static coordinate system.

### 4.2.2 motion acceleration compensation of big arm detection module

The output of accelerometer in big arm gesture detection module $\vec{a}_1$ can be expressed as

$$ \vec{a}_1 = \vec{\delta f}_1 + T_1 \vec{a}_{op1} + \vec{g} + \vec{v}_1 . \qquad (10) $$

Where, $\vec{a}_{op1}$ is the line acceleration of shoulder joint $O_{p1}$ in reference coordinate system $O_b$, $\vec{\delta f}_1$ is the acceleration of big arm lever arm, $\vec{g}$ is the component of gravity, $\vec{v}_1$ is the measure noise. $\vec{\delta f}_1$ and $T_1 \vec{a}_{op1}$ is namely motion acceleration components, which will affect the observation precision of gesture angle, hence, a compensation is needed. When the arm is moving, the shoulder joint can consider as in static, namely, $\vec{a}_{op1}=0$, big arm lever arm acceleration can be calculated by Eq(9).
The compensate method of acceleration components of big arm gesture detection module can be expressed as:

$$ \hat{\bar{a}}_1 = \bar{a}_1 - \delta\bar{f}_1 . \qquad (11) $$

where, $\hat{\bar{a}}_1$ is the observation value of gravity component of big arm gesture detection module by compensation.

### 4.2.3 motion acceleration compensation of forearm detection module

The accelerometer output of forearm detection module is affected by gravity components, forearm lever arm acceleration, shoulder joint acceleration, measure noise and the line acceleration of elbow joint $\bar{a}_{op2}$, which can be described as following

$$ \bar{a}_2 = \delta\bar{f}_2 + T_2\bar{a}_{op1} + T_2\bar{a}_{op2} + \bar{g} + \bar{v}_2 . \qquad (12) $$

According to Eq.(11), the main part of $\bar{a}_{op2}$ is the lever acceleration caused by the rotation of big arm, which can be calculated as:

$$ \bar{a}_{op2} = T_1^{-1} \delta\bar{f}_{op2} . \qquad (13) $$

$$ \delta\bar{f}_{op2} = \dot{\bar{\omega}}_1 \times \bar{r}_{op2} + \bar{\omega}_1 \times (\bar{\omega}_1 \times \bar{r}_{op2}) . \qquad (14) $$

where, $\bar{r}_{op2}$ is the position vector of elbow joint $O_{p2}$ in coordinate system $O_{p1}$, $\bar{\omega}_1$ is the rotational angle velocity vector of big arm. The compensate method of acceleration components of forearm gesture detection module can be expressed as:

$$ \hat{\bar{a}}_2 = \bar{a}_2 - T_2 T_1^{-1} \delta\bar{f}_{OP2} - \delta\bar{f}_2 . \qquad (15) $$

where, $\hat{\bar{a}}_2$ is the observation value of gravity component of forearm gesture detection module by compensation.

## 4.3 Experiments and results

In order to demonstrate the validity of compensation algorithm, we fix the big arm detection module and forearm detection module upon big arm and forearm, respectively. The arm is in random motion status. The observation effect of gravity vector by accelerometer is justified by following equation

$$ \varepsilon = \sqrt{a_x^2 + a_y^2 + a_z^2} - g . \qquad (16) $$

If $\varepsilon$ has a relative big absolute value, the outputs of accelerometers have relative big motion components, and the outputs can't be used for gesture calculation directly.
Figure 6 and figure 7 are the motion acceleration compensation effect of big arm and forearm, respectively. From figure 6 and figure 7, we can see that the peek value of ε is decreased from 0.7g to 0.1g by compensation, the motion acceleration components is filtered effectively. In order to study the influence of motion acceleration to the arm gesture observation, we compared the calculated gesture angle using origin acceleration as input and that using compensated acceleration. The results are shown in figure 6b, 6c and figure 7b, 7c. When the arm gesture changes slowly, the results are consistent, while the arm gesture change fast, the results that use origin acceleration as input will produce big error. The big arm gesture error peek is near 20 degree, and the forearm gesture error peek is near 50 degree.

<figure><img src="image_7.png" /><figcaption>(6a)</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>(7a)</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>(6b)</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>(7b)</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>(6c)</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>(7c)</figcaption></figure>

**Fig. 6.** The effect of acceleration compensation for big arm

**Fig. 7.** The effect of acceleration compensation for forearm
If the gesture angle with big error is introduced into the Kalman filter in section 4.1, the arm gesture angle estimation will produce big error. Therefore, the measure precision of arm gesture can be improved by the acceleration compensation method proposed in this paper.

# 5 The arm gesture movement track algorithm based on multi-sensors

According to the motion characteristic of arm, this paper established the motion model of arm. On the basis of triaxial accelerometer, triaxial gyroscope and triaxial magnetic field sensors, the arm movement track system hardware is designed. The arm gesture track method based on multi-sensors data fusion is researched. The motion acceleration components compensation algorithm is analyzed. The arm movement track experiments show that: when arm move quickly, the existence of arm motion acceleration will cause a big measure error of arm gesture; Measure error peek of big arm is about 20 degrees and that of forearm is about 50 degrees; the proposed compensation method in this paper can filter the motion acceleration component effectively, and then improve the arm track precision.

# References

1. Hou Wensheng, Dai Jiaman: Detection of human upper limb motion gesture based on acceleration sensor: Transducer and Microsystem Technologies, 1,106--108 (2009) (in Chinese)

2. Wenwu Feng: Design and Implementation of the Motion Attitude Detection system, Master degree, Chongqing university (2008) (in Chinese)

3. RYAN A, JOSEPH A P: A wireless, compact, multi-user system for interactive dance, Proceeding of the 2006 international conference on new interfaces for musical expression, Paris, France, pp. 134-139 (2006)

4. Liu Xiaoyan, Zhou Zhaoying: Design and Test of MEMS Attitude Measurement Unit for Human Motion Tracking: Instrument Technique and Sensor, 8, 225-227. (2009) (in Chinese)

5. Zhang Xiaona: Research on the Key Technologies of Economic Strapdown Attitude Heading Reference System: Master degree, Xi-Dian University, Xi'an, China, (2008) (in Chinese)

6. Gan Shuai: Research on the Lever Arm Effect of Speed Matching Transfer Alignment, Master degree, Harbin Engineering University, Harbin, China (2009) (in Chinese)

7. Feng Zhiquan: Research on 3D hand tracking method, Master degree, Shandong University, China (2006) (in Chinese)