Wheelchair Control Based on a Polynomial Function
Approximating a User’s Gaze Curve
Sawako. Furuya, Alexander. Alexopoulos, Essameddin. Badreddin, and Tetsuyou. Watanabe,
Member, IEEE

Abstract— We propose a new wheelchair control system basedon a polynomial function approximating a user’s gaze curve.
Conventional studies utilizing gaze data recognized threewheelchair movements: “go straight,” “turn right,” and “turnleft.” However, it was difficult for the system to assess when it
should switch wheelchair motions because the user’s gaze wasalways changing. To solve this problem, we divided “turn right”
and “turn left” wheelchair movements into three groups each inorder to control the wheelchair easily. Consequently, the system
has to recognize seven wheelchair movements: straight, andthree groups each for turning right and left. It is not sufficiency
for a system to assess seven wheelchair movements by only theuser’s gaze. Thus, we developed a wheelchair system considering
not only the user’s gaze but also the angular velocity andacceleration to control wheelchair motions. We approximatedthe user’s gaze using a polynomial function, and calculated thefine gaze angle, angular velocity and acceleration. The
effectiveness of the proposed method was shown by experimental
results.

and just turning to the right from going straight are different.
These motions are both “turn right,” but are different situations.
If these motions are both characterized as “turn right,”
controlling the wheelchair by the user’s gaze becomes difficult.
To overcome this, our study divided the wheelchair
movements of “turning right (left)” into three groups (Fig.1).

I. INTRODUCTION
Recently, many studies have examined moving awheelchair without using hands or arms for older and disabled
persons [1]-[13]. Our study also aims at developing awheelchair control system by using the user’s gaze. The key
design requirements for our control system are

only the horizontal gaze curve is used, andrecognizing seven different wheelchair motions.

Preliminary experiments show that a user’s vertical gazeangle is almost constant even if the wheelchair motion changes.
For stable control, we will construct a control system utilizing
only the horizontal gaze curve.
Conventional systems recognize the three wheelchair
movements of “go straight,”, “turn left,” and “turn right”[4][5]. Many wheelchair studies control the velocity of awheelchair according to the user’s gaze angle. However,controlling velocity according to a user’s gaze is difficult
because it is not always in a constant direction and sometimes
changes because of physiological factors. Thus, the systemneeds to control the velocity of a wheelchair based on a user’s
gaze for each of the wheelchair’s motions. However, different
trajectories exist in a wheelchair’s movement of “turn right(left).” For example, the trajectories of turning right at a corner*Research supported by Kanazawa University and Heidelberg University.
Sawako. Furuya is with Division of Mechanical Science and Engineering,
Kanazawa University, Natural Science Building 3, Nu 7, Kakuma, Kanazawa,
Ishikawa, Japan (corresponding Sawako. Furuya to provide phone: 076-2344684; fax: 076-234-4684; e-mail: s-furuya@stu.kanazawa-u.ac.jp).
Alexander. Alexpoulos is with Heidelberg University, B6 26, Mannheim,
Germany. (e-mail: alexander.alexopoulos@ziti.uni-heidelberg.de).

Figure 1. Deviation of gaze angle during wheelchair movement

Fig. 1 describes the deviation of the horizontal gaze angle androtational velocity of a joystick while a wheelchair is turning
right. By subdividing the wheelchair’s motion when turning
right, the system can easily control the velocity of thewheelchair. In our study, the system recognizes seven
wheelchair movements. However, recognizing seven
wheelchair movements by using only the horizontal gaze is not
sufficient in our study. Thus, our approach is that the system
considers not only the gaze angle but also the angular velocity
and acceleration when it controls wheelchair movement. In
this paper, to compute the gaze angle, angular velocity andacceleration, we propose a method of utilizing a polynomial
Essameddin. Badreddin is with Heidelberg University, B6 26, Mannheim,
Germany. (e-mail: essameddin.badreddin@ ziti.uni-heidelberg.de).
Tetsuyou. Watanabe is with Kanazawa University, Natural Science
Building 3, Nu 7, Kakuma, Kanazawa, Ishikawa, Japan. (e-mail: tewatanabe@ieee.org).

function to approximate the gaze curve. The merits of thismethod are Simplicity
 Expandability
 Noise rejection
Modelling the gaze with a polynomial function is clearlyeasy. Moreover, using a polynomial function approximating
the gaze curve, we can construct a local model, which makes
further extension of the method easy (for example, a model
based on observers can be applied). When the systemcomputes the angular velocity and acceleration, normally,
these values are computing by a differentiated gaze angle.
However, these values also include considerable noise. To
solve this problem, our approach is to compute these values
using a polynomial function approximating the user’s gaze.
This is tested through experiment to compare our approachwith that of using just the differentiated gaze.
A. Related WorkTomari et al. [4] [5] developed a system for severely motor
impaired individuals to operate a wheelchair by using headmotions and gaze directions in a narrow space. This system
was composed of multi-input devices: Kinect and a webcam.
The system is controlled by three wheelchair motions: straight,
right, and left, with large gaze deviation changes
corresponding to each of the wheelchair’s motions. Moreover,
Arai et al. [6] presented a system that detects a user’s
horizontal and vertical gaze directions. After capturing theimage of the user’s pupil, the system analyzes the image todetermine the direction in which the user is looking. Utilizing
both the horizontal and vertical gaze directions, the system isable to affect various wheelchair motions, stop, and pause, bygaze behaviors. These results were applied to the thresholds ofgaze deviation for each of the wheelchair’s motions. Thus, thesystem can recognize wheelchair movements. Purwanto et al.
[7] developed a system to control the velocity of a wheelchairusing the gaze direction. These directions are converted by an
original equation using facial images. Differentiated gazeangles are used to set the velocity, acceleration, and rotation of
the wheelchair. Pai et al. [8] conducted research to control
wheelchair motions by means of pupil images. By analyzing
where a user’s pupil is in an image, the system can control
wheelchair motions. Considering the calibration of thewheelchair is important. Lin et al. [9] noted the way in whichcalibration can be conducted for the wheelchair to be moved
using gaze direction; they developed an interface using image
processing. The system calculates the position of the pupil in
an image in real time. Applying these results to control
wheelchair motions, the system enhanced usability. Until now,
our work has been developed for a wheelchair system using a
Hidden Markov Model (HMM) [11]. We collected much of
the data on a subject’s gaze direction while the wheelchair
moves, and the data is applied to the HMM. System outputs ofsuitable wheelchair motions were based on sample data.
However, correcting sample data requires significant time andwork effort. Our approach does not need to use as muchsample data as our previous study. Our proposed systemshould reduce the time and work required to correct the sample
data considerably.

II. WHEELCHAIR SYSTEM
In this section, the utilized wheelchair system is introduced.
A. Devices
Our study uses the following devices. Fig. 2 left is anelectronic wheelchair made by Otto Bock Healthcare GmbH
that is used as the basis for the gaze-based assistance system.
This electric wheelchair was also utilized in our previous study[10]-[13]; it can estimate the user’s intentions and targeted
goals progressively. Fig. 2 right shows the equipment of Senso
Motoric Instruments GmbH used to capture gaze angles. These
glasses detect the user’s pupil image and convert it to the gazeangle. More detailed information about the wheelchair system
is given in Batrolein et al. [10]-[12].

Figure 2. Electronic wheelchair and pupil glasses

B. Overview of Control System and Problem Definition
An electronic wheelchair has various functions. Fig.3 shows
the detail of the wheelchair structure. In order to have multiple
functions, this system is applied with Recursive Nested
Behavior-based Control (RNBC) [10]-[13] (Fig.3). Fig. 3
shows the control systems using RNBC [12]. By using thehierarchical structure of RNBC, it enables the system toestimate user intention, output wheelchair motions, andperform more functions. These functions are related with eachlevel of the hierarchical structure. Bartolein et al. state,“Mission related global goals need to be decomposed into
spatial sub goals by path planning functionality. To reach theestimated or mission related spatial goal positions, suitabletrajectories must be generated by local navigation, knowing
about the wheelchair’s current position via position update. To
ensure safe drive through the environment, the reference
velocities generated by the motion estimation or the local
navigation can be modified by the underlying collision
avoidance behavior.” (Bartolein et al. 2007). Our study
targeted the white square area in Fig. 3. We developed thesystem of Gaze-based Motion Estimation, Position Update andGaze-based Goal Estimation in Fig. 3. Using data of the gazeangle from sensor input, the system controls wheelchairmotions and outputs wheelchair velocity corresponding to thewheelchair motions.
The Flowchart of our approach is shown in Fig. 4. Let theuser’s horizontal gaze angle from sensor be 𝐸𝐸ℎ (𝑡𝑡) ∈ 𝑅𝑅, gazeangle be 𝐸𝐸(𝑡𝑡) ∈ 𝑅𝑅 ,angular velocity be 𝑉𝑉(𝑡𝑡) ∈ 𝑅𝑅 , angularacceleration be 𝐴𝐴(𝑡𝑡) ∈ 𝑅𝑅 , and wheelchair movements be𝑊𝑊(𝑡𝑡) at time t. The input is the horizontal gaze angle. Usingthe gaze angle and method of Polynomial Function
Approximating user’s Gaze Curve (PFAGC), the systemcomputes the gaze angle, angular velocity and acceleration.
Using these values, the system controls wheelchair
movements. The output is in the form of seven wheelchair
movements. According to the wheelchair movements, thesystem controls the velocity of the wheelchair.
Mission Estimation

Additional
contextual
information

Mission related
global goal

Information of environment

Path Planning
Sub goal

Information of environment

Gaze-based Goal Estimation
Fused goal
location

Gaze direction, Wheelchair pose

A. Seven Wheelchair Motions
The seven wheelchair movements are as follows: Straight,
Curve in Right, Curve Right, Curve out Right, Curve in Left,
Curve Left, and Curve out Left. Fig. 1 shows more detail ofthe subdivided “turn right (left)”. We also set “turn left” thesame as “turn right.” Straight means that the wheelchair goesstraight. Curve in Right and Curve in Left mean that thewheelchair is going to turn right and left at interval (A) in Fig.
1. Curve Right and Curve Left mean that wheelchair justturned right and left at interval (B) in Fig. 1. Curve out Right
and Curve out Left mean that the wheelchair finishes turning
right or left and moves in a straight direction at interval (C) in
Fig. 1. In this study, the system recognizes these seven
wheelchair movements and outputs the velocity of each of thewheelchair’s movements. We summarize the abbreviations
corresponding to wheelchair movements in Table I.

Position Update&Local Navigation
Wheelchair
Pose

TABLE I.

Fused reference
velocities

Gaze direction

Gaze-based Motion Estimation
Eye tracker
data,
User input

Wheelchair Motions

Abbreviation

Straight

ST

Curve in Right

CIR

Curve RightCR

Curve out Right

COR

Curve in Left

CIL

Curve Left

CL

Curve out Left

COL

Overall reference
velocities

Wheelchair velocities

WHEELCHAIR MOTIONS AND ABBREVIATIONSCollision Avoidance
Collision freewheelchair movement

Square area is works of this paper

Figure 3. Structure of recursive nested behavior-based control

Control
Wheelchair
Motions

PFAGC

W

Input: Horizontal gaze angle
Output : Wheelchair Movement W

Figure 4. Structure of this system

III. WHEELCHAIR MOTIONSIn this section, we describe the seven wheelchair motions
and the method of recognizing between them in a wheelchairsystem.

Givin:

B. Thresholds for Controlling Wheelchair Movements
Here, an overview of the method of deciding wheelchairmotions is described. Fig. 5 shows the flowchart of control ofthe seven wheelchair motions. Table II describes the range ofeach of the thresholds for each of the wheelchair’s movements.
To recognize the seven wheelchair motions, we set thresholdsof the gaze angle, angular velocity, and acceleration for eachof the wheelchair motions. Let the minimum thresholds of∗gaze be 𝐸𝐸𝑚𝑚𝑚𝑚𝑚𝑚 , 𝐸𝐸𝑚𝑚𝑚𝑚𝑚𝑚, the maximum thresholds of gaze∗be 𝐸𝐸𝑚𝑚𝑚𝑚𝑚𝑚 , 𝐸𝐸𝑚𝑚𝑚𝑚𝑚𝑚 , the minimum thresholds of angular velocity∗, the maximum thresholds of angular velocity
be 𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚 , 𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚∗
be 𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚 , 𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚, the minimum threshold of angularacceleration be 𝐴𝐴𝑚𝑚𝑚𝑚𝑛𝑛 , and the maximum threshold of angularacceleration be 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 . Preliminary experiments show that the

S

No

L

Yes

No

No
Yes
No

No

Yes

CR

ST

CR

Yes

Yes

CIR

Yes

COR

CL

CIL

Yes
Yes

Yes

No

No

No

Yes

No
Yes

ST

No
0

No

Yes

No

0

COL

W(t)
Figure 5. Flowchart for controlling wheelchair movements

CL CIR

COR

CIL

COL

values of gaze angle, angular velocity, and acceleration are
different according to the wheelchair motions. Thus, weestablish the thresholds for when the system should switch
wheelchair motions; the system can then easily control
wheelchair motions.
The reason we established the L and S groups in Fig. 5
and Table II, the seven wheelchair motions have somefeatures for gaze angle, angular velocity, and acceleration. If
wheelchair motions are CIR, COR, CIL, and COL,
acceleration 𝑨𝑨(𝑡𝑡) at time t is likely to be over 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 or under𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 (the S group in Fig. 5). On the other hands, if wheelchairmotions are ST, CR, and CL, acceleration 𝑨𝑨(𝑡𝑡) is over 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚
and under 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 .Thus, considering acceleration first, thesystem can recognize CIR, COR, CIL, and COL, or ST, CR,
and CL. However, even if the wheelchair movements are ST,
CR, and CL, acceleration 𝑨𝑨(𝑡𝑡)may be over 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 or under𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 . Then, considering 𝑉𝑉(𝑡𝑡) and 𝐸𝐸(𝑡𝑡), the system can deal
with various situations in the S group in Fig. 5. In the L groupin Fig. 5, the main recognition is wheelchair motions of ST,
CR, and CL. Considering 𝐸𝐸(𝑡𝑡), the system can recognize ST
or CR and CL. However, even if wheelchair movements are
CIR, COR, CIL, and COL, acceleration 𝑨𝑨(𝑡𝑡) is likely to beover 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 and under 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 . Thus, we set the other thresholds
of angular velocity 𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚 , 𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚 in order to recognize CR and
CL or CIR, COR, CIL and COL. Utilizing this flowchart, thesystem can recognize the seven wheelchair motions.
TABLE II.

THRESHOLDS OF GAZE AND ANGULAR VELOCITY AND
ACCELERATION FOR EACH OF WHEELCHAIR’S MOTIONS

L

ST

S
L

CIR

S
L

CR

S
L

COR

S
L

CIL

S
L

CL

S
L

COL

S

Gaze Angle

Angular Velocity

AngularAcceleration

𝐸𝐸𝑚𝑚𝑚𝑚𝑚𝑚 ≤ 𝐸𝐸(𝑡𝑡) ≤ 𝐸𝐸𝑚𝑚𝑚𝑚𝑚𝑚

𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚 ≤ 𝑽𝑽(𝑡𝑡) ≤ 𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚

𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 ≤ 𝑨𝑨(𝑡𝑡) ≤ 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚

𝐸𝐸𝑚𝑚𝑚𝑚𝑚𝑚 < 𝐸𝐸(𝑡𝑡)𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚 < 𝑽𝑽(𝑡𝑡)𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 ≤ 𝑨𝑨(𝑡𝑡) ≤ 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚

𝐸𝐸𝑚𝑚𝑚𝑚𝑚𝑚 < 𝐸𝐸(𝑡𝑡)𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚 ≤ 𝑽𝑽(𝑡𝑡) ≤ 𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚

𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 ≤ 𝑨𝑨(𝑡𝑡) ≤ 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚

𝐸𝐸𝑚𝑚𝑚𝑚𝑚𝑚 < 𝐸𝐸(𝑡𝑡)

𝑽𝑽(𝑡𝑡) < 𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚

𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 ≤ 𝑨𝑨(𝑡𝑡) ≤ 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚

𝐸𝐸(𝑡𝑡) < 𝐸𝐸𝑚𝑚𝑚𝑚𝑚𝑚

𝑽𝑽(𝑡𝑡) < 𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚

𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 ≤ 𝑨𝑨(𝑡𝑡) ≤ 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚

𝐸𝐸(𝑡𝑡) < 𝐸𝐸𝑚𝑚𝑚𝑚𝑚𝑚

𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚 ≤ 𝑽𝑽(𝑡𝑡) ≤ 𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚

𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 ≤ 𝑨𝑨(𝑡𝑡) ≤ 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚

𝐸𝐸(𝑡𝑡) < 𝐸𝐸𝑚𝑚𝑚𝑚𝑚𝑚

𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚 < 𝑽𝑽(𝑡𝑡)𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 ≤ 𝑨𝑨(𝑡𝑡) ≤ 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚∗∗
𝐸𝐸𝑚𝑚𝑚𝑚𝑚𝑚≤ 𝐸𝐸(𝑡𝑡) ≤ 𝐸𝐸𝑚𝑚𝑚𝑚𝑚𝑚

0 < 𝐸𝐸(𝑡𝑡)∗
𝐸𝐸𝑚𝑚𝑚𝑚𝑚𝑚< 𝐸𝐸(𝑡𝑡)

0 < 𝐸𝐸(𝑡𝑡)𝐸𝐸(𝑡𝑡) < 0∗
𝐸𝐸(𝑡𝑡) < 𝐸𝐸𝑚𝑚𝑚𝑚𝑚𝑚

𝐸𝐸(𝑡𝑡) < 0∗
𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚≤ 𝑽𝑽(𝑡𝑡) ≤ 𝑉𝑉𝑚𝑚∗𝑚𝑚𝑚𝑚

0 < 𝑽𝑽(𝑡𝑡)∗
𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚≤ 𝑽𝑽(𝑡𝑡) ≤ 𝑉𝑉𝑚𝑚∗𝑚𝑚𝑚𝑚

𝑽𝑽(𝑡𝑡) < 0
𝑽𝑽(𝑡𝑡) < 0∗∗
𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚≤ 𝑽𝑽(𝑡𝑡) ≤ 𝑉𝑉𝑚𝑚𝑚𝑚𝑚𝑚

0 < 𝑽𝑽(𝑡𝑡)

𝑨𝑨(𝑡𝑡) < 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 ,𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 < 𝑨𝑨(𝑡𝑡)

𝑨𝑨(𝑡𝑡) < 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 ,𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 < 𝑨𝑨(𝑡𝑡)

𝑨𝑨(𝑡𝑡) < 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 ,𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 < 𝑨𝑨(𝑡𝑡)

𝑨𝑨(𝑡𝑡) < 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 ,𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 < 𝑨𝑨(𝑡𝑡)
𝑨𝑨(𝑡𝑡) < 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 ,𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 < 𝑨𝑨(𝑡𝑡)

𝑨𝑨(𝑡𝑡) < 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 ,𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 < 𝑨𝑨(𝑡𝑡)
𝑨𝑨(𝑡𝑡) < 𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 ,𝐴𝐴𝑚𝑚𝑚𝑚𝑚𝑚 < 𝑨𝑨(𝑡𝑡)Consider a time t = 𝑡𝑡𝑛𝑛 where we have n gaze data points
𝑃𝑃𝑖𝑖 (𝑡𝑡𝑖𝑖 , 𝐸𝐸h (𝑡𝑡𝑖𝑖 )) =(𝑡𝑡1 , 𝐸𝐸h (𝑡𝑡1 )), (𝑡𝑡2 , 𝐸𝐸h (𝑡𝑡2 )), (𝑡𝑡3 , 𝐸𝐸h (𝑡𝑡3 )), … , (𝑡𝑡𝑛𝑛 , 𝐸𝐸h (𝑡𝑡𝑛𝑛 )) ( 𝑡𝑡𝑖𝑖 <𝑡𝑡𝑖𝑖+1 , 𝑖𝑖 = 1,2,3, … 𝑛𝑛 − 1). Then n gaze data points 𝑃𝑃𝑖𝑖 (𝑡𝑡𝑖𝑖 , 𝐸𝐸ℎ𝑡𝑡 )are described by a polynomial function:
𝐸𝐸(𝑡𝑡) = 𝑎𝑎0 + 𝑎𝑎1 𝑡𝑡𝑖𝑖 + 𝑎𝑎2 𝑡𝑡𝑖𝑖2 + ⋯ + 𝑎𝑎𝑚𝑚 𝑡𝑡𝑖𝑖𝑚𝑚 ,(1)

where m is the dimension of the polynomial function, and𝑎𝑎0 , 𝑎𝑎1 , 𝑎𝑎2 , … 𝑎𝑎𝑚𝑚 aretheconstants.
Letabe[𝑎𝑎0 𝑎𝑎1 𝑎𝑎2 ⋯ 𝑎𝑎𝑛𝑛 ]𝑇𝑇 . We will derive a, which minimizes
the following squared error:𝑛𝑛� (𝑡𝑡𝑖𝑖 ) − 𝐸𝐸ℎ (𝑡𝑡𝑖𝑖 )�2
𝑒𝑒 = argmin ��E
𝒂𝒂

where

From

𝑖𝑖=1
= argmin 𝒂𝒂𝑇𝑇 𝑾𝑾𝑾𝑾 − 2𝒂𝒂𝑇𝑇 𝒃𝒃 + ∑𝑛𝑛𝑖𝑖=1 𝐸𝐸ℎ (𝑡𝑡𝑖𝑖 )2 ,
𝒂𝒂

∂e

∂𝒂𝒂

𝑾𝑾 = ∑𝑛𝑛𝑖𝑖=1 𝒄𝒄i 𝐜𝐜i𝑇𝑇 ,𝒃𝒃 = ∑𝑛𝑛𝑖𝑖=1 𝐸𝐸ℎ (𝑡𝑡𝑖𝑖 )𝒄𝒄i ,

𝒄𝒄i = �1 𝑡𝑡

𝑡𝑡 2

(3)
𝑇𝑇

⋯ 𝑡𝑡 𝑚𝑚 � .

= 0, the solution is represented by:

𝒂𝒂 = 𝑾𝑾+ 𝒃𝒃.

(2)

(4)

In our approach, first we apply the gaze angle data with alow-pass filter. Next, we set the filtered gaze curve as inequation (2) by using curve fitting and then calculate theangular velocity and acceleration. Let angular velocity be𝑽𝑽(𝑡𝑡) and angular acceleration be 𝑨𝑨(𝑡𝑡) at time t. When thedimension of the polynomial function m = 2, equation (5) is
set as equation (1):2
𝐸𝐸(𝑡𝑡) = 𝑎𝑎0 + 𝑎𝑎1 𝑡𝑡𝑛𝑛+1 + 𝑎𝑎2 𝑡𝑡𝑛𝑛+1.
(5)

Variables 𝑎𝑎0 , 𝑎𝑎1 , 𝑎𝑎2 are calculated using equation (4). For thesimplicity, we set m =2. Then, equation (5) is differentiatedas follows:(6)
𝑽𝑽(𝑡𝑡) = 2𝑎𝑎2 𝑡𝑡𝑛𝑛+1 + 𝑎𝑎1 ,A(t)= 2𝑎𝑎2 .

(7)Angular velocity 𝑽𝑽(𝑡𝑡) is calculated using equation (6) andacceleration A(t) is computed using equation (7). We named
this procedure of computing these values as PFAGC. The
merit of this algorithm is its consideration of past gaze angles.
Thus, this method has less noise than when just thedifferentiated angular velocity and acceleration of gaze areused. The significance of using a polynomial function for
curve fitting is that the gaze curve is similar to a polynomial.
Thus, by using a polynomial function, we get the ideal gaze
curve, removing outlier data.

IV. POLYNOMIAL FUNCTION APPROXIMATING USER’S GAZE

V. EXPERIMENTCURVE

In this section, we demonstrate the algorithm on awheelchair system. In conducting the experiment, wemeasure whether the system controls each of the wheelchairmotions or not according to the thresholds of gaze angle,
angular velocity, and acceleration. We conducted thisexperiment on a Windows PC, not on an electronic
wheelchair. The experimental device is a Windows 8.1
system with an, Intel® Core™ i7-4790 CPU. System RAM is
16 GB. The system is developed using C/C++. The system set
up is as follows: the number of point for curve fitting is eight,

In this section, we explain the method of deriving apolynomial function approximating a gaze curve and deriving
the velocity and acceleration. The purpose of theapproximation is to derive noise-reduced values for angularvelocity and acceleration. In our study, to achieve these
purposes, gaze angle, angular velocity and acceleration arecomputed by regression of the curve of gaze using apolynomial function. Here is the procedure.

and the sampling time is 0.1 [s]. The gaze angle data wascaptured on the wheelchair by a joystick beforehand. The
subject went straight, turned right, and then went straightagain (Fig. 6)

Figure 6. Road Map

The reference value is the rotation velocity of the joystick.
Comparing it with our approached method, we confirm
whether the system can output the exact wheelchair motions.
Moreover, we compare the results with other methods ofcomputing angular velocity and acceleration. One is themethod where the gaze angle is differentiated. The angularvelocity is simply the differentiated horizontal gaze data, andthe angular acceleration is differentiated by the angularvelocity. In this case, the input gaze data is not applied with alow-pass filter. We define this method as “Differential Without
Low-Pass Filter (DWLPF)”. The other algorithm also
differentiates the gaze angle; however, the gaze angle isapplied with a low-pass filter. We call this method“Differential After Low-Pass Filter (DALPF)”. We thencompared these three algorithms to determine which one is themost useful.
The result of the experiment is described in Fig. 7. Fig. 7 (a)is a graph of the horizontal gaze. Fig. 7 (b) is the graph of theangular velocity. Fig. 7 (c) is the graph of the angularacceleration. Fig. 7 (d) is the graph of the angular acceleration
except for the values of DWLPF. Because the values of
DWLPF were large, we excluded DWLPF in (d). Fig. 7 (e)shows the wheelchair movements. Because it is difficult tocompare each data because of the values of DWLPF, Fig. 7 (f)shows the wheelchair movements excluding the data of
DWLPF. Fig. 7 (g) is the rotation velocity of the joystick. The
gray curve line is the original horizontal gaze angle fromsensor. In addition, the blue curve line shows the algorithm of
PFAGC, the red curve line is DALPF, the green curve line
shows the algorithm of DWLPF, and the yellow curve line is
the rotation velocity of the joystick.
Essentially, the joystick data is the rotation velocity.
However, the outputs of the wheelchair’s motions are theseven motions. To compare the joystick and other algorithm
data easily, we made modifications discretely and arbitrarily
using the condition of the joystick. When the value of thejoystick is between -0.1 [rad/s] and 0.1 [rad/s], it is evaluatedas going straight (ST). If it is less than -0.1[rad/s], it isevaluated as Curve in Left (CIL). If it is between 0.1 [rad/s]
and 0.3 [rad/s], it is evaluated as Curve in Right (CIR) or Curve
out Right (COR). If it is over 0.3 [rad/s], it is evaluated as
Curve Right (CR). Using these conditions, we set the rotation
velocity of the joystick as the wheelchair’s motions. In terms

Figure 7.

Results of experiment: comparison of three algorithms

of the rotation velocity of the joystick for the time between 0[s] to 4 [s], the value is changed up and down; however, thewheelchair actually went straight. This was caused by the
position of the wheelchair’s wheels. When the wheelchair
moved straight, for the time between 0 [s] to 4 [s], the wheels
were not in the straight direction. The wheels changed wheel
position to straight, the wheelchair and joystick were vibrated.
By this condition, we compared the joystick results to theother algorithms. Considering these conditions, we evaluate
these results comparing our approach with the other methods.
First, seeing (b) and (c) in Fig. 7, the angular velocity andacceleration computed by DWLPF includes noise. As theresult, the output of wheelchair movements is unstable andchanged continuously. However, the values with PFAGC
removed more noise than DWLPF. Thus, the output ofwheelchair movements is stable. As a result, DWLPF is notsuitable to compute the angular velocity and acceleration. Next,
we compare PFAGC with DALPF. Comparing our proposed
algorithm to the method of DALPF, the value of the angularvelocity is almost the same but the angular acceleration isdifferent. The one with DALPF includes more noise than
PFAGC. In reviewing (e) in Fig. 7, the wheelchair motions
calculated by DALPF are unstable continuously. Moreover,when the wheelchair movements changed from ST to CIR andfrom CIR to CR DALPF showed more delay than PFAGC.
Thus, PFAGC is more suitable than DALPF in the wheelchairsystem.
Finally, we compare wheelchair movements of the joystickwith the three algorithms. When comparing commands ofwheelchair motions derived by DWLPF and the joystick,
DWLPF is obviously unstable. DWLPF changed wheelchair
movements every time and was not stable. Wheelchair
movements of DALPF seemed almost the same as the joystick,however, the wheelchair motions changed sometimes. The
wheelchair movements of PFAGC are close to the joystick
results. As a result of this experiment, PFAGC is useful in our
proposed wheelchair system.
The reason for this result is as follows. When angularvelocity and acceleration are calculated by PFAGC, PFAGC
uses the polynomial function of the gaze curve. It includes pastgaze data and removes errors. Thus, PFAGC can computes theangular velocity and acceleration while removing noise more
effectively than DALPF. DALPF applies the low-pass filter.
Even if applied with the low-pass filter, DALPF cannot
remove sufficient noise.
VI. CONCLUSION
In this paper, we developed a wheelchair control systemusing a polynomial function approximating the user’s gaze
curve. The key design elements of our control system were 1)
only the horizontal gaze curve is used and 2) system
recognizes seven different wheelchair motions. Wheelchairsystems of conventional studies recognized three wheelchairmotions by only using the horizontal gaze. However, it wasdifficult for our wheelchair system to control seven
wheelchair motions by using only the horizontal gaze. To
solve this problem, our system considered not only thehorizontal gaze angle but also the angular velocity andacceleration to control the seven wheelchair motions. We

approximated the gaze using a polynomial function,
determining the fine angular velocity and acceleration. The
purposes of using a polynomial function approximation were
“Simplicity,” “Expandability,” and “Noise Rejection.” As theresult of the experiment, in terms of the data of gaze angle,
angular velocity and acceleration, our approach removed
more noise than just differentiation. Moreover, the system
output wheelchair movements with stability resembling thatof a joystick. Thus, we achieved the purpose of developing asystem using a polynomial function approximation. Our
system requires thresholds of gaze, angular velocity, andacceleration to control each of the wheelchair’s motions.
These thresholds are different for each user. Before using oursystem for a user for the first time, we have to set up thethresholds manually. In future, we are going to develop asystem to control these thresholds automatically for each user.
Another future work might be to estimate whether gazemovement is related with wheel chair control or not.
REFERENCES[1] L. Montesano, M. Diaz, S. Bhaskar, and J. Minguez, “Towards anIntelligent Wheelchair System for Users With Cerebral Palsy,” IEEE
Transactions on Neural Systems and Rehabilitation Engineering, Vol.
18, No.2, April, 2010.
[2] E. Wästlund, K. Sponseller, and O. Pettersson, “What You See is
Where You Go: Testing a Gaze-Driven Power Wheelchair for
Indivisuals with Severe Multiple Disabilities,” ETRA’10 Proceedingsof the 2010 Symposium on Eye-Tracking Research & Applications, pp.
133-136.
[3] P. Novák, T. Krajník, L. Přeučil, M. Fejtová, and O. Stépánková, “AI
Support for a Gaze Controlled Wheelchair,” The 4th Conference on
Communication by Gaze Interaction – COGAIN, 2008.
[4] M. R. M Tomari, Y. Kobayashi, and Y. Kuno, “Development of Smart
Wheelchair System for a User with Severe Motor Impaorment,”
International Symposium on Robotics and Intelligent Sensors 2012,
Procedia Engineering 41, pp. 538-546.
[5] R. Tomari, Y. Kobayashi, Y. Kuno, “A Framework for Controlling
Wheelchair Motion by using Gaze Information,” International Journalof Integrated Engineering, vol. 5, no. 3, pp. 40-45, 2013.
[6] K. Arai, and R. Mardiyanto, “A Prototype pf Electric Wheelchair
Contorlled by Eye-Only for Paralyzed User,” Journal of Robotics andMechatronics, vol. 23, No.1, 2011.
[7] D. Purwanto, R. Mardiyanto, and K. Arai, “Electric wheelchair control
with gaze direction and eye blinking,” Artif Life and Robotics,
December 2009.
[8] S. Pai, S. Ayare, and R. Kapadia, “Eye Controlled Wheelchair,”International Journal of Scientific & Engineering Research, vol. 3,
October 2012.
[9] C. S. Lin, C. W. Ho, W.C. Chen, C. C. Chiu, and N. S. Yen, “Powered
wheelchair controlled by eye-tracking system,” Optica Application, vol.
XXXVI, no. 2-3, 2006.
[10] C. Bartolein, A. Wagner, M. Jipp, and E. Badreddin, “Multilevel
Intention Estimation for Wheelchair Control,” In Proc.of the European
Control Conference, Koa, Greece, 2007.
[11] C. Bartolein, A. Wagner, M. Jipp, and E. Badreddin, “Easing
Wheelchair Contoril by Gaze-Based Estimation of Intended Motion,”
Proceedings of the 17th World Congress The International Federationof Automatic Contorl Seoul, Korea, pp. 9162-9167, July 2008.
[12] E. Badreddin, “Recursive behavior-based architecture for mobile
robots,” Robotics and Autonomous Systems, vol. 8, 1991.
[13] M. Jipp, E. Badreddin, C. Bartolein, A Wagner, and W. W. Wittmann,“Congnitive Modeling to Enhance Usability of Complex Technical
Systems in Rehabilitation Technology on the Example of a Wheelchair,”
In Proc. of the 2nd Int. Conference on Mechatronics, vol. 2, pp. 11651172, 2005.