# Assisting power wheelchair driving on a sidewalk: A proof of concept

Louise Devigne, François Pasteau, Nicolas Le Borgne, Marie Babel, Tom

Carlson, Philippe Gallien

► To cite this version:

Louise Devigne, François Pasteau, Nicolas Le Borgne, Marie Babel, Tom Carlson, et al.. Assisting power wheelchair driving on a sidewalk: A proof of concept. Modelling, measurement and control C, 2018, 79 (4), pp.185-189. 10.18280/mmc_c.790406. hal-02124782

HAL Id: hal-02124782

https://inria.hal.science/hal-02124782

Submitted on 9 May 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
# Assisting power wheelchair driving on a sidewalk: A proof of concept

L. Devigne<fnref n="1" />,<fnref n="2" />*, F. Pasteau<fnref n="2" />, N. Le Borgne<fnref n="2" />, M. Babel<fnref n="2" />, T. Carlson<fnref n="3" />, P. Gallien<fnref n="1" />, Centre MPR Saint Hélier Pole<fnref n="1" />

Corresponding Author Email: marie.babel@irisa.fr

https://doi.org/10.18280/mmc_c.790406

## ABSTRACT

**Received:** 18 September 2018

**Accepted:** 31 October 2018

**Keywords:**

power wheelchair, driving assistance, sensor-based control, powered mobility, negative obstacles

The use of a power wheelchair allows to maintain mobility by providing better access to daily activities and thus positive impact on the quality of life. However, driving a power wheelchair is a complex task, particularly within an environment consisting of negative obstacles (e.g. steps, sidewalk edges). In this context, falling accidents can occur while driving a power wheelchair on a sidewalk. Therefore, driving assistance is required to prevent from falling off a curb edge. In order to meet these expectations, we here propose a semi-autonomous shared control framework assisting the user while driving on a sidewalk. We present simulations as well as an experiment carried out with our system embedded on a standard wheelchair. In both cases, our method allows progressive velocity adaptation when approaching a curb edge resulting in the wheelchair avoiding the risk of falling. The obtained results thus provide a proof of concept of our method.

## 1. INTRODUCTION

Some people with motor disabilities may benefit from the use of a power wheelchair to increase their mobility and independence. However, the use of motorized mobility devices can lead to risks for others, oneself or the property [1].

In this context, many research teams are involved in the development of solutions to assist in the driving of electric wheelchairs. Indeed, driving assistance can be provided in the form of autonomous [2-3] or semi-autonomous [4-5] assistance but are so far restricted to indoor use because they do not provide assistance with respect to negative obstacles.

However, negative obstacles constitute major physical barriers for safe power wheelchair driving. Indeed, a study in the form of regular users interviews, carers emphasized the difficulties encountered with curbsides [6]. In this context, driving assistance in outdoor environments would be expected to provide more safety while driving in an environment consisting of negative obstacles. Indeed, recent studies highlight the benefits of using motorized mobility devices in outdoor activities on user's self-esteem and sense of independence by providing easier access to outdoor activities thus maintaining daily-life activities and social interactions [7-8].

In the robotics field, few methods have been proposed for assisting robots navigation in an environment consisting of negative obstacles [9-10] typically based on detection relying on stereo vision or expensive sensors such as laser rangefinders. Moreover, these solutions generally represent the presence of obstacles in an occupancy grid and are based on the definition of local environmental maps requiring odometry as well as high computational load.

In the field of power wheelchair navigation, only a few studies have focused on the detection of negative obstacles. Although they use low-cost sensors and low-computational cost

methods, the solutions presented in [11] and [12] stop the power wheelchair when a drop-off (e.g. curbside) is detected. A detection of the edges of obstacles via stereo vision measurement is proposed in [13] but this method has important limitations such as the absence of detection of certain sidewalk edges as well as false detections. In addition, the few proposed method only provide detection with no semi-autonomous or autonomous control law to assist the user while navigating with the wheelchair by another means than stopping when a drop-off is detected.

In this article, we present a low-cost semi-autonomous wheelchair control solution for assisting wheelchair navigation on a sidewalk. The proposed method is based on the same mathematical principle as the approach presented in [14] which application has been until now restricted to indoor navigation assistance as the detection of negative obstacles was not ensured yet. This method has been tested and validated by wheelchair regular users within clinical trials which have been conducted within the Pôle Saint Hélier rehabilitation center in Rennes [14, 16]. In this paper, we propose to apply this principle to navigation on a sidewalk by considering detected drop-offs as virtual positive obstacles in the horizontal plane thus leading the wheelchair to progressively correct its trajectory to avoid the curbside the same way it avoids colliding with a wall. The control law is based on a reactive sensor-based servoing method which does not rely on the definition of a local map of the environment.

The paper is organized as follows. Section II presents the method for detecting negative obstacles. Section III presents the wheelchair velocity control method. The simulations of the proposed method are presented in Section IV. Finally, an experiment with a standard power wheelchair conducted in the UCL PAMELA facility is presented and commented on in Section V.

<footnote n="1">¹ 54 rue Saint Hélier Rennes 35000, France</footnote>

<footnote n="2">² Univ. Rennes, INSA, CNRS, Inria, Irisa-UMR6074, Rennes F-35000, France</footnote>

<footnote n="3">³ Aspire Centre of Rehabilitation and Assistive Technology, University College London, UK</footnote>
## 2. CURB DETECTION METHOD

In order to detect negative obstacles around the power wheelchair, we use distance sensors mounted on the wheelchair frame oriented towards the floor. Here, as the idea is to provide a proof of concept of the proposed method, the

distance to the floor measured by each sensor is used to calculate the distance to the curbside in the horizontal plane. With such a detection, the wheelchair will avoid approaching a curbside the same way it already avoids to collide with a wall, thus bringing us back to the same configuration as in [14].

<figure><img src="image_1.png" /><figcaption>Figure 1. Negative obstacle detection</figcaption></figure>

As shown in Fig. 1,

* let $d_i$ the distance measured by the sensor $s_i$,

* let $h_{s_i}$ be the sensor $s_i$ height from ground,

* let $\varphi_i$ be the orientation of the sensor with respect to the vertical,

* let $h_{i_n}$ be the height of the negative obstacle,

* let $h_{i_p}$ be the height of the positive obstacle,

* let $H_{i_n} = h_{s_i} + h_{i_n}$ the sensor $s_i$ height from negative obstacle level,

* $H_{i_p} = h_{s_i} - h_{i_p}$ the sensor $s_i$ height from positive obstacle level.

The detection is such that we can distinguish 3 cases:

1) if we define hip as the minimum height of a positive obstacle we observe, then we detect a positive obstacle when $d_i < d_p = \frac{h_z \oplus h_{i_p}}{\cos(\varphi_i)} = \frac{H_p}{\cos(\varphi_i)}$, then, the distance to the obstacle is defined as $x_i = d_i \sin(\varphi_i)$.

2) if we define $h_{i_n}$ as the minimum height of a negative obstacle we observe, then we detect a negative obstacle when $d_i > d_{i_n} = \frac{h_s+h_{i_n}}{\cos(\varphi_i)} = \frac{H_{i_n}}{\cos(\varphi_i)}$, then, the distance to the obstacle is defined as $x_i = h_{s_i} \cos(\varphi_i)$.

3) as long as the distance $d_i$ measured by the sensor $s_i$ is such that $d_{i_p}, d_i, d_{i_n}$, we can consider that there is no obstacle to detect. Then, the distance to the obstacle is defined as $x_i = x_m$ where $x_m$ is the maximum distance which a sensor can measure.

This detection method necessitates an extrinsic calibration of the sensors. Indeed, we use the sensor position information in our calculations.

## 3. SENSOR-BASED SERVING FRAMEWORK

In a previous paper, we presented a semi-autonomous approach providing adaptive assistance in the form of an intuitive obstacle avoidance [14]. So far, we applied this method to indoor navigation only as we did not detect negative obstacles. Here, although the method relies on the same principles, we detail the calculation of the allowed and forbidden areas defined from sensor constraints in the

wheelchair velocity domain. In particular, we emphasize the calculations for the case of corner sensors for which the geometric configuration allows us to estimate the orientation of the detected obstacle. This additional information allows the shared control scheme to achieve more comfortable trajectory corrections while approaching an obstacle.

The Modeling is given in previous work [14]. We can define an interaction matrix $L_{x_i}$ for each sensor $s_i$ such that

$$ \dot{x}_i = L_{x_i} v_{s_i}. \quad (1) $$

The interaction matrix $L_{x_i}$ which is defined in [15], is such that

$$ L_{x_i} = [-1 \tan(\alpha_i) \quad 0 \quad 0 \quad 0 \quad x_i \tan(\alpha_i)] \quad (2) $$

where $\alpha_i$ is the angle formed by the sensor $s_i$ x-axis and the perpendicular to the detected obstacle (Fig. 2).

From the distance measured by the sensor $s_i$, we are not able to determine the orientation of the obstacle. Therefore, we assume that the obstacle detected by the sensor $s_i$ is perpendicular to the sensor x-axis. Then, we approximate $\alpha_i = 0$ for the computation of the interaction matrix $L_{x_i}$. This leads to

$$ L_{x_i} = [-1 \quad 0 \quad 0 \quad 0 \quad 0 \quad 0] \quad (3) $$

To avoid collisions with obstacles, we constrain the distance $x_i$ measured by the sensor $s_i$ by a minimum value $\lambda e_{s_i}$ with $\lambda > 0$. We then get

$$ L_{x_i} s_i T_r u - \lambda e_{s_i}. \quad (4) $$

Based on the modelling given in [14], we find

$$ A_i u B_i \quad (5) $$

with $A_i L_{x_i} s_i T_r = [-a_i \quad -b_i]$ and $B_i = -\lambda e_{s_i}$. This defines a half-plane in the wheelchair velocity domain as shown on Fig. 2.
<figure><img src="image_2.png" /><figcaption>**Figure 2.** Half plane in the wheelchair velocity domain</figcaption></figure>

# 4. SIMULATIONS

In this section, we present the results of a simulation we performed as a first step to develop our curb-following solution.

## 4.1. Simulation conditions

As a part of our research activities, we developed a simulator for assisted power wheelchair driving. This simulator has been designed with Unity3D and consists of a Virtual Environment in which we operate a virtual robotized wheelchair equipped with virtual distance sensors. The simulator is presented in more details in [17].

The virtual wheelchair (Fig. 3b) is here equipped with a total of 12 distance sensors placed at each corner of the virtual wheelchair (Fig. 3a). The virtual environment is displayed on a computer screen as it is only used to demonstrate the properties of the curb-following solution.

<figure><img src="image_3.png" /><figcaption>(a) Simulated trajectory for the wheelchair</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>**Figure 3.** Wheelchair setup for the simulation</figcaption></figure>

As shown on Fig. 4, it consists of a L-shaped 15 cm height, 1.2 m wide and 5 m long sidewalk with a right turn at its end.

<figure><img src="image_5.png" /><figcaption>**Figure 4.** Virtual environment for the simulation.</figcaption></figure>

## 4.2. Simulation results

The virtual wheelchair starts from the position shown on Fig. 4 i.e. at the beginning of the sidewalk and parallel to the curb. The user's linear and angular inputs are constant and equal to the maximum speed of the wheelchair. This leads the wheelchair to go forward and right at maximum speed.

<figure><img src="image_6.png" /><figcaption>**Figure 5.** Simulation results</figcaption></figure>

As shown on Fig. 5, the wheelchair starts from the beginning, no negative obstacle is detected thus the wheelchair is going towards the curb. Then, when the curb is detected, the linear velocity progressively decreases and the rotational velocity is modified in order to follow the curb. When the wheelchair drives parallel to the curb, the linear velocity increases as there is no danger to fall from the curb. The rotational velocity is maintained to zero as long as the wheelchair is detecting the curb on its right. Then, when the wheelchair arrives at the point where there is 90° turn to the right, it turns right and stops when it detects the drop-off at the

end of the sidewalk in front of it.

# 5. EXPERIMENT

## 5.1 Experimental setup

We developed and tested our solution on a commercially available Typhoon II power wheelchair from Invacare. We had previously modified Invacare's DX controller to intercept the user input and adjust the control signals before sending them to the motors, allowing us to implement and test various shared
control algorithms on the platform [5]. The algorithm itself
runs on an embedded Beagle-Bone Black.

For the detection, we used Time-of-Flight (ToF) VL53L0X
laser-ranging sensors from STMicroelectronics. Unlike
conventional infrared sensors, these sensors are low-cost and
provide accurate distance measurement whatever the target
reflectance and can measure absolute distances up to 2 meters
with a field of view of 25 degrees.

Our detection method necessitates an extrinsic calibration
of the sensors as we use the sensor position information in our
distance calculations. Here, we rigidly attached the detection
modules to the wheelchair frame thus ensuring a fixed position
of the sensors.

## 5.2 Experimental results

An example of results obtained from a trial on the sidewalk
within the PAMELA facility is shown on Fig. 6. For each trial,
we used the same initial conditions as during the simulations:
the wheelchair starts parallel to the curb at the beginning of the
sidewalk.

<figure><img src="image_7.png" /><figcaption>**Figure 6.** Experiment results: linear and angular user input(blue) and shared control output (red) velocities</figcaption></figure>

## 6. DISCUSSION

As shown on Fig. 6, the user first applies a rotational
velocity to the wheelchair towards the curb. As long as no
drop-off is detected by our system, the user rotational velocity
input is directly applied to the wheelchair. Then, when a drop-
off is detected, the rotational velocity is set to zero in order to
avoid from falling down the curb. After that, the user
maintains his input for a moment and then starts to move
forward while keeping a constant rotational velocity input. The
shared control rotational velocity output is then maintained to
zero while the user linear velocity input is directly applied to
the wheelchair as there is no risk of falling from the curb while
driving forward. We can notice that at the end of the
experiment, the linear velocity is abruptly set to zero while the
rotational velocity is being directly applied to the wheelchair.
This happened several time as the floor was irregular with gaps
between the modules thus leading to false detections making
the wheelchair to stop.

The experiment we performed within the PAMELA facility
allowed us to assess the performances of our curb-following
solution while embedded on a real wheelchair. Although we

encountered false detections due to the floor irregularities, we
were able to validate our detection method and to avoid from
falling from the curb while driving. These results bring a proof
of concept of our curb-following solution for assisted power
wheelchair driving.

## 7. CONCLUSION

In this article, we have presented a generic shared control
wheelchair navigation assistance solution to assist power
wheelchair driving on a sidewalk. While the control law is
based on the same principle as the power wheelchair collision
avoidance assistance solution presented in [14], here the
proposed innovative approach is based on a different detection
method. Indeed, rather than detecting distances to obstacles,
the proposed system measures distance with sensors oriented
towards the floor. From these measurements, the distance from
the wheelchair to the curb side is determined. These
constraints are merged with user's input to provide a smooth
and progressive shared control by means of a velocity
adaptation applied to the wheelchair when approaching a curb
edge. The behaviour of the system in simulation as well as
embbeded on a standard power wheelchair gives a proof of
concept of our method. Future works will address navigation
assistance on slopes and ramps by calculating a dedicated
interaction matrix for each sensor which would incorporate the
local floor orientation. By doing so, we hope to expand the
assistance possibilities of our method to complete outdoor
navigation assistance. The method presented in this article is
independent of the type os sensor used and has a low
computationnaly cost, thus meeting smart power wheelchair
solution design requirements.

## REFERENCES

[1] Chen WY, Jang Y, Wang JD, Huang WN, Chang CC, Mao HF, Wang YH. (2011). Wheelchair-related accidents: relationship with wheelchair-using behavior in active community wheelchair users. Archives of Physical Medicine and Rehabilitation 92(6): 892-898.
https://doi.org/10.1016/j.apmr.2011.01.008

[2] Burhanpurkar M, Labb M, Guan C, Michaud F, Kelly J. (2017). Cheap or robust? The practical realization of self-driving wheelchair technology. ICORR, pp. 1079-1086.
https://doi.org/10.1109/ICORR.2017.8009393

[3] Baklouti E, Amor NB, Jallouli M. (2016). Autonomous wheelchair navigation with real time obstacle detection using 3D sensor. Automatika 57(3): 761-773.
https://doi.org/10.7305/automatika.2017.02.1421

[4] Horn O. (2012). Smart wheelchairs: Past and current trends. 2012 1st International Conference on Systems and Computer Science (ICSCS), pp. 1-6.
https://doi.org/10.1109/ICOnSCS.2012.6502470

[5] Ehz C, Trautman P, Devigne L, Bureau V, Babel M, Carlson T. (2017). Probabilistic vs linear blending approaches to shared control for wheelchair driving. IEEE Int. Conf. on Rehabilitation Robotics, ICORR'17.
https://doi.org/10.1109/ICORR.2017.8009352

[6] Wang RH, Korotchenko A, Hurd Clarke L, Mortenson WB, Mihailidis A. (2013). Power mobility with collision avoidance for older adults: User, caregiver, and prescriber perspectives. Journal of rehabilitation research
and development 50(9): 1287.
https://doi.org/10.1682/JRRD.2012.10.0181

[7] Löfqvist C, Pettersson C, Iwarsson S, Brandt A. (2012). Mobility and mobility-related participation outcomes of powered wheelchair and scooter interventions after 4-months and 1-year use. Disability and Rehabilitation: Assistive Technology 7(3): 211–218.
https://doi.org/10.3109/17483107.2011.619224

[8] Samuelsson K, Wressle E. (2014). Powered wheelchairs and scooters for outdoor mobility: A pilot study on costs and benefits. Disability and Rehabilitation: AT 9(4): 330–334.
https://doi.org/10.3109/17483107.2013.827244

[9] Larson J, Trivedi M. (2011). Lidar based off-road negative obstacle detection and analysis. Intelligent Transportation Systems (ITSC), pp. 192-197.
https://doi.org/10.1109/ITSC.2011.6083105

[10] Murarka A, Sridharan M, Kuipers B. (2008). Detecting obstacles and drop-offs using stereo and motion cues for safe local motion. IEEE/RSJ IROS, pp. 702-708.
https://doi.org/10.1109/IROS.2008.4651106

[11] Simpson R, LoPresti E, Hayashi S, Nourbakhsh I, Miller D. (2004). The smart wheelchair component system. JRRD 41(3B): 429.
https://doi.org/10.1682/JRRD.2003.03.0032

[12] Ka HW, Simpson R, Chung Y. (2012). Intelligent single switch wheelchair navigation. Disability and Rehabilitation: Assistive Technology 7(6): 501–506.

[13] Coughlan JM, Shen H. (2007). Terrain analysis for blind wheelchair users: Computer vision algorithms for finding curbs and other negative obstacles. CVHI.

[14] Devigne L, Narayanan VK, Pasteau F, Babel M. (2016). Low complex sensor-based shared control for power wheelchair navigation. Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on. IEEE, pp. 5434–5439.
https://doi.org/10.1109/IROS.2016.7759799

[15] Espiau B, Chaumette F, Rives P. (1992). A new approach to visual servoing in robotics. iee e Transactions on Robotics and Automation 8(3): 313-326.
https://doi.org/10.1109/70.143350

[16] Babel M, Pasteau F, Guégan S, Gallien P, Nicolas B, Fraudet B, Achille-Fauveau S, Guillard D. (2015). HandiViz project: clinical validation of a driving assistance for electrical wheelchair. IEEE ARSO, Lyon, France, Jul.

[17] Devigne L, Babel M, Nouviale F, Narayanan V, Pasteau F, Gallien P. (2017). Design of an immersive simulator for assisted power wheelchair driving. IEEE ICORR'17.
https://doi.org/10.1109/ICORR.2017.8009379