# In-Line: A Navigation Game for Visually Impaired People

Laura Giarré, Ilenia Tinnirello, Letizia Jaccheri

► To cite this version:

Laura Giarré, Ilenia Tinnirello, Letizia Jaccheri. In-Line: A Navigation Game for Visually Impaired People. 16th International Conference on Entertainment Computing (ICEC), Sep 2017, Tsukuba City, Japan. pp.147-153, 10.1007/978-3-319-66715-7_15 . hal-01771234

HAL Id: hal-01771234

https://inria.hal.science/hal-01771234

Submitted on 19 Apr 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# In-Line: A Navigation Game For Visually Impaired People

Laura GIARRÉ, Ilenia TINNIRELLO, Letizia JACCHERI *

**Abstract.** In-line is a novel game based on a navigation system, called ARIANNA (pAth Recognition for Indoor Assisted Navigation with Augmented perception, [1]), primarily designed for visually impaired people permitting to navigate and find some points of interests in an indoor and outdoor environment by following a path painted or stuck on the floor. The aim of the game is twofold: 1) let the users learn and familiarize with the system, 2) improve blind people spatial skills to let them learn and acquire an allocentric spatial representation. The impact stands in the possibility of enhancing the social inclusion of a large part of the society that is increasing with the aging of the population by augmenting their autonomous mobility. The paper presents the concept design and preliminary evaluation of a game specifically designed for blind people. The evaluation has been performed via qualitative and quantitative tests.

**Keywords:** Navigation, Assistive Technology, Game, Visually Impaired People

## 1 Introduction And Related Work

Worldwide, 161 million people are visually impaired (according to the 2002 World Health Organization statistics), with an average of 3.4 low-vision patients for each blind person. Visual impairments pose a number of challenges related to mobility and hand-eye coordination. With technology we try to face the challenge of helping Visually Impaired Person to live a more independent life, working and socializing via autonomously and safely moving in an unknown environment and retrieving information about the places nearby. An important aspect of the design of assistive technology for Visually Impaired People, underlined in [2], is the possibility for the blind community to be taught and accompanied by the use of the novel device. A learning phase trough games is envisioned as a possible solution to let the blind community to accept such aids, by learning the

<footnote n="1">¹ DIEF, Università di Modena e Reggio Emilia, Modena, Italy,
laura.giarre@unimore.it</footnote>

<footnote n="2">² DEIM, Università di Palermo, Palermo, Italy, ilenia.tinnirello@unipa.it</footnote>

<footnote n="3">³ Department of Computer Science, NTNU, Trondheim, Norway, letizia@idi.ntnu.no</footnote>

<footnote>* The authors acknowledge In.Sight s.r.l. (http://in.sight.srl) for developing the AR-IANNA (patented) and In-Line; they also thank Prof. Lofti Merabet, Harvard Medical school, for very fruitful discussions.</footnote>
use of them. This reasoning is the first motivation for the developing of In-line
to let the blind people being familiar with the ARIANNA system and its use.
The system permits to autonomously navigate and find some points of interests
in an indoor and outdoor environment. A path stuck or painted on the floor is
detected by the smartphone camera and a vibration signal providing a feedback
to the user for correcting his/her direction is generated. Some special landmarks
have been deployed for coding additional information, using Beacon technology
or Qr codes. With ARIANNA, then, by following pre-determined paths located
in the indoor and outdoor environment the visually impaired people can move
independently in unfamiliar environments.

The accessibility of Internet to blind persons, and in particular the design
of accessible game has been analyzed in the survey [3] where a large number
of accessible games are listed for different types of impairments, across several
game genres, from which a number of high- and low-level accessibility strate-
gies are distilled for game developers to inform their design. In that analysis it
has been stressed how the accessibility problems can be related to a) not being
able to receive feedback; b) not being able to determine in-game responses; c)
not being able to provide input using conventional input devices. The design of
our game is taking into account some of the suggestions presented here. Many
blind persons, especially if young or adolescent, have a very high technological
ability and many are the commercial games designed specifically for blinds. The
pervasive presence of low-cost smart devices with increased computation ability
allows the growth of innovative solution in several application fields. As [10] re-
ports that for profoundly blind individuals navigating in an unfamiliar building
represents a significant challenge, and investigated the use of an audio-based,
virtual environment called Audio-based Environment Simulator that can be ex-
plored for the purposes of learning the layout of an unfamiliar, complex indoor
environment, we were pushed towards the developing of our game. The highly
interactive and immerse exploration of the virtual environment greatly engages
a blind user to develop skills akin to positive near transfer of learning. Learning
through a game play strategy appears to confer certain behavioral advantages
with respect to how spatial information is acquired and ultimately manipulated
for navigation.

## 2 Concept Design Of In line

In line is a game primarily designed for visually impaired people which is based
on the navigation system ARIANNA.

2.1 ARIANNA System Description

ARIANNA system permits to navigate and find some points of interests in an
indoor environment by following a path on the floor. Descriptions of the system
were presented in [4], [5], and the Patent [1]. The system works as follows: the
camera of the smartphone continuously captures the scene in front of the person
<figure><img src="image_2.png" /><figcaption>**Fig. 1.** The line captured by the phone</figcaption></figure>

(see Fig. 1) and identifies the path by recognizing special colored strips painted or stuck on the floor. The users, then, receives a vibration signal through his cell phone. The vibration is related to the position and the direction of the user. In any installations of the system, the paths are marked with colored lines on the floor; QRcodes or beacon are also settled close to points of interest and they provide information on the right line to follow in order to get to the desired destination. The user interface employs tactile stimuli to receive feedback on the heading corrections to be employed, as better described in the following.

## 2.2 Aim Of The Game

The aim of the game is twofold: 1) let the users learn and familiarize with ARIANNA 2) to improve blind people spatial skills. The latter is obtained with the design of some 'find short-cuts' task during the play, (navigate trough locations not linked by pathways directly experienced during the game) in order to let them learn and acquire an allocentric spatial representation instead of the egocentric one. We recall that blind people have an egocentric spatial representation (based on themselves) while sighted people have also an allocentric representation, linked to a reference based on the external environment and independent of their current location in it. At first, we have started building up our game to fulfill the following scope: let the blind community play&learn. This will achieve also the secondary goal that is letting ARIANNA become a tool that the blind associations are happy with, by enforcing its use by the game helping blinds to familiarize with it. But learning trough games and amusement, is really the main scope of In line because, as reported in a recent panel, [6], the use of games as an educational tool provides opportunities for deeper learning.

During the process of designing the game, we then asked ourselves if the game could be also a tool that could offer more to the blind community than a simple game to learn and be familiar with an app. Then, along what has been studied by [7], we have decided to use In line to improve blind people spatial skills. To this aim, we design some 'find short-cuts' tasks during the play, (navigate trough locations not linked by pathways directly experienced during the game). While understanding the presence of short cuts may let them learn, acquire and improve an allocentric spatial representation. Egocentric type of representation is a route-
based representation, while allocentric one is a survey-based one. For example, the reproduction of previously explored pathway only requires memory trace of the route, while finding shortcuts requires to build a survey representation and to compute a completely new pathway in the brain. Many have been the studies regarding whether the congenital blind individual prefers egocentric over allocentric spatial representation, [7] and references therein, but many studies are also suggesting the use of educational programs helping and training blind person in developing such an allocentric skill. Similar experiments have been already carried out, see [8], [9] and [10].

## 2.3 The Game In line

The first prototype of InLine was a game where a virtual line was projected on the floor. The person stands on the path and follows the line that is moving in front. During the projection of the line, some QR codes were also depicted and the blind (or blind folded) users had to catch it. Clearly, this previous release could be used only in a specific settings, with external teachers guiding the user trough the game. The line was projected on the floor, and the area was restricted so that no one could enter it and be an obstacle for the gamers. The aim of this first release was to teach the blind person how to use the system. We wanted to achieve the following: to be able to follow tactile indication from a common device such as a smartphone. The main objective of the game is to collect max possible scores in a precise time interval, start and the end of the game are reported by different audio signals. During the play, the ARIANNA path or line is projected in front of the player. During the game, the user is not walking on the path but the path flow moves in front of it. The game consists of following the line movements with the smartphone, the player tries to follow the movements of the line in front of him/her. If the line is captured the phone vibrates, the skill of the player is to predict the line changes and keeping on the smartphone vibration. The game has many levels of difficulty related to different paths, every level has a speed line runs and many changes direction different from other levels. The maximum score is reached when the player never looses the line, to all the game time, and consequently, the smartphone never stops the vibration. If the player obtains at least 3/4 of the maximum score, then it can move to the next level. Moreover, the player can intercept markers, such as QR code on the path. If this occurs further information about the point of interests are reported, with more scores and bonus to quickly go at next level.

The second release of the game was designed to be entirely developed on the smartphone. The Game is designed according to the second modalities of ARIANNA, that is the user is following the virtual line with the finger and the phone vibrates only if the finger captures the line. There is a virtual path on the smartphone screen, the finger is used to intercept the line; an on/off vibration feedback will tell the user if he/she gets the line or not. This second release could be used by the blind people in complete autonomy. The main objective of the game is to collect max possible scores during a session, no time limits are considered in a session. Every session is composed of three phases, during the
<figure><img src="image_3.png" /><figcaption>**Fig. 2.** In line virtual path in the second release of the game</figcaption></figure>

first phase a line shape is shown on smartphone touch screen, the smartphone vibrates when the player finger intersects the line. The first phase consists of following the line shape with the player finger, from start point to end point and store the shape. In the second phase, the player must remember the line and uses the finger to reproduce the line on smartphone touch screen. If the reproduced line is completely matched with the original line shape, the player reaches the maximum score and it moves to the third stage. Otherwise, the player can retry unlimited time, any attempt produces a penalty and the total score is reduced. In the third stage, the player must remember the start and stop line points, the player skill is to connect them together through a shortcuts path. Lower path enables additional scores for the player. The game has different levels with different difficulty related to paths shape. An example of the virtual path to follow is depicted in Fig. 2.a). To let the player acquire more points the virtual paths is intended to be followed in a more complicated way, in order to design also the shortcuts necessary for the developing of Allocentric vs. egocentric spatial perception (see Fig. 2.b)).
To let the player acquire more points the virtual paths is intended to be fol-
lowed in a more complicated way, in order to design also the shortcuts necessary
for the developing of Allocentric vs. egocentric spatial perception (see Fig. 2.b))

3 Evaluation And Discussion

The game, in its first release, was tested during the smart city exhibition in Bologna. ARIANNA system has been deployed in various test environments, and also in real installations, such as in the Farm Cultural Park in Favara (AG) ([12]) and in the Blind Institute of Palermo Florio Salamone ([13]), (see [11]). The latter has been designed to allow blind people to move completely independently from one area of the institute complex to another. Specifically, the route crosses both courtyards, from the entrance to the computer classroom, and it enables to reach all the building entrances that are located within the perimeter of the three courtyards. On the way, 10 points of interests are placed allowing the users to orient themselves and achieve full autonomy in any part of the building. Qualitative and quantitative tests with blind and low vision people have been collected and analyzed. The complete analysis has been described in details in [15]. To design our tests, we decided to use Google Drive for our online survey, because it is compatible with many accessibility features for visually impaired people and is supported by a variety of platforms and smartphones. To get information on the user's experience, we created a survey where the scoring of the results provides an overview of the consumer satisfaction. The data can be analyzed and processed to verify the strengths and weaknesses of the system To determine the consumer opinions we use the Likert scale [14], a psychometric scale commonly involved in research that employs questionnaires. It consists of a series of statements linked to the attitudes that are investigated. A similar approach will be used to design the tests for the In-Line game.

4 Conclusions

Overall, the survey shows that the system was received with good enthusiasm
and the users enjoyed the tests. They consider it a breakthrough and would
definitely use it. An interesting aspect that emerged is that all users wish to
obtain as much information as possible about the environment around them.
A key aspect of the formation of mental maps in blind subjects is linked to
the memory of the paths and the location of objects that they develop with
repetition and practice. If a person follows in his mind a small-scale model of
the external reality, he is able to think of various alternatives, choose the best
among them, react to future situations before they happen The obtained results
stimulate us to further develop the game and in particular to let the blind people
at the Blind Institute use the game together with ARIANNA and then running
more tests and collect qualitative and subjective feedback trough questionnaires.
References

1. Italian Patent N. BG2014A000054, Sistema di navigazione per non vedenti, presente 2015, patented 2017.

2. G. Elli, S. Benetti and O. Collignon. Is There a Future for Sensory substitution Outside Academic Laboratories. Multisensory Research, 27 (2014) 271-291

3. B. Yuan, E. Folmer, F. C. Harris Jr. Game accessibility: a survey, in Universal Access in the Information Society. Vol. 10, Issue 1, pp 81-100, 2011.

4. D. Croce, P. Gallo, P. D. Garlisi, L. Giarré, S. Mangione, I. Tinnirello. ARIANNA: A smartphone-based navigation system with human in the loop. 22nd Mediterranean Conference of Control and Automation (MED), Page(s): 8 - 13, 2014.

5. D. Croce, L. Giarré, F. G. L. Rosa, E. Montana and I. Tinnirello. Enhancing tracking performance in a smartphone-based navigation system for visually impaired people. 2016 24th Mediterranean Conference on Control and Automation (MED), Athens, 2016, pp. 1355-1360. doi: 10.1109/MED.2016.7535871

6. R. F. Mackay. Playing to learn Stanford Report. March 1, 2013 http://news.stanford.edu/news/2013/march/games-education-tool-030113.html

7. Z. Cattaneo and T. Vecchi. Blind Vision. The Neuroscience of Visual Impairment. MIT press, 2011.

8. H. Connors EC, J. Snchez, L. Merabet. Real world navigation independence in the early blind correlates with differential brain activity associated with virtual navigation. Hum Brain Mapp. 2013 PMID: 24027192, 2013.

9. E. Connors, L. Yazzolino, J. Snchez, L. Merabet. Development of an audio-based virtual gaming environment to assist with navigation skills in the blind. J Vis Exp. 2013 PMID: 23568182, 2013.

10. E. Connors, E. Chrastil, J. Snchez and L. Merabet. Virtual environments for the transfer of navigation skills in the blind: a comparison of directed instruction vs. video game based learning approaches. Front. Hum. Neurosci. 8:223. doi: 10.3389/fnhum.2014.00223, 2014.

11. http://in.sight.srl/installations/

12. http://www.farmculturalpark.com/

13. http://www.istciechipalermo.it/

14. Likert scale: https://en.wikipedia.org/wiki/Likert_scale

15. *Deliverable Technical Report of the EU project FIC-3, SUB-GRANT AGREE-MENT N CALL 2-28.*