# Perception of Tactile Symbols by Visually Impaired Older Adults

Vojtech Gintner, Miroslav Macik, Zdenek Mikovec

► To cite this version:

Vojtech Gintner, Miroslav Macik, Zdenek Mikovec. Perception of Tactile Symbols by Visually Impaired Older Adults. 17th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2019, Paphos, Cyprus. pp.325-334, 10.1007/978-3-030-29381-9_20. http-02544568

HAL Id: hal-02544568

https://inria.hal.science/hal-02544568

Submitted on 16 Apr 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Perception of tactile symbols by visually impaired older adults

Vojtech Gintner, Miroslav Macik, and Zdenek Mikovec

Faculty of Electrical Engineering,
Czech Technical University in Prague,
Prague, Czech Republic
macikmir@fel.cvut.cz

**Abstract.** We present a design of an indoor orientation terminal for visually impaired older adults. Interaction is based on buttons, tactile symbols, and audio feedback. The terminal consists of five parts dedicated to a particular function. The tactile symbols mimic real-world objects. We performed three design iterations and conduct evaluations with a total of 17 participants, their mean age was 84.2 years. The results show that usage of real-world objects and low level of symbol abstraction leads to an unambiguous pairing of user expectations and real functions. Introduction of complicated and abstract artifacts like contour objects or complex tactile map was very hard to understand and recognize. Our final design was well accepted by all participants and allowed the participants to orient themselves in the indoor environment.

**Keywords:** tactile perception; visually impaired; older adults;

## 1 Introduction

Visually impaired people challenged with severe impairment or even blindness (according to WHO classification [12]) appear mostly among older adults. According to Bourne et al. [3], 86.3 % of blind people are older than 50 years and 52.8 % older than 70 years.

Unfortunately, the research attention on visually impaired older adults is limited. We have analyzed 39 papers focusing on visually impaired people presented on last three CHI conferences (2016-2018). When excluding studies focused primarily on children and young adults, the mean age of study participant was 37.3 years (weighted mean, sample size as weight). This clearly reveals a bias in favor of the younger part of the population that forms a minority of the visually impaired population.

As visual impairment limits mainly a person's mobility and reduces travel-related activities [6], solutions helping visually impaired people with spatial orientation and navigation are of high importance. A tactile user interface is one of the efficient ways of interaction for visually impaired people as their passive tactile acuity is superior [5].
In our research, we focus on the design of tactile symbols used for buttons of an interactive indoor orientation system helping visually impaired older adults with orientation and navigation inside a large complex building. The previous research [10] shows, that visually impaired older adults experience serious problems with understanding the meaning of abstract tactile symbols. Our research shows a way how to design tactile symbols whose meaning is understandable by visually impaired older adults.

## 2 Related work

There exist extensive research in the field of tactile symbols used especially in the framework of tactile maps like [1, 15]. Besides 2D symbols researches experiment also with 3D (volumetric) tactile symbols [8, 7]. Moreover, the tactile perception of visually impaired people is superior [5].

There are also other approaches to solve the orientation and navigation by means of a combination of active and passive elements. PERCEPT [4] uses RFID tags with a special glove to read them. Another similar solution [16] is based on RFID tag grid on the floor and speaker placed on the white cane. Other solutions typically use a smartphone or another dedicated electronic device which should be carried by the user in combination with Bluetooth beacons like [9].

Unfortunately, all currently widely used orientation and navigation systems require the user to carry a special artifact or electronic device. We want to avoid that as our user group can often include people with mobility issues [13] or mental illness (approximately 20 % of adults aged 60 and over suffer from a mental disorder) [11].

Analysis of related work shows that there is a lot of research dedicated to tactile symbols and alternative solutions. However, there is a lack of research focusing on tactile symbols for visually impaired older adults being a majority in the user group of visually impaired.

## 3 Design of tactile symbols

As the tactile symbols were designed in the context of an orientation system, we had to define the user requirements related to orientation and navigation at first. We cooperated with special housing for visually impaired older adults and gathered all requirements and recruited experiment participants there. Following the methodology of participatory design [14], we conducted a workshop with four employees of the special housing (two nurses and two social workers), 21 semi-structured interviews with residents, and numerous observations. From the requirements we can state four primary purposes of the system: spatial orientation, navigation to a known place, general information (about time, date, schedule and lunch menu) and emergency call for help. The proposed orientation system consists of three types of components (orientation terminals, guiding lines, and mini-info buttons). In this paper, we focus on the design of the tactile symbols of the orientation terminal.
The orientation terminal is a small box placed on the wall on the most strategic places in the building, like entry halls or next to elevators; it outputs all the information as natural language voice instructions. It serves as the central hub of interaction for the other parts of the system that relate to the information provided by an orientation terminal. It serves the following functions:

- Orientation - where I am, what's around me, where this direction leads.

- Navigation - how do I get back to safety (to my room).

- Information - what time is it, what's the date today, what's for lunch or dinner, what's the weather, what's todays schedule.

- Emergency - call for help.

Specifics of the target user audience create significant challenges for designing interactive devices. We decided to use buttons that will invoke the voice recordings (or TTS). The choice of buttons was made with three factors in mind. It is one of the simplest types of interaction, and it can be easily used by visually impaired as well as sighted individuals and everyone ever used a button at some point in their life (elevator, radio or television, etc.).

The previous research [10] showed that simplified or abstract tactile pictograms did not work for visually impaired older adults. The symbol representing a clock (circle with little and big hand), for instance, was recognized by one participant only. On the other hand, simple abstract symbols, like a triangle, were recognized by almost everybody, but without any notion what function it can represent. We started looking for a different tactile representation of corresponding terminal functions, that will be both easy to recognize and assign a corresponding function of an orientation terminal.

## 3.1 Design A - Mimic the real world

We organized a design studio [2] with four employees of the special housing (two nurses and two social workers) and came out with an idea to focus on the objects from daily-life of clients they are used to from past period. We also decided to mimic the real world as much as possible during the design of the symbols. We created a lo-fi prototype of an orientation terminal with four squared panels corresponding to four functions of the terminal (see Fig. 1). On each panel, there was an object representing a function and a square button. We have picked the following four objects as symbols:

- Wristwatch - represents time-related information. The button provides information about current time and date, daily schedule and weather.

- A spoon - represents eating. The button provides information about lunch time and lunch menu.

- A key - represents going in/out of the room, walking. The button provides spatial information about the nearby surroundings and directions as well as navigation instructions back to a known place (their room).

- A whistle - represents loud noise, draw attention to yourself. The button provides a way to call for help.
<figure><img src="image_2.png" /><figcaption>Fig. 1: Design A – lo-fi prototype introducing four panels with real-world symbols and buttons representing four functions of an orientation terminal.</figcaption></figure>

<table><caption>Table 1: Table of participants for evaluation of design A, P5 is also P3 from evaluation of design C</caption><thead><tr><th>Participant</th><th>Sex</th><th>Age</th><th>Impairment</th><th>Impairment duration</th></tr></thead><tbody><tr><td>P1</td><td>F</td><td>90</td><td>severe</td><td>5+</td></tr><tr><td>P2</td><td>M</td><td>86</td><td>blind</td><td>70</td></tr><tr><td>P3</td><td>F</td><td>86</td><td>severe</td><td>70</td></tr><tr><td>P4</td><td>F</td><td>85</td><td>blind</td><td>8</td></tr><tr><td>P5 (P3C)</td><td>F</td><td>81</td><td>blind (light perception)</td><td>10</td></tr><tr><td>P6</td><td>F</td><td>98</td><td>blind (light perception)</td><td>5+</td></tr></tbody></table>

**Participants.** We recruited six visually impaired older adults from the special housing (5 women, 1 man), mean age 87.7 (MED = 86, MIN = 81, MAX = 98, SD = 5.8), see table 1. The recruitment and execution of all experiments were under the supervision of the special housing authorities.

**Procedure.** The low-fi prototype was introduced to the participants, and then they were asked to perform exploration. We encourage them to comment aloud their exploration. Then we asked them to inspect the explored objects and identify them. After that, participants were given a task to determine what information would be provided by the orientation terminal if they would press the button next to the corresponding object.

**Results.** All participant correctly identified all four objects. All of them also assigned wristwatch to information about time or schedule. All participants assigned the key to a movement; three assigned it to “getting back to room”, three assigned it to “leaving a room” or “leaving a building”. All participants assigned a spoon to food and drink, four of six assigned spoon to a lunch menu, two assigned the information to the time of the lunch. All participants assigned the whistle to a call for help action.

**Conclusion.** The real-life objects were identified without any error. Moreover, the assignment to a terminal function was except the navigation function unambiguous. This can be explained by overloading this option with more slightly different functions (what is nearby and way back to a room). Our suggestions were to split these functions into more objects on the orientation terminal.
<figure><img src="image_3.png" /><figcaption>Fig. 2: Design B – lo-fi prototype introducing a tactile map replacing the panel with a key symbol.</figcaption></figure>

<table><caption>Table 2: Table of participants for evaluation of design B, P4 is P5 from evaluation of design C</caption><thead><tr><th>Participant</th><th>Age</th><th>Sex</th><th>Impairment</th><th>Impairment duration</th></tr></thead><tbody><tr><td>P1</td><td>81</td><td>F</td><td>blind (light perception)</td><td>13</td></tr><tr><td>P2</td><td>94</td><td>F</td><td>severe</td><td>5</td></tr><tr><td>P3</td><td>81</td><td>F</td><td>severe</td><td>23</td></tr><tr><td>P4 (P5<sub>C</sub>)</td><td>70</td><td>M</td><td>severe</td><td>10</td></tr><tr><td>P5</td><td>79</td><td>F</td><td>blind (light perception)</td><td>1</td></tr><tr><td>P6</td><td>90</td><td>F</td><td>severe</td><td>30</td></tr><tr><td>P7</td><td>86</td><td>F</td><td>blind (light perception)</td><td>5</td></tr></tbody></table>

## 3.2 Design B - Adding a tactile map

According to results from the *Design A*, we focused on splitting “what is nearby” and “way back to room” functions. We have replaced the panel with a key symbol by a tactile map, which consisted of four symbols: square representing a room, triangle/arrow representing a continuation of a corridor, a line representing corridor, and a small dot representing the current position (see Fig. 2).

**Participants.** We recruited 7 visually impaired older adults (6 female) from the special housing, mean age 83 ($MED = 81, MIN = 70, MAX = 94, SD = 7, 9$).

**Procedure.** Participants were standing in a hall next to the elevator on the second floor in front of the lo-fi prototype of the terminal. They were asked to explore the terminal and describe aloud the parts of it. We used the Wizard of Oz technique to play corresponding voice information from a nearby tablet after pressing the buttons. After exploration, participants answered the following six questions one by one.

1. Can it tell you what time is it? How? Which button?

2. Can it tell you where you are? How? Which button?

3. Can it tell you what is for lunch? How? Which button?

4. Can it tell you what is around? How? Which button?

5. Can it tell you how to get to a room XY? How? Which button?
6. Can it tell you which direction the nearest staircase is? How? Which button?

**Results.** *P2* and *P7* did not recognize whistle, they thought it is a keychain. All participants correctly recognized all other objects. Questions 1 and 3 were answered correctly in all cases. No one identified “Where I am” button. *P1* and *P6* missed the tactile map part of the terminal. *P1, P2, and P6* did not realize it is a map. *P3* and *P7* though it is a side projection instead of a floor plan. *P5* did not explore the whole range of the tactile map, only near surroundings. *P4* answered that the hierarchy and connections are too confusing.

**Conclusion.** The real-world objects worked again almost perfectly, however, the tactile map did not work at all. It seems that the tactile map is too complicated and too abstract for our target group.

## 3.3 Design C - Replacing a map with tactile cursor

Evaluation of the *Design B* showed that tactile map integrating more functions at once become too complicated and not understandable. We decided to introduce back the key symbol, but we reduced the functionality to “navigating back to the room” only. For the remaining functions “What is around me” and “Current position” we have introduced new artifact called tactile cursor. It consists of three triangle/arrow buttons and square button. The arrow buttons describe what is on the left/right-hand side and behind the participant. The square button describes the current location.

The whistle was changed for a metal one as these were more common during the time participants could encounter them during their life. The design of an orientation terminal now consisted of five color panels with corresponding symbols and buttons on it (see Fig. 3). All distances between objects on the panels were defined to ensure discovery of all object on a particular panel. The distances (button-to-symbol, or button-to-button) are set to a width of one finger. To manifest clearly the borders between panels, the distance between objects and the edge of a panel and the distance between edges of two neighboring panels is set to the width of two fingers. All panels also have different colors to support recognition for people with residual sight.

**Participants.** We recruited 6 visually impaired older adults (4 female) from the special housing, mean age 79.5 (MED = 86, MIN = 52, MAX = 93, SD = 15.8).

**Procedure.** Participants were standing in a hall next to the elevator on the second floor in front of the lo-fi prototype of the terminal. They were asked to explore the terminal a describe aloud the parts of the prototype. We used the Wizard of Oz technique to play corresponding voice information from a nearby tablet after pressing the buttons. Participants were asked to predict what function or information could the buttons serve. After the exploration, participants answered the following eight questions one by one.

1. Can it tell you what time is it? How? Which button?

2. Can it tell you where you are? How? Which button?
<figure><img src="image_4.png" /><figcaption>Fig. 3: Design C – lo-fi prototype introducing a tactile cursor providing information about the surrounding and current location. The panel with key symbol served to navigate the user back to the room. Other three panel’s function did not change.</figcaption></figure>

<table><caption>Table 3: Table of participants for evaluation of design C, P3 is P5 from evaluation of design A, P5 is P4 from design B</caption><thead><tr><th>Participant</th><th>Age</th><th>Sex</th><th>Impairment</th><th>Impairment duration</th></tr></thead><tbody><tr><td>P1</td><td>90</td><td>F</td><td>severe</td><td>20</td></tr><tr><td>P2</td><td>93</td><td>F</td><td>blind</td><td>15</td></tr><tr><td>P3 (P5<sub>A</sub>)</td><td>82</td><td>F</td><td>blind (light perception)</td><td>10</td></tr><tr><td>P4</td><td>90</td><td>F</td><td>moderate</td><td>2</td></tr><tr><td>P5 (P4<sub>B</sub>)</td><td>70</td><td>M</td><td>severe</td><td>10</td></tr><tr><td>P6</td><td>52</td><td>M</td><td>blind</td><td>27</td></tr></tbody></table>

3. Can it tell you what's for lunch? How? Which button?

4. Can it tell you what's around? How? Which button?

5. Can it call you help? How? Which button?

6. Can it tell you how to get back to your room? How? Which button?

7. Can it tell you how to get to a room XY? How? Which button?

8. Can it tell you which direction is the nearest staircase? How? Which button?

**Results.** All objects on the panels were correctly identified. This also includes tactile cursor and its buttons. This panel was often described as a sign-post or pointer. Exchanging the whistle for a metal one improved identification. All predictions of functions were correct except for *P5* who could not assign a key to any reasonable function. All participants correctly answered each of the eight questions. Every participant had a suggestion on modifications of the instructions and information given. Some requested enhanced information and more details; others suggested shortening the information.

**Conclusion.** Tactile cursor was correctly connected with the meaning of directions and movement by all participants. Real-world objects again proved to work as intended. Even the key object (back to my room) was experiencing correct predictions of purpose caused by narrowing down possible functions by the neighboring tactile cursor. Diversion in requests on information granularity and detail suggest further possible improvements by using different possible activations (single push, double push, long push) or usage of the contextual model and serving of personalized information.
# 4 Discussion

The level of abstraction is the key to our design. As we observed, the more abstract is the interface, the less correct mental connections are made between the interface and expected functionality. The usage of real-world objects worked in its full capability exactly as intended as real-world objects are the lowest level of abstraction we could employ. It is also important how close and related are the objects to the intended functionality and interaction context. We observed that even minor details, like the material of the object, can play a key role. For example, the metal whistle worked better than the plastic one. As the metal whistle represented exactly the object visually impaired older adults had a chance to encounter in an earlier life period.

Overloading of a single symbol with too many functions and meanings leads to ambiguous expectations and thus less reliable functionality. It turned out that fine-tuning of minor details of the symbols can improve the functionality of the design. The next step is a series of experiments exploring which aspects are important. For example, if the wristwatch should have a leather or metal watch-band. Should it produce a typical tick-tock sound or be tactfully readable. An interesting idea could be the usage of the main function of the symbols. The wristwatch could really show time and date, as well as the whistle, could be blown on to draw attention.

We also observed that various people with various mental capabilities and preferences require different information and even different level of detail and granularity of the information given. The terminal should then provide a way of identification of the user and users needs and contextual adaptation of the provided information.

# 5 Conclusion and Future Work

We have conducted several iterations of participatory design and qualitative evaluation leading to a functioning interaction with orientation terminal. Usage of real-world objects and low level of symbol abstraction leads to an unambiguous pairing of user expectations and real functions with our user group of visually impaired older adults recruited from special housing.

According to our observation, future steps should be focused on improvements in audio feedback and especially contextual level of detail based on individual users capabilities and cognitive load as users directly asked for this. In terms of both extremes, some users asked for more detail while others for less detailed audio feedback.

# 6 Acknowledgments

This research has been supported by the TACR research program TE01020415 and the project RCI (reg. no. CZ.02.1.01/0.0/0.0/16_019/0000765) supported by EU.
References

1. NS Amick, JM Corcoran, S Hering, and D Nousanen. 2002. Tactile graphics kit. Guidebook. (2002).

2. Eli Blevis, Youn-kyung Lim, Erik Stolterman, Tracee Vetting Wolf, and Keichi Sato. 2007. Supporting design studio culture in HCI. In *CHI'07 Extended Abstracts on Human Factors in Computing Systems*. ACM, 2821–2824.

3. Rupert RA Bourne, Seth R Flaxman, Tasanee Braithwaite, Maria V Cicinelli, Aditi Das, Jost B Jonas, Jill Keeffe, John H Kempen, Janet Leasher, Hans Limburg, and others. 2017. Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: a systematic review and meta-analysis. *The Lancet Global Health* 5, 9 (2017), e888–e897.

4. Aura Ganz, James Schafer, Siddhesh Gandhi, Elaine Puleo, Carole Wilson, and Meg Robertson. 2012. PERCEPT indoor navigation system for the blind and visually impaired: architecture and experimentation. *International journal of telemedicine and applications* 2012 (2012), 19.

5. Daniel Goldreich and Ingrid M Kanics. 2003. Tactile acuity is enhanced in blindness. *Journal of Neuroscience* 23, 8 (2003), 3439–3445.

6. Reginald G Golledge. 1993. Geography and the disabled: a survey with special reference to vision impaired and blind populations. *Transactions of the Institute of British Geographers* (1993), 63–85.

7. Jaume Gual, Marina Puyuelo, and Joaquim Lloveras. 2015. The effect of volumetric (3D) tactile symbols within inclusive tactile maps. *Applied ergonomics* 48 (2015), 1–10.

8. Jaume Gual-Ortí, Marina Puyuelo-Cazorla, and Joaquim Lloveras-Macia. 2015. Improving tactile map usability through 3D printing techniques: an experiment with new tactile symbols. *The Cartographic Journal* 52, 1 (2015), 51–57.

9. Zikitapp Ltd. 2019. Right Hear - App for Visually Impaired & Blind People. (2019). https://right-hear.com/

10. Miroslav Macik, Vojtech Gintner, Dominika Palivcova, and Ivo Maly. 2018. Tactile symbols for visually impaired older adults. In *Cognitive Infocommunications (CogInfoCom), 2018 9th IEEE International Conference*. IEEE.

11. World Health Organization. 2017. Mental health of older adults. (12 Dec 2017). https://www.who.int/news-room/fact-sheets/detail/mental-health-of-older-adults

12. World Health Organization. 2018a. International Classification of Diseases 11 - Vision impairment including blindness. (2018). https://icd.who.int/browse11/l-m/en#/http://id.who.int/icd/entity/1103667651

13. World Health Organization. 2018b. Musculoskeletal conditions. (15 Feb 2018). https://www.who.int/news-room/fact-sheets/detail/musculoskeletal-conditions

14. Clay Spinuzzi. 2005. The methodology of participatory design. Technical communication 52, 2 (2005), 163–174.

15. Zheshen Wang, Baoxin Li, Terri Hedgpeth, and Teresa Haven. 2009. Instant tactile-audio map: enabling access to digital maps for people with visual impairment. In *Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility*. ACM, 43–50.

16. Scooter Willis and Sumi Helal. 2005. RFID information grid for blind navigation and wayfinding. In *null*. IEEE, 34–37.