<figure><img src="image_1.png" /></figure>

HAL
open science

# Mindtraining: Playful Interaction Techniques for People with Dementia

Elisabeth Hackner, Michael Lankes

► To cite this version:

Elisabeth Hackner, Michael Lankes. Mindtraining: Playful Interaction Techniques for People with Dementia. 15th International Conference on Entertainment Computing (ICEC), Sep 2016, Wien, Austria. pp.223-228, 10.1007/978-3-319-46100-7_21. hal-01640286

HAL Id: hal-01640286

https://inria.hal.science/hal-01640286

Submitted on 20 Nov 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Mindtraining: Playful Interaction Techniques for People with Dementia

Elisabeth Hackner and Michael Lankes

University of Applied Sciences Upper Austria,
Department of Digital Media, Softwarepark 11, 4232 Hagenberg, Austria
elisabeth.hackner@students.fh-hagenberg.at
michael.lankes@fh-hagenberg.at

**Abstract.** The paper presents the preliminary studies regarding appli-
cable interaction techniques in the field of tablet games for dementia.
Serious games in the dementia context are a well-researched topic. How-
ever, there is very little knowledge about performing gestures and inter-
actions on tablet computers by users suffering from dementia. Since they
already encounter many restrictions, a touch interface might be another
obstacle. Tablet games often require a different kind of interaction, such
as single tap, swipe, or drag and drop. We developed the Android ap-
plication *Mindtraining* for dementia patients, which integrates multiple
interaction techniques. The purpose of the tablet game is to facilitate
an intuitive and efficient usage of gestures for people with early- and
middle-stage dementia aged over 65 years. In our study we will investi-
gate how people with dementia perform different gestures and how much
help they need each time they use the application.

**Keywords:** Serious games; dementia; Alzheimer's disease; tablet inter-action, cognitive impairment, tablet activation.

# 1 Introduction

The number of people with dementia, especially Alzheimer's disease (AD), is increasing and no cure has yet been found [6]. In addition to medicine-based therapies, also non-medicine-based therapies exist to slow down the progress of the disease. Current studies present a meaningful integration of tablet computers, such as iPads, into the therapy context with the aim of benefiting cognitive abilities or acting as trigger for the patient [2, 8]. Particularly mobile games show great potential to stimulate dementia patients [1, 4]. An advantage of mobile technology is that the hardware is affordable and easy to handle as no external input control (such as a mouse or a keyboard) is needed. Studies often choose children games and educational games for evaluation in the dementia context [1, 2]. Although these types of games seem to be easy to use, they still often require too complex interactions or result in a cognitive overload for people with dementia. A major problem is that most technologies are not designed for people with cognitive disabilities and therefore making operating the device very hard to use.
Many dementia patients, especially people over an age of 65 years, also suf-
fer from visual and/or hearing impairment due to their advanced age. However,
mobile applications integrate assistive technologies very infrequently. These ap-
plications need to fulfill special requirements to ease the usage for people who are
cognitively impaired. Basically, designing for dementia patients means moving
back in time and thinking of events that happened in the past of one individual
and the respective generation [1]. Metaphors, which are frequently used in mobile
applications, have to tie in with the level of knowledge, the experiences and the
cultural background of earlier years. Required gestures for touch displays have
to be simple or alternatively be explained in a tutorial-like style. As dementia
patients tend to have almost no short-term memory, they might need a tutorial
with every time they use the application.

In this paper, we present preliminary findings of interaction techniques in
the tablet game *Mindtraining* in terms of ease of use and how good dementia
patients are able to remember different kinds of gestures. The application serves
as an interactive playful photo album, which shows various content on each
page and integrates multiple different gestures, such as tap, drag and drop,
and swipe. As there seems to be an absence of best-practice approaches for
interaction techniques on tablet devices we focus on a mixture of gestures in our
application. Intuitive interaction techniques could support a more independent
usage of applications on tablets. Caretakers would benefit from this, as they
cannot afford the time to provide technical support and training for every single
patient. Already a small amount of additional workload for nurses can be an
obstacle for using a beneficial serious game in a nursing home.

## 2 Related Work

A dementia disease can affect multiple cognitive abilities, such as memory, atten-
tion, language, reasoning, judgment, reading, and writing [9]. Recent research
shows that video games with cognitive content have the potential of delaying
the cognitive decline [1]. But only a few games were created for the target group
dementia patients in mind, addressing their specific health restrictions.

In a research project, a mobile app was developed that offered personalized
music based on the patient's life, with a special focus on usability [7]. Music and
photos also played an important role in a pilot study, where tablets were used in
dementia therapy for over three months [1]. In this study, they proposed listening
and singing to music or watching photos (animals, patients themselves, family,
children, babies, etc.) and videos as initiating and ending practice in therapy.
This inspires communication and affects the patient's mood in a positive way.
In the same study they mentioned that the caretaker had to give continuous
positive feedback, especially at the end of the tablet therapy to motivate and
encourage the patient. Games, which integrate continuous positive feedback and
encouragement, would relieve the caretaker [1].

Software, especially games, for dementia is a well researched area and there
are many recommendations available for developing dementia games [5, 8]. How-
ever, there seems to be a lack of guidelines for interaction techniques - especially regarding touch interfaces. Senior people might not have any prior knowledge of operating touch devices. Hence, performing different gestures in an application might be already too complex for people with dementia. The developed prototypes in past research projects integrate different interaction techniques, either input with mouse and keyboard, WiiMote controller, touch or ecological gestures. In conclusion, all researchers of the prototypes tried to find a simple and natural way to interact with the respective system to reduce the cognitive overload for people with dementia. Either touch interaction or ecological gestures seemed to be the most successful interaction paradigms. These do not require any external controller, such as a mouse or keyboard. In a study with mobile games and elderly people, the participants reported that it is much easier to touch the objects on the screen than using a mouse [10]. Learning how to interact with a software system is a complex task, in particular for users, who never used computer systems before. Yet, studies have shown that elderly people with dementia are willing to learn new technologies [11, 3]. In another study with dementia patients, half of the participants had never used a mobile device before. However, all of them indicated that they would like to see more technology and believed they could benefit from it [4]. Moreover tablets cover many positive features like high-quality responsiveness multi-touch screen, mobility, accessibility and ease of aquition [8].

## 3 Game Prototype Mindtraining

Based on related work, we established the requirements for *Mindtraining* and extended the application with different interaction techniques. The requirements include: (1) providing visual and auditory help; (2) matching the mental model with the conceptual model; (3) giving motivational messages and positive feedback when achieving a goal or making a mistake; (4) providing short and simple messages; and (5) counteracting the declining cognitive abilities of the persona [5, 1, 3]. *Mindtraining* is an interactive photo album with a playful integration of cognitive games. A photo album is an item, which is also known from the earlier years of nowadays dementia patients, hence it should present a good mental model for the players. Visual aids, especially photos, can stimulate the memories and are therefore a proofed concept in dementia therapy [1]. Playing the game requires different gestures and interaction techniques. In the album you have to swipe through the pages, tap on photos to see them in full screen and use tap for turning on and off a music radio. The radio plays songs of the earlier years of the patients and should serve as alternate stimulation of the user. Also a cognitive game with completing of sayings is implemented on multiple pages with two different scenarios: (1) tapping on the right answer, (2) dragging the right answer to a target location. Wordplays, including sayings, are widely used in mind training therapies for dementia patients to exercise their mind.

The system is complemented with a logging module which tracks the interaction of the user and logs the data in JSON format to files on the device. The
module logs single tap, double tap, long press, swipe from left to right and vice versa, swipe from top to bottom and vice versa and dragging interactions of draggable elements. The logging is applied to the following view types of Android: **TextView**, **Image**, **Button**, **ImageView** and **ImageButton**.

Our objective is to investigate how people with dementia cope with different gestures in a mobile playful application. On the one hand it is interesting how good people can remember gestures and how intuitive they are. On the other hand we want to find out if one of the gestures might be easier to perform than another one and give recommendations of applying gestures in a mobile application.

# 4 Preliminary Study

The application was tested by eight participants, see Fig. 1 (age: 56-90, gender: 3 male, 5 female). Three of the participants have early-stage dementia, one has middle-stage dementia, the others don't have dementia. The results of these tests were used to optimize the application and eradicate remaining usability problems of *Mindtraining*. The device used is a 10.1" sized ASUS MeMO Pad 10 Android tablet. A protection case covers the back of the device, which firstly gives additional grip and secondly provides a stable base to position it on a table in the right angle.

<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Typical setting of the field tests: Subject playing the game, a carer sitting next to her and supporting her when questions arise and the observer in the background, taking notes.</figcaption></figure>
In total, four main iterations, each included planning, implementing, testing and evaluation, took place. Besides adapting sizes for fonts and graphics, two important issues were optimized during these iterations. At first people were instructed which gestures had to be performed, such as tapping on a photo or swiping for flipping a page. When it came to dementia patients, switching between the different gestures caused problems. As they had learned that swiping from right to left resulted in the intended outcome, they also tried swiping on photos and buttons. To solve the cognitive overload of switching between different gestures, simple animations for tapping and swiping were implemented at the desired positions to provide a visual instruction how to perform the gesture on the element. This optimization showed that dementia patients needed fewer instructions from the caretaker on how to interact with the different elements.

The second main usability problem, which had been observed, is that a single tap is often difficult to perform by elderly people including dementia patients. The main issues are sideslipping of the finger and the duration of the tap. Also the user sometimes touches the screen with the fingernails, which has no effect on a capacitive display. With Android's standard functionality of clickable elements the action is only executed when performing a single tap on the element. To solve this issue we extended Android's standard functionality for clickable elements in a way that these elements also react to gestures, such as swipe, double tap and long press. The logging module tracks, which gesture has been performed on an element. This will allow an analysis of the usage of gestures afterwards. Due to the single tap problem, the scenario with dragging the right answer to a target location in the sayings game, could be performed much faster and more fluently than single tapping on the right answer.

As next step, a field test with three participants with early-stage dementia with 75-90 years in age, will be conducted to observe the interactions. The test will take place in a small day-care facility where dementia patients get cared for three times a week. To examine which gestures on a tablet device are best suited for this target group, we integrated an automatic logging module in the final prototype. The logging module tracks how a user interacts with visible elements on the screen. Additional to the logging module, an observation protocol will serve as a documentation of qualitative data during the field test.

# 5 Conclusion and Future Work

In this paper we presented the purpose of serious games in the dementia context and introduced *Mindtraining*, an Android application that facilitates an intuitive usage of gestures. Software design guidelines for people with dementia are a common topic in research, but there seems to be a lack of recommendations when it comes to touch devices and their interaction techniques. In our application we are tracking and observing the user's interaction with the tablet to find out which gestures can be performed better than others. The next step will be conducting a field test with dementia patients and analysing the used interactions in the application *Mindtraining*.
References

1. Aktivierung von Menschen mit Demenz im Pflegeheim. ZQP-Studie (2014)

2. McCallum, S. and Boletsis, C.: A Taxonomy of Serious Games for Dementia. In Schouten, B., Fedtke, S., Bekker, T., Schijven, M. and Gekker, A., ed., Games for Health, Springer Fachmedien Wiesbaden, pp. 219-232 (2013)

3. Lim, F. S., Wallace, T., Luszcz, M. A. and Reynolds, K. J.: Usability of Tablet Computers by People with Early-Stage Dementia. Gerontology (2013)

4. Kong, A. P.-H.: Conducting Cognitive Exercises for Early Dementia With the Use of Apps on iPads. Communication Disorders Quarterly (36:2), pp. 102-106 (2015)

5. Bouchard, B., Imbeault, F., Bouzouane, A. and Menelas, B.-A.: Developing Serious Games Specifically Adapted to People Suffering from Alzheimer. In Ma, M., Oliveira, M., Hauge, J., Duin, H. and Thoben, K.-D., ed., Serious Games Development and Applications, Springer Berlin Heidelberg, pp. 243-254 (2012)

6. Alzheimer's Disease International: Global Knowledge, Dementia statistics, http://www.alz.co.uk/research/statistics

7. Nezerwa, M., Wright, R., Howansky, S., Terranova, J., Carlsson, X., Robb, J. and Coppola, J. F.: Alive Inside: Developing mobile apps for the cognitively impaired. In Systems, Applications and Technology Conference (LISAT) IEEE Long Island, IEEE, pp. 1-5 (2014)

8. Tom, R., Pereira, J. and Oliveira, M.: Using Serious Games for Cognitive Disabilities. In Ma, M., Oliveira, M. and Baalsrud Hauge, J., ed., Serious Games Development and Applications, Springer International Publishing, pp. 34-47 (2014)

9. Gogia, P. and Rastogi, N.: Clinical Alzheimer Rehabilitation, Springer Publishing Company (2008)

10. Cota, Tlio Teixeira and Ishitani, Lucila and Vieira Jr., Niltom: Mobile game design for the elderly: A study with focus on the motivation to play, Computers in Human Behavior (2015)

11. Lauriks, S. and Reinersmann, A. and Van der Roest, H. G. and Meiland, F. J. M. and Davies, R. J. and Moelaert, F. and Mulvenna, M. D. and Nugent, C. D. and Dres, R. M.: Review of ICT-based services for identified unmet needs in people with dementia, Ageing Research Reviews (2007)