# MyCompanion: A Digital Social Companion for Assisted Living

Fernando Loizides, Kathryn E. Jones, Daniel Abbasi, Christopher Cardwell, Ieuan Jones, Liam D. Turner, Athanasios Hassoulas, Ashley Bale, Scott Morgan

<figure><img src="image_1.png" /></figure>

## To cite this version:

Fernando Loizides, Kathryn E. Jones, Daniel Abbasi, Christopher Cardwell, Ieuan Jones, et al., MyCompanion: A Digital Social Companion for Assisted Living. 17th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2019, Paphos, Cyprus. pp.649-653, 10.1007/978-3-030-29390-1_55. hal-02878621

HAL Id: hal-02878621

https://inria.hal.science/hal-02878621

Submitted on 23 Jun 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# MyCompanion: A Digital Social Companion for Assisted Living

Fernando Loizides¹[0000-0003-0531-6760], Kathryn E.

Jones¹[0000-0001-7939-9314], Daniel Abbasi¹, Christopher Cardwell¹, Ieuan

Jones¹, Liam D. Turner¹[0000-0003-4877-5289], Athanasios Hassoulas¹, Ashley

Bale², and Scott Morgan³

¹ Cardiff University, UK

{floizides,jonesk90,abbasid,cardwellcj,jonesin,turner19,hassoulasa2}

@cardiff.ac.uk

² Innovate Trust, UK

ashley.bale@innovate-trust.org.uk

³ Bridgend College, UK

smorgan@bridgend.ac.uk

**Abstract.** In this work, we present the initial progressive build of a prototype system, dubbed **MyCompanion**. MyCompanion to improve assisted living, ultimately targeting alleviating social exclusion and loneliness. The system, based on a Raspberry Pi, connects to a user's television set and tracks data from sensors placed around the house. An inbuilt artificial intelligence chat-bot then listens to, as well as instantiates, conversation with the user based on self-imposed triggers. Data from the sensors inform MyCompanion in real time user behaviour and records these readings. We also port MyCompanion software to an anthropomorphic version inside a humanoid robot for us to be able to give a physical dimension to the digital character.

**Keywords:** Digital Companion · Robot · Loneliness · Digital Exclusion

## 1 Introduction and Motivation

Depression is one of the prevalent chronic disease in ageing adults, with nearly half of the UK population (7.7 million) aged 55+ experiencing it at some point [1,2]. Loneliness has been shown to be a contributor towards depression [17], with a direct correlation between loneliness, and increasingly ageing population [3].

Technological intervention to provide aid, support and assist all stakeholders involved, including the elderly, carers, family, and friends has been of increasing research interest. In particular, we note that the emerging capability of AI and Robotics have been to shown to have positive effects on reducing the feeling of loneliness among the elderly. Robots for example, have been able to stimulate interaction between elderly people that are co-located [20]. Social robots are “designed to interact with people in a socio-emotional way during interpersonal interaction” [15]. The timely fashion of this kind of research is paramount as
there is an increased understanding of “the health care sector inability to provide qualitative care for older people” [15]. Other reasons include global ageing, shortage of health care professionals, to tackle the growing demands of patients and chronic diseases such as depression [12]. As research has shown, elderly now respond well to robots, even claiming emotions such as love for them [20]. Feelings of people with depression can also be improved by the use of robot intervention [25].

We hypothesise that with the appropriate use of anthropomorphic robots as companions in the home (instead of care homes) can strongly benefit the prevention or improvement of loneliness. We aim for the robots to have the attributes of ‘companions’ rather than helpers, since we are aware “that people who help others gain a sense of well-being compared to people who are only in need of help” [15]. In this paradigm the robot becomes something that needs to be governed. In fact, “the company of robots may be preferred to humans because we as human beings feel that we are in control in a humanrobot relationship and that robots will not judge us like other people do” [23]. Furthermore, we are confident that the successful completion of a prototype for this project reduce isolation and increase conversational opportunities with the robot itself as well as with other humans [22]; a robot or telecare-system might reduce certain risks and help the user to self-manage their health and well-being [13]. If technology is applied correctly, it can “enable and empower older people” [18]. This method of using a social robot, with all the characteristics we have included, has been shown to provide positive “feel well” results amongst elderly people [21] and therefore suitable for our goal of reducing mild depression through loneliness and social exclusion. In this work, we present the first working prototype system of **MyCompanion**, a digital assistant that provides company and enables individuals who are socially excluded to become more connected and healthier.

## 2 Artefact Creation and Description

The process of requirements gathering was performed by semi-structured interviews with stakeholders. Our participants ranged from health care workers to robotisists, carers and government medical advisers. Upon requirements elicitation, storyboards and use cases were created to guide the design and a structured dialogue design approach [14] was taken to collate the functional and non-functional components. Low fidelity prototypes were used as an initial point internally to the design and developer team. From these a round robin approach was used to produce the components that covered a minimum viable product in terms of stakeholder requirements.

The prototype system uses several components to create an interactive digital companion. It currently has four main functions. The first, is to ‘be needy’ and allow the user to take care of the companion, thus feeling useful and needed. Unlike digital assistants that are currently commercially available, MyCompanion will initiate a conversation and ask the user for help in some matters, such as for example asking what the weather is outside or for the user to tell it a joke
or story. This invokes feelings of usefulness and of being needed by the users, a factor that has serious benefits to the health of an older adult [16]. The second, is to seamlessly present social media content from the users friends and family to them. The system will detect Twitter posts and ask the user whether they would like to see pictures of what their friends have posted or read out what they are saying. Thirdly, MyCompanion is also fitted with algorithms for Behavioural Activation for Depression [24] to ask the user at regular intervals how they are feeling and store the data for analysis, as well as respond with an appropriate suggestion to improve the situation. Finally, as a fourth main function, MyCompanion is connected to sensors around the home. These sensors include motion, temperature, door and light sensors currently implemented. The sensors take readings that are currently at an experimental phase. Using these readings, we are able to then apply machine learning techniques to detect normal routine over time and also deviations from this that may constitute a dangerous situation. As a predicted example, if one enters the kitchen at night time, and does not switch on the light for a few minutes then this would be flagged as abnormal behaviour, and MyCompanion would attempt to engage the user and establish that all is well. The full system technical architecture can be seen in Figure 1.

<figure><img src="image_3.png" /><figcaption>Fig. 1. MyCompanion System Architecture</figcaption></figure>

The system utilises Electron [4] to create a responsive front-end (See Figure 2: Right). The artificial intelligence and conversational module of MyCompanion is built on the foundations of Mycroft [5] for the text to speech and vice versa and for natural language processing. This also allowed us to build a more robust system which can work as a basic version when an internet connection is made
unavailable. Raspberry Pis (version 3 B+) [6] are used to control different sensors around a home. The sensors then transmit this data with timestamps to a cloud based Python server and uses Azure [7] to store a user's data. ROS [8] is used as the main SDK for communicating the concepts with the robot. The ability of this SDK to allow for messages (scripts) to communicate on different nodes makes it ideal for communication between the robot body and the main server. For this prototype, we chose a Pepper type robot [9] to represent the anthropomorphic body (see Figure 2: Left) of MyCompanion. Finally, a wireless omni-directional microphone is used in a generic area of the home as an input device for the user's speech.

<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Left: Anthropomorphic version of MyCompanion, using ROS to connect the functionality to a Pepper Robot. Right: On Screen (non-anthropomorphic) version of MyCompanion</figcaption></figure>

## 3 Conclusions and Future Work

We presented the initial stages of a project named MyCompanion, which uses sensors and a chat bot loaded on a raspberry pi to create interactions with individuals. We are now ready to begin user testing and refining the AI to understand user requirements and react to them. The next stages of the project involve the implementation of the technology to individuals' homes and extract data through in the field research. We plan to implement the research design process (slightly adapted) by J. Khakurel et al. [19] in which two stages, identifying and prioritising, is used. From the findings, we aim to recognise the effects of MyCompanion's need for interaction and create a second iteration of the system to build upon positive findings. We will then implement the installations on a larger scale and allow for larger data streams for behavioural analysis via machine learning which will be able to give us we are hoping will give us insight into changes of behaviour which are not clear with visual inspection.

## 4 Acknowledgements

We would like to thank Claire George & Carl Bickle (Bridgend College) [10] and Carl Clement (Emotion Robotics) [11] for their support.
## References

1. https://tinyurl.com/y6bwo8yv, May 2019.

2. https://tinyurl.com/y4syk49f, May 2019.

3. http://www.monmouthshire.gov.uk/govtech challenge, May 2019.

4. https://electronjs.org/, May 2019.

5. https://mycroft.ai/, May 2019.

6. https://www.raspberrypi.org/, May 2019.

7. https://azure.microsoft.com/en-gb/, May 2019.

8. http://www.ros.org, May 2019.

9. https://tinyurl.com/y5qtxqcf, May 2019.

10. https://www1.bridgend.ac.uk/, May 2019.

11. http://www.emotion-robotics.com, May 2019.

12. Cynthia L Breazeal. *Designing sociable robots*. MIT press, 2004.

13. Blanca Callén, Miquel Domènech, Daniel López, and Francisco Tirado. Telecare research:(cosmo) politicizing methodology. *ALTER-European Journal of Disability Research/Revue Européenne de Recherche sur le Handicap*, 3(2):110-122, 2009.

14. Alexander N Christakis and Kenneth C Bausch. *CoLaboratories of Democracy: How People Harness Their Collective Wisdom to Create the Future*. IAP, 2006.

15. Susanne Frennert and Britt Östlund. Seven matters of concern of social robots and older people. *International Journal of Social Robotics*, 6(2):299-310, 2014.

16. Tara L Gruenewald, Arun S Karlamangla, Gail A Greendale, Burton H Singer, and Teresa E Seeman. Feelings of usefulness to others, disability, and mortality in older adults: The macarthur study of successful aging. *The Journals of Gerontology Series B: Psychological Sciences and Social Sciences*, 62(1):P28-P37, 2007.

17. Bonnie M Hagerty and Arthur Williams. The effects of sense of belonging on social support, conflict, and loneliness on depression. *Nursing res.*, 48(4):215-219, 1999.

18. Kelly Joyce and Meika Loe. A sociological approach to ageing, technology and health. *Sociology of health & illness*, 32(2):171-180, 2010.

19. Jayden Khakurel, Antti Knutas, Helinä Melkas, Birgit Penzenstadler, Bo Fu, and Jari Porras. Categorization framework for usability issues of smartwatches and pedometers for the older adults. In *International Conference on Universal Access in Human-Computer Interaction*, pages 91-106. Springer, 2018.

20. Cory D Kidd, Will Taggart, and Sherry Turkle. A sociable robot to encourage social interaction among the elderly. In *Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006.*, pages 3972-3976. IEEE, 2006.

21. Tobias Körtner, Alexandra Schmid, Daliah Batko-Klein, Christoph Gisinger, Andreas Huber, Lara Lammer, and Markus Vincze. How social robots make older users really feel well-a method to assess users concepts of a social robotic assistant. In *International Conference on Social Robotics*, pages 138-147. Springer, 2012.

22. Amanda Sharkey and Noel Sharkey. Granny and the robots: ethical issues in robot care for the elderly. *Ethics and information technology*, 14(1):27-40, 2012.

23. Sherry Turkle. *Alone together: Why we expect more from technology and less from each other*. Hachette UK, 2017.

24. David Veale. Behavioural activation for depression. *Advances in Psychiatric Treatment*, 14(1):29-36, 2008.

25. Kazuyoshi Wada, Takanori Shibata, Tomoko Saito, and Kazuo Tanie. Analysis of factors that bring mental effects to elderly people in robot assisted activity. In *IEEE/RSJ International Conference on Intelligent Robots and Systems*, volume 2, pages 1152-1157. Ieee, 2002.