Dynamically Adapting the Environment for
Elderly People Through Smartwatch-based
Mood Detection
Antonio Capodiecia1, Pascal Budnerb, Joscha Eirichc, Peter Gloord, Luca
Mainettia
Università del Salento, Lecce, Italy. {antonio.capodieci, luca.mainetti}@unisalento.it
University of Cologne, Cologne, Germany, pbudner@smail.uni-koeln.decUniversity of Bamberg, Bamberg, Germany, joscha-friedrich.eirich@stud.uni-bamberg.dedMIT, Cambridge, MA, USA, pgloor@mit.edu
abAbstract The ageing population and age-related diseases are some of the mosturgent challenges in healthcare. This leads to an increasing demand in innovative
solutions to afford a healthy and safe lifestyle to the elderly. Towards this goal, theCity4Age project, funded by the Horizon 2020 Programme of the European
Commission, focuses on IoT-based personal data capture, supporting smart cities
to empower social/health services. This paper describes the combination of thesmartwatch-based Happimeter with City4Age data capture technology. Throughmeasuring the mood of the wearer of the smartwatch, a signal is transmitted to thePhilips Hue platform, enabling mood controlled lighting. Philips Hue allows thewireless remote control of energy-efficient LED light bulbs. Thus, measuring themood through the Happimeter, the living environment for elderly people can be
dynamically adapted. In our experiments, by changing colors and brightness oflight bulbs using the Philips Hue platform, their quality of life can be improved. A
validation test will be done in the context of the City4Age project, involving 31
elderly people living in a Southern Italian city.

1 Introduction
The early detection of risks related to a specific health condition can help
clinicians to enact appropriate interventions that can slow down the progression of
the condition, with beneficial effects on both patients’ quality of life and treatment
costs. A disease, which is diagnosed late, in addition to decreasing the chance ofrecovery, also increases the cost of treatments borne by the community. In thiswork we try to avoid diseases by detection depression and emerging pain early,
tracking the mood of elderly people, and initiate action if the mood drops.
Mood is very important for the health of a person [1, 2]. Light intensity and color
affect and alter the mood [3] and reduce or even avoid depression altogether. In
addition there exists a ripple effect of positive emotional contagion, where group
1

Corresponding author: Antonio Capodieci, email: antonio.capodieci@unisalento.it

2

members experience improved cooperation, decreased conflict, and increase
perceived task performance [4].
The goal of this paper is to present a work in progress in the context of the
City4Age research project (assisted ambient living for elderly people), and todescribe how the MIT smartwatch-based Happimeter, by controlling the lighting
environment (Philips Lamp Hue), could positively impact the mood of elderly
people.

1 Research context
2.1 The City4Age project“City4Age - Elderly-friendly city services for active and healthy ageing”, a project
co-funded by the Horizon 2020 Programme of the European Commission,
supports smart cities to empower social/health services for the elderly population.
The City4Age [5] project aims to use digital technologies [6] at home and in thecity to monitor behaviors of elderly people with the goal of improving their
quality of life. The geriatricians involved in the project have defined a model ofrisk, which is based on a set of Geriatric Factors (GEFs, like mobility, physicalactivity, socialization, etc.), and Geriatric Sub-factors (GESs, like walking,climbing stairs, attending senior centres, restaurants, etc.). Each GEF and GES isrelated to a set of measures, which we call “Low Level Activity” (LEAs, like thetimestamp when a user enters a pharmacy, the timestamp when a user leaves a
pharmacy) extracted directly from the IoT sensors. By using a range of digitalsensors, information concerning movements, locations, and activities will berecorded from participants and analyzed by a central engine to detect and predict
behavioral patterns. Additionally, as a result of their behavioral pattern analysis,
participants will receive interventions aimed at improving their behaviors. In thelast few years, several projects funded by the European Union have dealt with theapplication of Information and Communication Technology (ICT) to capture
personal and/or environmental data in order to derive behavioral patterns useful toprevent mental or physical diseases, like for example the Monarca project [5] that
deals with bipolar disorder. This approach is also adopted by the City4Age project
that, different from other projects, proposes dynamic intervention technologies torecord and observe behavioral changes.

3Fig. 1. Pebble Smartwatch user interface

2.2 The Happimeter project
The Happimeter is a novel device to instantaneously measure individual happiness[7]. It uses commercially available smartwatches to build a body sensing systemthat can measure individual mood states and interactions between people. It
consists of a Pebble smartwatch, and a smartphone app. The smartwatch is
integrated with each user’s smartphone to access the phone’s location sensing anddata transmission capacity, as well as its processing power. The smartwatches
Fig.2. Happimeter smartphone app user interface

4

provide data on the body movement through accelerometer, lighting level andheart rate. Unlike sociometric badges, which we have used in earlier experiments[8], the watches are designed to be worn constantly, naturally and non-intrusively,
and their rechargeable batteries have robust charge length. Their displays alsoenable easy two-way communication to give status updates to wearers (Fig. 1).
The Pebble Happimeter app is extended with a smartphone Happimeter app foriPhone and Android, which provides additional information on individual andgroup mood state (Fig. 2).
With the “Happimeter” users are able to track how they feel and how happy they
are for extended periods of time. Through machine learning algorithms, they will
get regular reports with insights about their happiness as well as the factors that
influenced the happiness. We use this sensing system to track work groupmood/interactions. To increase accuracy of the prediction model, users can also doa standardized personality test (Neo-FFI) to learn about their personality
characteristics, and further increase the precision of the Happimeter.

2.3 Philips Hue lighting system
Philips Hue [9] is a wireless home lighting system, based on LED lights, whichenables users to create and control, remotely over WI-FI, the lights using theirmobile smart device. Philips Hue was launched by Philips in 2012 and now is at
the second version. The system is based on three main components: The Lights –smart bulbs containing 3 types of LED able to produce a wide range of colors andintensities. Lights Bulbs are connected to a bridge via an open standards protocolcalled ZigBee[10]. It’s possible to create a mesh network, between the bulbs,
extending the range and making the system more interactive. The Bridge is a
device used to enable smart bulbs to communicate with controllers as well as othersmart bulbs. A set of APIs is offered by the Bridge. These allow users to control
all settings of the lights. These APIs are accessible only from local networks. The
Portal is a web based control panel, which connects the Bridge to the Internet anddelivers commands from outside.

2.4 Privacy Issues
One of the key aspects of the Internet of Things is its impact on privacy[11,12]. In a world of smart communication and computation devices, everything
we say, do, and sometimes feel, can be digitized, stored, and tracked. This calls for
systems to be designed to require users to explicitly agree before taking privacydecreasing actions [13]. To protect the privacy of participants [14], we designed
our system to preserve anonymity, we also inform the participants of the datacollected in the platform, and educate them on how the system works.

5

3 Adapting the Environment System and Test Bed Description
The goal of our experiment is to demonstrate the combination of the City4Age
technology with the Happimeter by monitoring and adapting the environment of asmall set of elderly people. This will be achieved by tracking elderly
indoor/outdoor activities and mood, with the use of unobtrusive technologies, andrequesting voluntary data input and professional cognitive assessing tools.
The main stakeholders involved in the test bed are elderly people, their formal
caregivers (including care coordinators) affiliated to social services, seniorcenters, and city services.
The elderly person is the main actor in this scenario, who needs to maintain good
physical self-sufficiency and an active social engagement. Users have been
identified among those who frequent the social centers offered to seniors by theCity of Lecce and willing to participate in our experiment. The sample iscomposed of 31 seniors: 11 males and 20 females, they are 70+ years old and they
are in good physical condition, moving independently and frequently attending thesocial centers to facilitate progress monitoring. They have been selected becauseof their pre-medical fragility, particularly cognitive-functional, to include asegment of the population otherwise not addressed by public services. 45% of thesample is made up of pairs of husband and wife (7 couples) to analyze genderdifferences. In order to monitor citizens’ movements, we chose people who residein two different city districts.
The goal is to detect user behavior and mood both in indoor and outdoor
environments, and to adapt their environment (mainly indoor) based on thepredicted mood. Elderly users will be provided with a smartphone and asmartwatch with heart rate device. Moreover, their homes will be equipped with acomplete kit of the Phips Hue Light system and beacon based indoor positioning
infrastructure and some smart plugs to detect the usage of appliances (like TV andwashing machine). In the city environment, beacons and smartphone GPS
receivers will be used to detect elderly people’s outdoor mobility.
Based on the feedback received from the Happimeter, the Philips Hue system willchange the value of colors and brightness of the light. For example, if the mood is
low the light bulbs might emit a soft and relaxing light.
In fig. 3 we show the logical architecture of the system (on the left) and a photo of
the prototype environment (on the right). The smartphone collects the informationfrom the City4Age platform and other IoT devices (beacons, accelerometers,
gyroscopes), such as a smartwatch with the heart rate module. The information issent to the Happimeter platform that answers with the mood prediction. Based onmood prediction the smartphone dynamically changes the state of light bulbs.

6

We will equip the 31 seniors with a hue system in their home, as well as with the
Pebble happimeter. As we do not anticipate that the seniors will be able to train
the individual happimess models, we will employ the generic preconfigured model
to track their happiness. A researcher will visit each senior two times per week tocheck in on the system, and assist the elderly person in recharging the device. The
hue system will be programmed to reflect the mood of the wearer, throughadapting the warmth and intensity of the light. The color will change from cold
white to yellowish warm white, based on the mood, while the intensity willchange from dimmed light at low activity to very bright light at high activity.

4 Conclusion and Expected Results
In this paper we presented a research project in progress based on the idea of
integrating two different running projects: (i) City4Age – which utilizes data fromsmart cities and ad-hoc sensors for the prevention of mild cognitive impairment
and frailty of aged people, detecting behavior changes and intervening on it – and(ii) Happimeter – a combined hardward and software technology to predict theindividual mood based on body sensors. As first integrated prototype has been
implemented, demonstrating that exploiting such technologies it is possible todynamically adapt the user living environment to his/her mood, making it more
confortable. This way we intend to improve the quality of life of elderly people
with the long-term goal of unobtrusively stimulating behavior changes.
Our next research steps will be focused on defining and implementing moresophisticated intervention strategies on the user’s living environment, interactingwith it through sensors/actuators. We are also interested in experimenting with our
prototypes and learning from actual users who participate in Lecce’s City4Age
test bed.

Fig. 3. Logical Architecture (left) – A photo of the prototype environment (right)

7

Acknowledgment
This work is supported by the City4Age project funding from European Union’s
Horizon 2020 research and innovation programme under the grant agreement No
689731, and by Philips Lighting as part of an overall MIT research grant. Special
thanks to Paolo Panarese for his contribution in developing the test prototype.

References1.

DeLongis A, Folkman S, Lazarus R (1988) The impact of daily stress on health and mood:
psychological and social resources as mediators. J Pers Soc Psychol 54:486–95. doi:
10.1037/0022-3514.54.3.486

2.

Salovey P, Birnbaum D (1989) Influence of mood on health-relevant cognitions. J Pers Soc
Psychol 57:539–551. doi: 10.1037/0022-3514.57.3.539

3.

Daurat A, Foret J, Touitou Y, Benoit O (1996) Detrimental influence of bright light exposureon alertness, performance, and mood in the early morning. Neurophysiol Clin Neurophysiol
26:8–14. doi: https://doi.org/10.1016/0987-7053(96)81529-2

4.

Barrett LF (2006) Solving the Emotion Paradox: Categorization and the Experience ofEmotion. Personal Soc Psychol Rev 10:20–46. doi: 10.1207/s15327957pspr1001_2

5.

Mainetti L, Patrono L, Rametta P (2016) Capturing behavioral changes of elderly people
through unobtruisive sensing technologies. 2016 24th Int Conf Software, Telecommun
Comput Networks 1–3. doi: 10.1109/SOFTCOM.2016.7772126

6.

Mainetti L, Manco L, Patrono L, et al (2016) An ambient assisted living system for elderly
assistance applications. 2016 IEEE 27th Annu Int Symp Pers Indoor, Mob Radio Commun
1–6. doi: 10.1109/PIMRC.2016.7794963

7.

Gloor P (2017) Consistent Excitement Correlates with Happiness - Predicting Mood ThroughBody

Sensing

with

Smartwatches.

In:

ICKN

White

Pap.

January2017.

http://www.ickn.org/documents/Mood_Tracking_Smartwatch.pdf.
8.

Olguin O, Gloor P a, Pentlanda (2009) Wearable Sensors for Pervasive Healthcare

Management. Proc IEEE 66:1–4. doi: 10.4108/ICST.PERVASIVEHEALTH2009.6033
9.

Philips Hue. http://www.meethue.com. Accessed 20 May 201710.

ZigBee Alliance. http://www.zigbee.org/. Accessed 20 May 201711.

Langheinrich, M. (2001) Privacy by Design — Principles of Privacy-Aware Ubiquitous
Systems, in: Ubicomp 2001 Ubiquitous Comput., 2001: pp. 273–291. doi:10.1007/3-54045427-6_23.

12de Montjoye, Y-A. Hidalgo, C.A. Verleysen,M. Blondel, V.D. (2013) Unique in the Crowd:
The privacy bounds of human mobility, Sci. Rep. 3 (2013) 1376. doi:10.1038/srep01376.

13

Neustaedter, C. Greenberg, S. (2003) The Design of a Context-Aware Home Media Space
for Balancing Privacy and Awareness, UbiComp 2003 Ubiquitous Comput. 297–314.
doi:10.1007/978-3-540-39653-6_24.

14

Butler,D.

(2007)

doi:10.1038/449644a.

Datasharing

threatens

privacy.,

Nature.

449:

644–645.