# An Open Dataset for Human Activity Analysis using
Smart Devices

Sébastien Faye, Nicolas Louveton, Sasan Jafarnejad, Roman Kryvchenko,

Thomas Engel

► To cite this version:

Sébastien Faye, Nicolas Louveton, Sasan Jafarnejad, Roman Kryvchenko, Thomas Engel. An Open Dataset for Human Activity Analysis using Smart Devices. 2017. [hal-01586802](https://hal.science/hal-01586802)

HAL Id: hal-01586802

https://hal.science/hal-01586802

Preprint submitted on 13 Sep 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# An Open Dataset for Human Activity Analysis using Smart Devices

Sébastien Faye<fnref n="1" />, Nicolas Louveton<fnref n="2" />, Sasan Jafarnejad<fnref n="1" />, Roman Kryvchenko<fnref n="1" />, and Thomas Engel<fnref n="1" />

¹University of Luxembourg, SnT
6 avenue de la Fonte, L-4362 Esch-sur-Alzette, Luxembourg
Email: firstname.lastname@uni.lu

²Université de Poitiers, CNRS, Centre de Recherches sur la Cognition et l'Apprentissage
5 rue Théodore Lefebvre, F-86000 Poitiers, France
Email: nicolas.louveton@univ-poitiers.fr

## Abstract

The study of human mobility and activities has opened up to an incredible number of studies in the past, most of which included the use of sensors distributed on the body of the subject. More recently, the use of smart devices has been particularly relevant because they are already everywhere and they come with accurate miniaturized sensors. Whether it is smartphones, smartwatches or smartglasses, each device can be used to describe complementary information such as emotions, precise movements, or environmental conditions. In this short paper, we release the applications we have developed and an example of a collected dataset. We propose that opening multi-sensors data from daily activities may enable new approaches to studying human behavior.

**Keywords:** Human Mobility, Smart Devices, Sensing Systems, Opendata

## 1 Overview

Our sensing system relies on the parallel use of three complementary devices, as described in Table 1 and Figure 1.

<table><caption>Table 1: Specification of the devices used in our studies.</caption><thead><tr><td>Device</td><td>Type</td><td>Main metrics</td><td>Battery during data collection</td><td>Network Interfaces</td></tr></thead><tbody><tr><td>Google Nexus 5X</td><td>Phone</td><td>Contextual data</td><td>Up to 20h</td><td>LTE, Wi-Fi, Bluetooth</td></tr><tr><td>LG Watch Urbane 2</td><td>Watch</td><td>Physiological data</td><td>Up to 20h</td><td>LTE, Wi-Fi, Bluetooth</td></tr><tr><td>Jins MEME ES.R</td><td>Glasses</td><td>User activity</td><td>Up to 16h</td><td>Bluetooth</td></tr></tbody></table>

First of all, a smartphone is used to capture mainly contextual data. Two applications are used: a simple data collection application based on the SWIPE open-source sensing system<fnref n="1" /> [FLGE16], and a logbook application for obtaining real data on user activity (aTimeLogger<fnref n="2" />). SWIPE is a platform for sensing, recording and processing human dynamics using smartwatches and smartphones.

Then, a smartwatch is used primarily to capture the user's heart rate. Motion data is also collected, without being at the heart of the dataset due to its need to be configured with a low

<footnote n="1">¹https://github.com/sfaye/SWIPE</footnote>

<footnote n="2">²http://www.atimelogger.com/</footnote>
<figure><img src="image_2.png" /><figcaption>Figure 1: Overview</figcaption></figure>

sampling frequency, which would drastically increase the dataset and drain the battery as well. An application based on SWIPE is used.

Finally, JINS MEME smartglasses are used. This model has the advantage of being compact and simple to carry. It does not have a camera or a screen; it simply has three types of sensors: an accelerometer (for detecting steps or activities), a gyroscope (for head movements) and an oculographic sensor (eye blinking, eye orientation). The official DataLogger application from JINS MEME is used<fnref n="3" />.

As stated in [FBTE17], the use of smart devices as key elements in an activity monitoring platform has been discussed for many years, in both industrial and research communities. By combining multiple smart devices and building a sensing system, it is possible to interpret physical actions, social interactions, IT environments and so on (e.g. [LL13, HLL+12]). Interested readers can refer to [FLGE16] to get an overview of existing sensing system architectures and solutions.

## 2 Dataset

In July 2017, a dataset has been collected from one of the co-authors from morning until evening for 15 consecutive days. During the data collection, the smartphone has been carried in the pocket for a considerable amount of time. The smartglasses have been used a few hours everyday.

This dataset is provided completely free of charge online<fnref n="4" />. The metrics collected by the different applications and their main parameters are described in Table 2.

<footnote n="3">³https://github.com/jins-meme/ES_R-DataLogger-for-Android</footnote>

<footnote n="4">⁴https://goo.gl/RNx1SX</footnote>
<table><caption>Table 2: Key metrics collected by the sensing systems.</caption><thead><tr><th>Device</th><th>Metric</th><th>Source</th><th>Recording rate</th><th>Comments</th></tr></thead><tbody><tr><td rowspan="4">Watch</td><td>Heart rate</td><td>Optical heart rate sensor</td><td>Event-based</td><td>Heart rate, in beats per minute, provided by the optical heart rate sensor. Each value comes with an accuracy representing the status of the monitor during the reading.</td></tr><tr><td>Step Detector</td><td>Accelerometer</td><td>Event-based</td><td>Indicates whether the user is taking a step or not.</td></tr><tr><td>Step Counter</td><td>Accelerometer</td><td>Event-based</td><td>Number of steps taken by the user, detected by the Android system as a function of the accelerometer.</td></tr><tr><td>Battery</td><td>Android</td><td>5,000 ms</td><td>Battery level.</td></tr><tr><td rowspan="10">Phone</td><td>Ambient sound</td><td>Microphone</td><td>1,000 ms</td><td>Maximum absolute sound amplitude returned by the microphone.</td></tr><tr><td>Ambient light</td><td>Light sensor</td><td>~5,000 ms</td><td>Ambient light level.</td></tr><tr><td>Bluetooth devices</td><td>Network</td><td>5,000 ms</td><td>List and number of Bluetooth devices.</td></tr><tr><td>Wi-Fi APs</td><td>Network</td><td>5,000 ms</td><td>List and number of Wi-Fi Access Points.</td></tr><tr><td>Speed</td><td>GPS</td><td>~30,000 ms</td><td>Travel speed, in m.s<sup>-1</sup>.</td></tr><tr><td>Activity</td><td>Activity Recognition API</td><td>Event-based</td><td>List of activities performed by the user, sorted by the most probable activity first. A confidence is associated with each activity.</td></tr><tr><td>Step Detector</td><td>Accelerometer</td><td>Event-based</td><td>Same as above.</td></tr><tr><td>Step Counter</td><td>Accelerometer</td><td>Event-based</td><td>Same as above.</td></tr><tr><td>Battery</td><td>Android</td><td>5,000 ms</td><td>Same as above.</td></tr><tr><td>Real activity</td><td>aTimeLogger app.</td><td>—</td><td>Activity tags manually selected by the user.</td></tr><tr><td rowspan="3">Glasses</td><td>Acceleration</td><td>Three-axis accelerometer sensor</td><td rowspan="3">10 ms</td><td>Three values describing the current acceleration.</td></tr><tr><td>Angular velocity</td><td>Three-axis gyroscope sensor</td><td>Three values describing the current angular velocity.</td></tr><tr><td>Corneo-retinal standing potential</td><td>Three-point Electrooculography Sensor</td><td>Four values extracted from the electrodes. See the official documentation for more information<sup>5</sup>.</td></tr></tbody></table>
## 3 Research Perspectives

While activity detection from smart-things sensors has largely been understood as quantification of physical activity, a greater intelligence of how humans are managing their time and their personal engagement in their various activities is both possible and desirable. Indeed, our daily lives are incorporating a continuously growing number of interactive systems. Obviously, these systems are bringing a good deal of disruption and distraction [Rod11], or might be disappointing in terms of usefulness and engagement. Solving these issues will make interactive services more context-relevant for users.

Such issues have been pointed out recently by [MLK], but scientific work in this field is still far from being consolidated. Conceptualization efforts have been made to better understand organization of time (e.g. [New94, STB09]). Work is also being undertaken to understand how an activity could be resolved as a combination of smaller chunks (e.g. [CIT16, CTIB15]). Finally, in [FLGE16, FBTE17], we recently proposed that wearable devices along with machine learning techniques might help classifying micro- and macro-activities, thus leading to new ways of understanding human activities and mobility.

The value of releasing the dataset and the data collection system presented in this paper is to allow the scientific community to grow beyond small-scale studies and to get a greater insight into what make a person engaged in an activity. We believe that wearable sensors are opening the way to new perspectives by bridging phenomenological description and quantification of daily and multi-scale behaviors.

## Acknowledgments

This work was performed within the eGLASSES project, which is partially funded by NCBiR, FWF, SNSF, ANR and FNR under the ERA-NET CHIST-ERAII framework.

## References

[CIT16] Carrie J Cai, Shamsi T Iqbal, and Jaime Teevan. Chain Reactions: The Impact of Order on Microtask Chains. In *Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems*, pages 3143–3154, 2016.

[CTIB15] Justin Cheng, Jaime Teevan, Shamsi T. Iqbal, and Michael S. Bernstein. Break It Down: A Comparison of Macro- and Microtasks. *Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI '15*, pages 4061–4064, 2015.

[FBTE17] Sébastien Faye, Walter Bronzi, Ibrahim Tahirou, and Thomas Engel. Characterizing user mobility using mobile sensing systems. *International Journal of Distributed Sensor Networks*, 13(8), 2017.

[FLGE16] Sébastien Faye, Nicolas Louveton, Gabriela Gheorghe, and Thomas Engel. A two-level approach to characterizing human activities from wearable sensor data. *Journal of Wireless Mobile Networks, Ubiquitous Computing, and Dependable Applications (JoWUA)*, September 2016.
[HLL+12] Manhyung Han, Young-Koo Lee, Sungyoung Lee, et al. Comprehensive context rec-
ognizer based on multimodal sensors in a smartphone. *Sensors*, 12(9):12588-12605,
2012.

[LL13] Oscar D Lara and Miguel A Labrador. A survey on human activity recognition using wearable sensors. *IEEE Communications Surveys & Tutorials*, 15(3):1192-1209, 2013.

[MLK] Akhil Mathur, Nicholas D Lane, and Fahim Kawsar. Engagement-Aware Computing: Modelling User Engagement from Mobile Contexts.

[New94] Allen Newell. *Unified theories of cognition*. Harvard University Press, 1994.

[Rod11] Claudia Roda. *Human attention in digital environments*. Cambridge University Press, 2011.

[STB09] Dario D Salvucci, Niels A Taatgen, and Jelmer P Borst. Toward a unified theory of the multitasking continuum: from concurrent performance to task switching, interruption, and resumption. In *Proceedings of the SIGCHI conference on human factors in computing systems*, pages 1819–1828. ACM, 2009.