# User Recognition Based on Daily Actigraphy Patterns

Enrique Garcia-Ceja, Brice Morin

► To cite this version:

Enrique Garcia-Ceja, Brice Morin. User Recognition Based on Daily Actigraphy Patterns. 13th IFIP International Conference on Trust Management (IFIPTM), Jul 2019, Copenhagen, Denmark. pp.73-80, 10.1007/978-3-030-33716-2_6. hal-03182611

HAL Id: hal-03182611

https://inria.hal.science/hal-03182611

Submitted on 26 Mar 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# User Recognition Based on Daily Actigraphy Patterns

Enrique Garcia-Ceja and Brice Morin

SINTEF Digital, Oslo, Norway
{enrique.garcia-ceja,brice.morin}@sintef.no

**Abstract.** The use of inertial sensors such as accelerometers and gyroscopes, which are now often embedded in many wearable devices, has gained attention for their applicability in user authentication applications as an alternative to PINs, passwords, biometric signatures, etc. Previous works have shown that it is possible to authenticate users based on fine-grained kinematic behavior profiles like gait, hand gestures and physical activities. In this work we explore the use of actigraphy data for user recognition based on daily patterns as opposed to fine-grained motion. One of the advantages of the former, is that it does not require to perform specific movements, thus, easing the training and calibration stages. In this work we extracted daily patterns from an actigraphy device and used a random forest classifier and a majority voting approach to perform the user classification. We used a public available dataset collected by 55 participants and we achieved a true positive rate of 0.64, a true negative rate of 0.99 and a balanced accuracy of 0.81.

**Keywords:** user recognition · accelerometer · authentication · machine learning.

## 1 Introduction

The use of inertial sensors (accelerometers, gyroscopes) for continuous monitoring has become very common due to their ubiquity. Such sensors are increasingly embedded in many wearable devices such as smartphones, smart-watches, fitness bracelets, actigraphy devices and so on. These sensors have been used to monitor physical activities [14], sport activities [16], mental health [10], social interactions [8], to name a few. Recently, the use of inertial units has been explored for user identification and user authentication applications [1, 15]. As pointed out by Yang et al. [21], current authentication schemes for wearable devices are often impractical. For example, given the small screen of a smart-watch, the use of a PIN is cumbersome. Furthermore, remembering PINs and passwords for different programs is difficult. An appealing alternative is the use of kinetic patterns based on user movement data. Previous works have demonstrated the use of wrist worn devices for user authentication [1, 21]. The majority of those works build a biometric profile based on fine grained motion patterns. For example, Kwapisz et al. [15] performed biometric user identification and authentication
based on physical activity patterns captured with a smartphone. The type of activities they used were walking, jogging and climbing stairs.

One of the limitations of those approaches is that they require high sam-
pling rates in order to capture detailed movement patterns. This translates into
higher computational and storage demands, thus, reducing battery lifetime. Fur-
thermore, in order to build biometric profiles, the users need to perform specific
movements or activities to train and use the system. The process of collecting
this calibration data is generally tedious and time consuming, which hinders the
applicability of the systems. In this work we perform user recognition using mo-
tion data collected with actigraphy devices based on daily activity patterns. The
advantage of this approach is that it can be used with reduced sampling rates
since it is based on higher level patterns as opposed to previous works that use
fine grained patterns. Furthermore, this approach is more flexible since it does
not require its users to follow detailed instructions such as performing specific
movements or activities. Users just need to wear the device and perform their
daily activities as usual. To test our approach, we used a dataset collected by 55
participants, which is one of the biggest ones used for user recognition based on
wearable sensors. Our results for user recognition showed that there is potential
to use daily patterns for user identification.

The rest of this paper is organized as follows: Section 2 presents the back-
ground and related works. Section 3 describes the details of the used dataset.
Section 4 explains the preprocessing, feature extraction and user recognition ap-
proach. In section 5 we present the experiments and results. Finally, in section 6
we draw conclusions and propose possible future directions.

## 2 Background

Advances in sensor miniaturization enabled new capabilities for wearable devices. Specifically, inertial sensors have become very popular and are now embedded in many devices. Inertial sensors capture acceleration forces and orientation measurements. Within the context of smartphones, these sensors were initially intended for some specific purposes such as finding the phone's orientation so the screen can be rotated accordingly, and as a means of interaction for games and applications. Later, these sensors started to be used to infer contextual information about the device and user. For example, it is now common to have pedometer smartphone applications that are capable of counting the number of steps [11]. Inertial sensors in a smartphone can also be used for indoor localization using odometry techniques [20]. Inertial sensors have even been used for mood recognition in combination with physiological sensors [22].

Recently, it has been shown that the data produced by inertial sensors can
also be used to generate behavior profiles for user authentication applications
[1]. For example, Buroiro et al. [4] used the accelerometer and gyroscope from
a smart-watch to perform user authentication based on finger-snapping. They
evaluated three approaches: a Bayesian network, a multilayer perceptron and
a random forest classifier. Their best results were obtained with the multilayer
perceptron with a true acceptance rate of 66.14%. In another work, the same authors used a similar smartwatch setting but instead of recording finger-snapping, the users were asked to write their names in the air [3]. They achieved a true acceptance rate of 69.55% with a one-class multilayer perceptron trained with 10 samples. Some previous works have also used touchscreen interactions to capture behavior. For example, Jain & Kanhangad [12] proposed a method for user verification based on touchscreen interactions using common gestures such as scroll, swipe, zoom, tap, etc. Their method achieved an equal error rate of 0.31% on a relatively large dataset of 104 users. Another recent trend is the use of gait analysis using smartphones for user authentication [18]. Mufandaidza et al. [17] used a combination of dynamic time warping and a feed forward neural network to analyze walking patterns and achieved a true positive rate of 0.73, i.e., the system correctly identifies the authorized user 73% of the time.

Another potential source of information that could be used for user authentication is *actigraphy data*. Actigraphy is a method to monitor user activity levels using inertial sensors. Actigraphy devices are commonly worn on the wrist and they have less computational power compared to a smartphone or smartwatch but their battery lasts for longer periods of time which makes them suitable for long period studies, specially, in the medical field. These devices record activity levels based on acceleration and can be used to monitor sleep patterns [19], bipolar disorder patients [13], differentiate between bipolar disorder and attentiondeficit/hyperactivity disorder [5], to name a few. In this work we explore the potential use of actigraphy data for user authentication. Instead of looking at fine grained movements, we analyze daily activity patterns in order to recognize users.

## 3 Dataset

For our experiments, we used the *DEPRESJON dataset* which is publicly available [9]. This dataset was collected by 55 participants wearing an actigraphy watch (Actiwatch, Cambridge Neurotechnology Ltd, England, model AW4) on their right wrist. On average, the participants wore the device for 20 continuous days. The device captures activity levels using a piezoelectric accelerometer that captures the amount and duration of movement in all directions. The sampling frequency is 32Hz and movements over 0.05 g are recorded. A corresponding voltage is produced and is stored as an activity count in the memory unit of the device. The number of counts is proportional to the intensity of the movement. The accumulated activity counts are continuously recorded in one minute intervals. 23 of the participants were depressed patients and 32 were non-depressed controls. In this study, we did not take into account the participants' condition (depressed/non-depressed) since the main objective is to recognize the user and not the condition. Fig. 1 shows an example of a portion of the collected data by one of the participants. The first column is the time of the day, the second column is the date and the third column contains the activity count.
<figure><img src="image_2.png" /><figcaption>Fig. 1. Example data from one of the participants.</figcaption></figure>

## 4 Feature extraction and user recognition

From the raw activity counts, 51 features were extracted in a per day basis. The first set of features are the mean activity counts for each hour of the day (f1-f24). We also computed the standard deviation for each hour of the day (f25-f48). Finally, we computed 3 more features. The overall activity *mean*, *median* and *standard deviation* across all hours of the day. After that, the features were normalized between 0 and 1. Missing values were set to -1 and the first day from each user was excluded since it usually began at midday, thus, having a lot of missing values before that. After feature extraction, the total number of instances (feature vectors) were 1089, each, representing a single day of data.

For the recognition part, we trained a random forest classifier [2] since they have been shown to produce good overall results on many tasks [6]. The target label was the user id. A subset of the data from each user was used for training and an independent subset was used for testing (more details in section 5). To get the final prediction, a majority vote approach was used. That is, for each user, predict the label for each of her/his testing days and output the most common label (user id).

## 5 Experiments and results

For our experiments, we selected at random 50% of the users' days for training and the remaining for testing and we repeated this procedure 50 times to account for variability. The number of trees for the random forest was set to 500. Table 1 shows the resulting performance metrics. The overall obtained accuracy was 0.64 whereas the balanced accuracy was 0.82. The later is the average of the True Positive Rate (TPR) and the True Negative Rate (TNR). Figure 2 shows the resulting confusion matrix for the 55 users' classification. The antidiagonal shows the TPR. The white color represents a low value (0.0) whereas more solid colors represent higher values with a maximum of 1.0. Here, it can be seen that several users have high TPRs. Six users have a TPR of 1.0. On the other hand, some users have very low TPRs. Two of them have a TPR of 0.02. Although these results are far from perfect, they look promising given that we have data just for 1
minute epochs compared to previous works where they have several samples per second. All in all, these results show potential for further improvement and the possibility to use daily activity patterns as means to perform user authentication.

<table><caption>**Table 1.** User recognition results</caption><thead><tr><td>Metric</td><td>Value</td></tr></thead><tbody><tr><td>Accuracy</td><td>0.64</td></tr><tr><td>True Positive Rate (TPR)</td><td>0.64</td></tr><tr><td>True Negative Rate (TNR)</td><td>0.99</td></tr><tr><td>Balanced accuracy</td><td>0.81</td></tr></tbody></table>

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Confusion matrix. The antidiagonal shows the true positive rates</figcaption></figure>
Figure 3 shows the feature importance as obtained by the random forest classifier using the out-of-bag data based on mean decrease accuracy for the top 20 features. It can be seen that the two most important variables are the standard deviation (sd) and the mean. These two features capture patterns across all hours of the day. The next top important feature is f7 which represents the mean activity counts between 6:00 and 7:00 a.m. Based on this, the most important features are based on the captured patterns across all hours of the day and the mean activity counts for each individual hour of the day. Within the top 20 features, three of them (f31, f33 and f40) are based on the standard deviation of the individual hours of the day.

<figure><img src="image_4.png" /><figcaption>Fig. 3. Random forest feature importance.</figcaption></figure>

# 6 Conclusions

In this work we used actigraphy data to recognize users based on their daily activity patterns. The activity patterns were characterized by extracting statistical features from the activity counts. Finally, a random forest classifier and majority voting were used to get the final predictions. The user classification true positive rate was 0.64. From the feature importance analysis, we found that the most significant features are the ones that capture overall patterns as opposed to hourly patterns. These results are still far from optimal but they show potential for improvement or to be used in combination with other techniques which could boost the system performance. One of the limitations of this approach is that it can suffer from within-user variances over time, i.e., users' patterns may change. Situations in which this can happen are when moving to another city, starting a new job, etc. This is known as concept drift [7] and we will explore
possible solutions for future work. Another future direction is the extraction of
more meaningful features, e.g., differentiating between morning, evening, and
night patterns. Furthermore, the use of deep learning methods to find features
automatically from the raw data is another possible future direction.

**Acknowledgements** Research leading to these results has received funding from the EU ECSEL Joint Undertaking under grant agreement no 737459 (project Productive4.0) and from the Research Council of Norway.

References

1. Al-Naffakh, N., Clarke, N., Li, F.: Continuous User Authentication Using Smartwatch Motion Sensor Data. In: Gal-Oz, N., Lewis, P.R. (eds.) Trust Management XII. pp. 15–28. Springer International Publishing, Cham (2018)

2. Breiman, L.: Random forests. Machine learning **45**(1), 5–32 (2001)

3. Buriro, A., Acker, R.V., Crispo, B., Mahboob, A.: Airsign: A gesture-based smartwatch user authentication. In: 2018 International Carnahan Conference on Security Technology (ICCST). pp. 1–5 (Oct 2018). https://doi.org/10.1109/CCST.2018.8585571

4. Buriro, A., Crispo, B., Eskandri, M., Gupta, S., Mahboob, A., Van Acker, R.: SNA-PAUTH: A Gesture-Based Unobtrusive Smartwatch User Authentication Scheme. In: International Workshop on Emerging Technologies for Authorization and Authentication. pp. 30–37. Springer (2018)

5. Faedda, G.L., Ohashi, K., Hernandez, M., McGreenery, C.E., Grant, M.C., Baroni, A., Polcari, A., Teicher, M.H.: Actigraph measures discriminate pediatric bipo-lar disorder from attention-deficit/hyperactivity disorder and typically developing controls. Journal of Child Psychology and Psychiatry **57**(6), 706–716 (2016)

6. Fernández-Delgado, M., Cernadas, E., Barro, S., Amorim, D.: Do we need hundreds of classifiers to solve real world classification problems? The Journal of Machine Learning Research **15**(1), 3133–3181 (2014)

7. Gama, J., liobait, I., Bifet, A., Pechenizkiy, M., Bouchachia, A.: A survey on con-cept drift adaptation **46**(4), 44 (2014)

8. Garcia-Ceja, E., Osmani, V., Maxhuni, A., Mayora, O.: Detecting walking in syn-chrony through smartphone accelerometer and wi-fi traces. In: Aarts, E., de Ruyter, B., Markopoulos, P., van Loenen, E., Wichert, R., Schouten, B., Terken, J., Van Kranenburg, R., Den Ouden, E., O’Hare, G. (eds.) Ambient Intelligence. pp. 33–46. Springer International Publishing, Cham (2014)

9. Garcia-Ceja, E., Riegler, M., Jakobsen, P., rresen, J.T., Nordgreen, T., Oedegaard, K.J., Fasmer, O.B.: Depresjon: A motor activity database of depression episodes in unipolar and bipolar patients. In: Proceedings of the 9th ACM on Multimedia Systems Conference. pp. 472–477. MMSys’18, ACM, New York, NY, USA (2018). https://doi.org/10.1145/3204949.3208125, http://doi.acm.org/10.1145/3204949.3208125

10. Garcia-Ceja, E., Riegler, M., Nordgreen, T., Jakobsen, P., Oedegaard, K.J., Torresen, J.: Mental health monitoring with multimodal sensing and machine learning: A survey. Pervasive and Mobile Computing **51**, 1 – 26 (2018). https://doi.org/https://doi.org/10.1016/j.pmcj.2018.09.003, http://www.sciencedirect.com/science/article/pii/S1574119217305692
11. Gowda, A.K.S.N., Babu, S.R., Sekaran, D.C.: Umoisp: Usage mode and orientation invariant smartphone pedometer. IEEE Sensors Journal **17**(3), 869–881 (Feb 2017). https://doi.org/10.1109/JSEN.2016.2635691

12. Jain, A., Kanhangad, V.: Exploring orientation and accelerometer sensor data for personal authentication in smartphones using touchscreen gestures. Pattern Recognition Letters **68**, 351 – 360 (2015), special Issue on Soft Biometrics

13. Jones, S.H., Hare, D.J., Evershed, K.: Actigraphic assessment of circadian activity and sleep patterns in bipolar disorder. Bipolar disorders **7**(2), 176–186 (2005)

14. Kwapisz, J.R., Weiss, G.M., Moore, S.A.: Activity recognition using cell phone accelerometers. SIGKDD Explor. Newsl. **12**(2), 74–82 (Mar 2011). https://doi.org/10.1145/1964897.1964918; http://doi.acm.org/10.1145/1964897.1964918

15. Kwapisz, J., Weiss, G., Moore, S.: Cell phone-based biometric identification. In: Biometrics: Theory Applications and Systems (BTAS), 2010 Fourth IEEE International Conference on. pp. 1–7 (Sep 2010). https://doi.org/10.1109/BTAS.2010.5634532

16. Mitchell, E., Monaghan, D., O'Connor, N.E.: Classification of sporting activities using smartphone accelerometers. Sensors **13**(4), 5317–5337 (2013)

17. Mufandaidza, M.P., Ramotsoela, T.D., Hancke, G.P.: Continuous user authentication in smartphones using gait analysis. In: IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society. pp. 4656–4661 (Oct 2018). https://doi.org/10.1109/IECON.2018.8591193

18. Patel, V.M., Chellappa, R., Chandra, D., Barbello, B.: Continuous user authentication on mobile devices: Recent progress and remaining challenges. IEEE Signal Processing Magazine **33**(4), 49–61 (July 2016). https://doi.org/10.1109/MSP.2016.2555335

19. Sadeh, A., Acebo, C.: The role of actigraphy in sleep medicine. Sleep medicine reviews **6**(2), 113–124 (2002)

20. Solin, A., Cortes, S., Rahtu, E., Kannala, J.: Inertial odometry on handheld smartphones. In: 2018 21st International Conference on Information Fusion (FUSION). pp. 1–5 (July 2018). https://doi.org/10.23919/ICIF.2018.8455482

21. Yang, J., Li, Y., Xie, M.: Motionauth: Motion-based authentication for wrist worn smart devices. In: 2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops). pp. 550–555 (March 2015). https://doi.org/10.1109/PERCOMW.2015.7134097

22. Zenonos, A., Khan, A., Kalogridis, G., Vatsikas, S., Lewis, T., Sooriyabandara, M.: HealthyOffice: Mood recognition at work using smartphones and wearable sensors. In: 2016 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops). pp. 1–6 (Mar 2016). https://doi.org/10.1109/PERCOMW.2016.7457166