DOI: 10.5281/zenodo.4420038
title: Wrist-mounted IMU data towards the investigation of free-living human eating behavior - the Free-living Food Intake Cycle (FreeFIC) dataset

description: <strong>Introduction</strong> The Free-living Food Intake Cycle (FreeFIC) dataset was created by the Multimedia Understanding Group towards the investigation of <em>in-the-wild</em> eating behavior. This is achieved by recording the subjects’ meals as a small part part of their everyday life, unscripted, activities. The FreeFIC dataset contains the \(3D\) acceleration and orientation velocity signals (\(6\) DoF) from \(22\) in-the-wild sessions provided by \(12\) unique subjects. All sessions were recorded using a commercial smartwatch (\(6\) using the Huawei Watch 2™ and the MobVoi TicWatch™ for the rest) while the participants performed their everyday activities. In addition, FreeFIC also contains the start and end moments of each meal session as reported by the participants. <strong>Description</strong> FreeFIC includes \(22\) in-the-wild sessions that belong to \(12\) unique subjects. Participants were instructed to wear the smartwatch to the hand of their preference well ahead before any meal and continue to wear it throughout the day until the battery is depleted. In addition, we followed a self-report labeling model, meaning that the ground truth is provided from the participant by documenting the start and end moments of their meals to the best of their abilities as well as the hand they wear the smartwatch on. The total duration of the \(22\) recordings sums up to \(112.71\) hours, with a mean duration of \(5.12\) hours. Additional data statistics can be obtained by executing the provided python script <em>stats_dataset.py</em>. Furthermore, the accompanying python script <em>viz_dataset.py </em>will visualize the IMU signals and ground truth intervals for each of the recordings. Information on how to execute the Python scripts can be found below. <pre><code># The script(s) and the pickle file must be located in the same directory. # Tested with Python 3.6.4 # Requirements: Numpy, Pickle and Matplotlib # Calculate and echo dataset statistics $ python stats_dataset.py # Visualize signals and ground truth $ python viz_dataset.py</code></pre> FreeFIC is also tightly related to Food Intake Cycle (FIC), a dataset we created in order to investigate the <em>in-meal</em> eating behavior. More information about FIC can be found here and here. <strong>Publications</strong> If you plan to use the FreeFIC dataset or any of the resources found in this page, please cite our work: <pre><code>@article{kyritsis2020data, title={A Data Driven End-to-end Approach for In-the-wild Monitoring of Eating Behavior Using Smartwatches}, author={Kyritsis, Konstantinos and Diou, Christos and Delopoulos, Anastasios}, journal={IEEE Journal of Biomedical and Health Informatics}, year={2020}, publisher={IEEE}}</code></pre> <pre><code>@inproceedings{kyritsis2017automated, title={Detecting Meals In the Wild Using the Inertial Data of a Typical Smartwatch}, author={Kyritsis, Konstantinos and Diou, Christos and Delopoulos, Anastasios}, booktitle={2019 41th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)}, year={2019}, organization={IEEE}} </code></pre> <strong>Technical details</strong> We provide the FreeFIC dataset as a pickle. The file can be loaded using Python in the following way: <pre><code class="language-python">import pickle as pkl import numpy as np with open('./FreeFIC_FreeFIC-heldout.pkl','rb') as fh: dataset = pkl.load(fh)</code></pre> The dataset variable in the snipet above is a dictionary with \(5\) keys. Namely: 'subject_id' 'session_id' 'signals_raw' 'signals_proc' 'meal_gt' The contents under a specific key can be obtained by: <pre><code class="language-python">sub = dataset['subject_id'] # for the subject id ses = dataset['session_id'] # for the session id raw = dataset['signals_raw'] # for the raw IMU signals proc = dataset['signals_proc'] # for the processed IMU signals gt = dataset['meal_gt'] # for the meal ground truth </code></pre> The <em>sub</em>, <em>ses</em>, <em>raw</em>, <em>proc </em>and <em>gt </em>variables in the snipet above are lists with a length equal to \(22\). Elements across all lists are aligned; e.g., the \(3\)rd element of the list under the 'session_id' key corresponds to the \(3\)rd element of the list under the 'signals_proc' key. <em>sub</em>: list<br> Each element of the sub list is a scalar (integer) that corresponds to the unique identifier of the subject that can take the following values: \([1, 2, 3, 4, 13, 14, 15, 16, 17, 18, 19, 20]\). It should be emphasized that the subjects with ids \(15, 16, 17, 18, 19\) and \(20\) belong to the held-out part of the FreeFIC dataset (more information can be found in \( \)the publication titled "A Data Driven End-to-end Approach for In-the-wild Monitoring of Eating Behavior Using Smartwatches" by Kyritsis <em>et al).</em> Moreover, the subject identifier in FreeFIC is in-line with the subject identifier in the FIC dataset (more info here and here); i.e., FIC’s subject with id equal to \(2\) is the same person as FreeFIC’s subject with id equal to \(2\). <em>ses: </em>list<br> Each element of this list is a scalar (integer) that corresponds to the unique identifier of the session that can range between \(1\) and \(5\). It should be noted that not all subjects have the same number of sessions. <em>raw</em>: list<br> Each element of this list is dictionary with the 'acc' and 'gyr' keys.<br> The data under the 'acc' key is a \(N_{acc} \times 4\) numpy.ndarray that contains the timestamps in seconds (first column) and the \(3D\) raw accelerometer measurements in \(g\) (second, third and forth columns - representing the \(x, y \) and \(z\) axis, respectively). The data under the 'gyr' key is a \(N_{gyr} \times 4\) numpy.ndarray that contains the timestamps in seconds (first column) and the \(3D\) raw gyroscope measurements in \({degrees}/{second}\)(second, third and forth columns - representing the \(x, y \) and \(z\) axis, respectively). All sensor streams are transformed in such a way that reflects all participants wearing the smartwatch at the same hand with the same orientation, thusly achieving data uniformity. This transformation is in par with the signals in the FIC dataset (more info here and here). Finally, the length of the raw accelerometer and gyroscope numpy.ndarrays is different \((N_{acc} \neq N_{gyr})\). This behavior is predictable and is caused by the Android platform. <em>proc: </em>list<br> Each element of this list is an \(M\times7\) numpy.ndarray that contains the timestamps, \(3D\) accelerometer and gyroscope measurements for each meal. Specifically, the first column contains the timestamps in seconds, the second, third and forth columns contain the <em>\(x,y\)</em> and \(z\) accelerometer values in \(g\)<strong> </strong>and the fifth, sixth and seventh columns contain the <em>\(x,y\)</em> and \(z\) gyroscope values in \({degrees}/{second}\). Unlike elements in the <em>raw </em>list, processed measurements (in the <em>proc</em> list) have a constant sampling rate of \(100\) Hz and the accelerometer/gyroscope measurements are aligned with each other. In addition, all sensor streams are transformed in such a way that reflects all participants wearing the smartwatch at the same hand with the same orientation, thusly achieving data uniformity. This transformation is in par with the signals in the FIC dataset (more info here and here). <em>No other preprocessing is performed on the data</em>; e.g., the acceleration component due to the Earth's gravitational field is present at the processed acceleration measurements. The potential researcher can consult the article "A Data Driven End-to-end Approach for In-the-wild Monitoring of Eating Behavior Using Smartwatches" by Kyritsis <em>et al. </em>on how to further preprocess the IMU signals (i.e., smooth and remove the gravitational component). <em>meal_gt: </em>list<br> Each element of this list is a<strong> \(K\times2\)</strong> matrix. Each row represents the meal intervals for the specific in-the-wild session. The first column contains the timestamps of the meal start moments<strong> </strong>whereas the second one the timestamps of the meal end moments. All timestamps are in seconds. The number of meals \(K\) varies across recordings (e.g., a recording exist where a participant consumed two meals). <strong>Ethics and funding</strong> Informed consent, including permission for third-party access to anonymised data, was obtained from all subjects prior to their engagement in the study. The work has received funding from the European Union's Horizon 2020 research and innovation programme under Grant Agreement No 727688 - BigO: Big data against childhood obesity. <strong>Contact</strong> Any inquiries regarding the FreeFIC dataset should be addressed to: Dr. Konstantinos KYRITSIS Multimedia Understanding Group (MUG)<br> Department of Electrical & Computer Engineering<br> Aristotle University of Thessaloniki<br> University Campus, Building C, 3rd floor<br> Thessaloniki, Greece, GR54124 Tel: +30 2310 996359, 996365 <br> Fax: +30 2310 996398<br> E-mail: kokirits [at] mug [dot] ee [dot] auth [dot] gr