DOI: 10.5281/zenodo.4507450
title: Wrist-mounted IMU data towards the investigation of free-living smoking behavior - the Smoking Event Detection (SED) and Free-living Smoking Event Detection (SED-FL) datasets

description: <strong>Introduction</strong> The Smoking Event Detection (SED) and the Free-living Smoking Event Detection (SED-FL) datasets were created by the Multimedia Understanding Group towards the investigation of smoking behavior, both while smoking and <em>in-the-wild</em>. Both datasets contain the triaxial acceleration and orientation velocity signals ( DoF) that originate from a commercial smartwatch (Mobvoi TicWatch E™). The SED dataset consists of \(20\) smoking sessions provided by \(11\) unique subjects, while the SED-FL dataset contains \(10\) all-day recordings provided by \(7\) unique subjects. In addition, the start and end moments of each puff cycle are annotated throughout the SED dataset. <strong>Description </strong> <em>SED</em> A total of \(11\) subjects were recorded while smoking a cigarette at interior or exterior areas. The total duration of the \(20\) sessions sums up to \(161\) minutes, with a mean duration of \(8.08\) minutes. Each participant was free to smoke naturally, with the only limitation being to not swap the cigarette between hands during the smoking session. Prior to the recording, the participant was asked to wear the smartwatch to the hand that he typically uses in his everyday life to smoke. A camera was already set facing the participant, including at least the whole length of the arms in its field of view. The purpose of video recording was to obtain ground truth information for each of the puff cycles that occur during the smoking session. Participants were also asked to perform a clapping hand movement both at the start and end of the meal, for synchronization purposes (as this movement is distinctive in the accelerometer signal). No other instructions were given to the participants. It should be noted that the SED dataset does not contain instances of electronic cigarettes (also known as vaping devices), or heated tobacco products. <em>SED-FL</em> SED-FL includes \(10\) <em>in-the-wild</em> sessions that belong to \(7\) unique subjects. This is achieved by recording the subjects’ meals as a small part part of their everyday life, unscripted, activities. Participants were instructed to wear the smartwatch to the hand of their preference well ahead before any smoking session and continue to wear it throughout the day until the battery is depleted. In addition, we followed a self-report labeling model, meaning that the ground truth is provided from the participant by documenting the start and end moments of their smoking sessions to the best of their abilities as well as the hand they wear the smartwatch on. The total duration of the recordings sums up to \(78.3\) hours, with a mean duration of \(7.83\) hours. For both datasets, the accompanying Python script <em>read_dataset.py </em>will visualize the IMU signals and ground truth for each of the recordings. Information on how to execute the Python scripts can be found below. <pre><code class="language-python"># The script and the daataset's pickle file must be located in the same directory. # Tested with Python 3.6.4 # Requirements: Pandas, Pickle and Matplotlib # Visualize signals and ground truth python read_datasets.py</code></pre> <strong>Annotation</strong> For all recordings, we annotated the start and end points for each puff cycle (i.e., smoking gesture). The annotation process was performed in such a way that the start and end times of each smoking gesture do not overlap each other. <strong>Technical details</strong> <em>SED</em> We provide the SED dataset as a pickle. The file can be loaded using Python in the following way: <pre><code class="language-python">import pickle as pkl import pandas as pd with open('./SED.pkl','rb') as fh: dataset = pkl.load(fh)</code></pre> The <em>dataset </em>variable in the snippet above is a dictionary with keys, each corresponding to a unique subject (numbered from to ). It should be mentioned that the subject identifier in SED is in-line with the subject identifier in the SED-FL dataset; i.e., SED’s subject with id equal to is the same person as SED-FL’s subject with id equal to . The content of a <em>dataset </em>‘s subject is a list with length equal to corresponding subject’s number of recorded smoking sessions. For example, assuming that subject has recorded smoking sessions, the command: <pre><code class="language-python">sessions = dataset['8']</code></pre> would yield a list of length equal to . Each member of the list is a <em>Pandas DataFrame</em> with dimensions , where is the length of the recording. The columns of a session’s DataFrame are: 'T': The timestamps in seconds 'AccX': The accelerometer measurements for the axis in \(m/s^2\) 'AccY': The accelerometer measurements for the axis in \(m/s^2\) 'AccZ': The accelerometer measurements for the axis in \(m/s^2\) 'GyrX': The gyroscope measurements for the axis in \(rad/s\) 'GyrY': The gyroscope measurements for the axis in \(rad/s\) 'GyrZ': The gyroscope measurements for the axis in \(rad/s\) 'GT': The manually annotated ground truth for puff cycles The contents of this DataFrame are essentially the accelerometer and gyroscope sensor streams, resampled at a constant sampling rate of Hz and aligned with each other and with their puff cycle ground truth. All sensor streams are transformed in such a way that reflects all participants wearing the smartwatch at the same hand with the same orientation, thusly achieving data uniformity. This transformation is in par with the signals in the SED-FL dataset. The ground truth is a signal with value during puff cycles, and elsewhere. No other preprocessing is performed on the data; e.g., the acceleration component due to the Earth's gravitational field is present at the processed acceleration measurements. The potential researcher can consult the article "Modeling Wrist Micromovements to Measure In-Meal Eating Behavior from Inertial Sensor Data" by Kyritsis <em>et al. </em>on how to further preprocess the IMU signals (i.e., smooth and remove the gravitational component). <em>SED-FL</em> Similar to SED, we provide the SED-FL dataset as a pickle. The file can be loaded using Python in the following way: <pre><code class="language-python">import pickle as pkl import pandas as pd with open('./SED-FL.pkl','rb') as fh: dataset = pkl.load(fh)</code></pre> The <em>dataset </em>variable in the snippet above is a dictionary with keys, each corresponding to a unique subject. It should be mentioned that the subject identifier in SED-FL is in-line with the subject identifier in the SED dataset; i.e., SED-FL’s subject with id equal to is the same person as SED’s subject with id equal to . The content of a <em>dataset </em>‘s subject is a list with length equal to corresponding subject’s number of recorded daily sessions. For example, assuming that subject has recorded 2 daily sessions, the command: <pre><code class="language-python">sessions = dataset['8']</code></pre> would yield a list of length equal to \(2\). Each member of the list is a Pandas DataFrame with dimensions \(M \times 8\), where \(M\) is the length of the recording. The columns of a session’s DataFrame are <em>exactly </em>the same with the ones in the SED dataset. However, the 'GT' column contains ground truth that relates with the smoking sessions during the day (instead of puff cycles in SED). The contents of this DataFrame are essentially the accelerometer and gyroscope sensor streams, resampled at a constant sampling rate of \(50\) Hz and aligned with each other and with their smoking session ground truth. All sensor streams are transformed in such a way that reflects all participants wearing the smartwatch at the same hand with the same orientation, thusly achieving data uniformity. This transformation is in par with the signals in the SED dataset. The ground truth is a signal with value \(+1\) during smoking sessions, and \(-1\) elsewhere. No other preprocessing is performed on the data; e.g., the acceleration component due to the Earth's gravitational field is present at the processed acceleration measurements. The potential researcher can consult the article "Modeling Wrist Micromovements to Measure In-Meal Eating Behavior from Inertial Sensor Data" by Kyritsis <em>et al. </em>on how to further preprocess the IMU signals (i.e., smooth and remove the gravitational component). <strong>Ethics and funding</strong> Informed consent, including permission for third-party access to anonymized data, was obtained from all subjects prior to their engagement in the study. The work leading to these results has received funding<br> from the EU Commission under Grant Agreement No. 965231, the REBECCA project (H2020). <strong>Contact</strong> Any inquiries regarding the SED and SED-FL datasets should be addressed to: Mr. Konstantinos KYRITSIS (Electrical & Computer Engineer, PhD candidate) Multimedia Understanding Group (MUG)<br> Department of Electrical & Computer Engineering<br> Aristotle University of Thessaloniki<br> University Campus, Building C, 3rd floor<br> Thessaloniki, Greece, GR54124 Tel: +30 2310 996359, 996365 <br> Fax: +30 2310 996398<br> E-mail: kokirits [at] mug [dot] ee [dot] auth [dot] gr