# Real-Time Machine Learning for Human Activities Recognition Based on Wrist-Worn Wearable Devices
## 3 Proposed System
### Data Preprocessing App: Movement Data Plot Images Generation  
In order to simplify the classic time-series data classification task, based on the sensor data, we can handle the human activity recognition task as an image classification one. The numeric sensor data from the smartwatch are used to transform a series of values, consisting of a data window, into a single data image. This way we can provide a visual representation of a data chunk that is easier for a human to analyze and interpret the data manually. Each image can show certain characteristics for that particular activity type and different plot styles can be used. The main preprocessing application is written in C# 6 and uses the \"ScottPlot\" plotting library for.NET. The preprocessing application is written as a.NET console application.  
In order to reduce the amount of data generated in the preprocessing phase, only certain activity types and users are selected for processing. The following activities are selected for processing from the source dataset: walking, jogging, stairs, sitting, standing,  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Activity Category No.** & **Activity Category** & **Activities** \\ \hline
1 &
\begin{tabular}{c} Non-hand-oriented \\ activities \\ \end{tabular} & walking, jogging, stairs, standing, \\
2 & \
\begin{tabular}{c} General hand-oriented \\ activities \\ \end{tabular} &
\begin{tabular}{c} dribbling, playing catch, typing, \\ writing, clapping, brushing teeth, \\ folding clothes \\ \end{tabular} \\
3 & \
\begin{tabular}{c} Eating hand-oriented \\ activities \\ \end{tabular} &
\begin{tabular}{c} eating pasta, eating soup, eating a \\ sandwich, eating chips, drinking \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table}
Table 1: Dataset activity categories.  
typing, and brushing teeth. To further decrease the number of generated plot images, the users that performed the activities are also filtered and only the first five users are used.  
The \"ScottPlot\" 4.1.59 plotting library was chosen as it is open-source, free, and provided under the permissive MIT license. This library makes displaying and saving images easy and fast. As mentioned in [32], there are multiple platforms of.NET that are supported by this library (including.NET core) which means that it can even be used in Azure cloud directly, under the form of Azure function.  
Based on the chosen plot image type from the ScottPlot library, we have two main scenarios:  
* Scatter plot images;
* Population images.  
The generated plot images have a different width and height based on the chosen scenario, as shown in Table 2. The window size chosen for overlapping scenarios is 40 records.  
An example of generated scatter plot images with overlapping scatter plots with a data sequence for walking activity is shown in Figure 2.  
An example of generated scatter plot images with non-overlapping scatter plots with a data sequence for the walking activity is shown in Figure 3.  
\begin{table}
\begin{tabular}{c c c} \hline
**Scenario** & **Image Width** & **Image Height** \\ \hline Scatter plot images with overlapping scatter plots & 600 & 400 \\ Scatter plot images with non-overlapping scatter plots & 500 & 300 \\ Population images & 500 & 300 \\ \hline \end{tabular}
\end{table}
Table 2: Generated image output dimensions.  
Figure 2: The scatter plot image for a data sequence with overlapping windows for an activity using the.NET plotting library.  
An example of generated population images with the \"BarMeanStDev\" option is shown in Figure 4.