# Real-Time Physical Activity Recognition on Smart Mobile Devices Using Convolutional Neural Networks
## 3 Materials and Methods
### Data Pre-Processing and Feature Generation  
The proposed implementation combines the automatic feature creation of convolutional layers with statistical features, which are then fed into a fully-connected layer. Before feeding the raw data into the network, it is necessary to apply pre-processing and create statistical features.  
The accelerometer data of each channel are first divided in time windows, in order to exploit the temporal information and periodicity of the signals. If the time duration of each window is \(N\) seconds, this means that there are \(N_{w}=20\cdot N\) data points in each window, since the frequency of the accelerometer data collection is 20 Hz. Furthermore, the step between time windows is constant and is 20 data points or 1 s. Consequently, there are 3 vectors \(\mathbf{a}_{x,y,z}\), one for each axis.  
Convolutional layers create a representation of the raw data, but the extracted features are local, so the global characteristics of the time series also need to be encoded. This is achieved by creating 40 statistical features for each window, which were proposed by the creators of the dataset themselves [26]. The features are described below, with the number of features generated for each feature type noted in brackets:  
* Average [3]: \(M_{i}=\frac{1}{N_{w}}\sum_{j=1}^{N_{w}}a_{i}[j]\), for \(i=x,y,z\)
* Standard deviation [3]: \(S_{i}=\sqrt{\frac{1}{N_{w}}\sum_{j=1}^{N_{w}}(a_{i}[j]-M_{i})^{2}}\), for \(i=x,y,z\)
* Average absolute difference [3]: \(D_{i}=\frac{1}{N_{w}}\sum_{j=1}^{N_{w}}|a_{i}[j]-M_{i}|\), for \(i=x,y,z\)
* Average resultant acceleration [1]: \(ra_{i}=\frac{1}{N_{w}}\sum_{j=1}^{N_{w}}\sqrt{a_{x}[j]^{2}+a_{y}[j]^{2}+a_{z}[ j]^{2}}\)
* Binned distribution [30]: The range of values for each axis \((max-min)\) is calculated; it is divided into 10 equal sized bins, and then, what fraction of the \(N_{w}\) values fell within each of the bins is recorded.  
Each time window of each channel is first centered around its average, before being fed to the network. The description of the datasets after preprocessing with window sizes of 50 and 200 can be seen in Tables 4 and 5.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**50**} & \multicolumn{2}{c}{**200**} \\ \hline Total number of samples & 54,096 & 100\% & 51,222 & 100\% \\ \hline Number of attributes & 6 & \multicolumn{2}{c}{6} \\ \hline Activity-wise distribution & Total No. of samples & Percentage & Total No. of samples & Percentage \\ Walking & 21,105 & 39.0\% & 20,680 & 40.4\% \\ Jogging & 16,997 & 31.4\% & 16,590 & 32.4\% \\ Upstairs & 5888 & 10.9\% & 5007 & 9.8\% \\ Downstairs & 4781 & 8.8\% & 3940 & 7.7\% \\ Sitting & 2953 & 5.5\% & 2795 & 5.5\% \\ Standing & 2372 & 4.4\% & 2210 & 4.3\% \\ \hline \hline \end{tabular}
\end{table}
Table 4: Windowed WISDM dataset description.  
\begin{table}
\begin{tabular}{c c c} \hline \hline Total number of samples & 2,980,765 & 100\% \\ \hline Number of attributes & 6 & \\ \hline Number of missing values & 0 & \\ \hline Activity-wise distribution & Total number of samples & Percentage \\ \hline Walking & 1,255,923 & 42.1\% \\ Jogging & 438,871 & 14.7\% \\ Stairs & 57,425 & 1.9\% \\ Sitting & 663,706 & 22.3\% \\ Standing & 288,873 & 9.7\% \\ Lying Down & 275,967 & 9.3\% \\ \hline \hline \end{tabular}
\end{table}
Table 3: Actitracker dataset description.