# Real-Time Physical Activity Recognition on Smart Mobile Devices Using Convolutional Neural Networks
## 3 Materials and Methods
### Convolutional Neural Network  
CNN is a hierarchical Feed-Forward Neural Network (FFNN) whose structure is inspired by the biological visual system. Its principal difference from standard neural networks is that apart from fully-connected layers, it has a number of convolutional layers, where it learns filters that are sliding along the input data and applied to its sub-regions. The fully-connected layer is the building block of neural networks. There are different ways to describe the intuition behind this layer, but at its core, it is a non-linear function that maps the inputs to the outputs. In this case, the input is a one-dimensional vector \(\mathbf{x}\), sized \(M\times 1\). Every layer has a weight matrix \(\mathbf{W}\), which contains the weights connecting every element of the input to the corresponding node of the layer, so \(\mathbf{W}\) is an \(N_{n}\times M\) matrix, where \(N_{n}\) is the number of the layer's nodes. In addition to the weight matrix, a bias vector \(\mathbf{b}\), which outputs constant values for any input, is also part of every layer and is added to the output. Another integral part of a layer is its non-linear activation function \(f(\cdot)\). Three commonly used activation functions are sigmoidal, hyperbolic tangent, and Rectified Linear Unit (ReLU) [62]. The third one is defined as \(f(Z)=max(0,Z)\), which is a thresholding operation. Finally, the overall function of the layer can be described as:  
\[\mathbf{s}=f(\mathbf{W}\cdot\mathbf{x}+\mathbf{b})=max(0,\mathbf{W}\cdot \mathbf{x}+\mathbf{b}) \tag{1}\]  
where the output \(\mathbf{s}\) is an \(N_{n}\times 1\) vector and \(\mathbf{b}\) is a bias vector. The convolutional layer provides feature extraction by exploiting the temporal information of the data. 1D convolution is applied, which means that the applied filters are slid to the direction of only one dimension and not necessarily that the data themselves are one-dimensional. The convolutional layer's parameters consist of a set of learnable filters. During the forward pass, each filter is slid (more precisely, convolved) across the temporal axis of the input volume, and dot products are computed between the entries of the filter and the input at any position. The output's width is calculated as:  
\[W_{out}=(W_{in}-K+2P)/S+1 \tag{2}\]  
where \(K\) is the filter's kernel size, \(S\) is the stride, and \(P\) is the number of zero padding that is added, while the output depth will be equal to the number of filters \(F\). The output of convolving the input with the layer's filters is the following:  
\[\mathbf{o_{j}}=f\left(\sum_{k=1}^{W_{in}}\mathbf{W_{kj}}\cdot\mathbf{X_{k}}+ \mathbf{b_{j}}\right) \tag{3}\]  
The matrices \(\mathbf{W_{kj}}\) represent the filters that are convolved with the input, while \(\mathbf{b_{j}}\) is the bias vector that is added to the output. Finally, \(j\) runs from 1 to \(F\), the number of convolutional filters, and \(f(\cdot)\) is the activation function just like in the fully-connected layer, which is the ReLU activation in this case. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**50**} & \multicolumn{2}{c}{**200**} \\ \hline Total number of samples & 148,055 & 100\% & 144,232 & 100\% \\ \hline Number of attributes & 6 & \multicolumn{2}{c}{6} \\ \hline Activity-wise distribution & Total No. of samples & Percentage & Total No. of samples & Percentage \\ \hline Walking & 62,493 & 42.2\% & 61,280 & 42.5\% \\ Jogging & 21,772 & 14.7\% & 21,133 & 14.7\% \\ Stairs & 2832 & 1.9\% & 2673 & 1.9\% \\ Sitting & 32,969 & 22.3\% & 32,142 & 22.3\% \\ Standing & 14,274 & 9.6\% & 13,613 & 9.4\% \\ Lying Down & 13,715 & 9.3\% & 13,391 & 9.3\% \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Windowed Actitracker dataset description. Pooling layers are usually used after a convolutional layer to reduce the complexity of the implementation and compress the representation. There are two dominant variations, the max-pooling layer and the average pooling layer [63]; the former is used in this model.