## 3 The Proposed Model
### Activity Recognition: HCNN-BiRNN Model  
At the final stage, the classification of human activities is carried out by the use of CNN-BiRNN model. The CNN-BiRNN hybrid models contain two major mechanisms: a BiRNN with attention model [24] on the top half and a CNN with \(L\) convolution layers on the bottom half. These components of the models are jointly trained in an end-to-end manner. One sample \(X\in\mathbb{R}^{1\times W\times 1}\) is a real value vector (\(W\) refers to the range dimension length); therefore, a 1D-CNN module is applied, from which the convolutional process occurs in the range dimension. For the initial layer, convolutional operations using stride length 1 employ \(K_{1}\) filters, \(W^{(1)}\in\mathbb{R}^{1\times w^{(1)}\times 1}\), to \(X\), resulting in feature maps of layer 1, \(H^{(1)}\in\mathbb{R}^{1\times W^{(1)}\times K_{1}}\). For the next \(L-1\) convolutional layers, convolution-pooling operations repetitively employ \(K_{l}\) filters, \(W^{(l)}\in\mathbb{R}^{1\times w^{(l)}\times K_{l-1}}\), to \(H^{(l-1)}\), and feature map \(H^{(L)}\in\mathbb{R}^{1\times W^{(l)}\times K_{L}}\) is obtained after the \(L\) convolution layer. Every convolution layer is followed by a pooling layer; therefore, the time dimension is shortened, and the temporal dependency increases as the convolution layer grows. After dropping the single dimension, \(H^{(L)}\in\mathbb{R}^{W^{(hl)}\times K_{L}}\) is regarded as a series of length \(W^{(hl)}\) with \(K_{L}\) feature vector at every time step. Next, we replace the \(W^{(hl)}\) time step, using \(T\) for notation convenience. The forward recurrent neural network (RNN) reads \(H^{(L)}\) in its novel order and produces hidden state \(fh_{t}\) at all the time steps, and the backward RNN reads \(H^{(L)}\) in its reverse order and generates \(bh_{t}\) as follows  
\[fh_{t}=f\Big{(}W_{fkh}\cdot H^{(L)}_{t}+W_{fkh}\cdot fh_{t-1}\Big{)},\ t=1\dots T \tag{10}\]\[bh_{t}=f\Big{(}W_{bxh}\cdot H_{t}^{(L)}+W_{bhh}\cdot bh_{t-1}\Big{)},\ t=T\ldots \tag{11}\]  
From the expression, \(W_{fxh}\in\mathbb{R}^{m\times K_{L}}\) and \(W_{bxh}\in\mathbb{R}^{m\times K_{L}}\) refer to the input-hidden weights, \(W_{fhh}\in\mathbb{R}^{m\times m}\) and \(W_{bhh}\in\mathbb{R}^{m\times m}\) denotes the weight that connects the hidden layer, \(m\) denotes the dimensionality of the hidden state, and \(f\) indicates the sigmoid function. Next, the concatenation of \(fh_{t}\) and \(bh_{t}\) forward and backward states create \(h_{t}=[fh_{t},\ bh_{t}]\). Consequently, every \(h_{t}\) hidden state comprises data of the entire target, with stronger emphasis on the part near the region at \(t\) step. In BiRNN, the data at all the time steps might slowly be lost alongside the backward and forward propagation. To prevent data loss and focus on the discriminatory time step automatically, the attention module is proposed, which, as a byproduct, is capable of relaxing the misalignment problem. In this technique, we adapt a multilayer perceptron for calculating the attention weight depending on the hidden state and indicate \(g\) as an invariant feature vector, i.e., weighting sum of \(h_{t},t=1\ldots T\), i.e.,  
\[g=\sum_{r=1}^{T}\alpha_{t}\cdot h_{t} \tag{12}\]  
The weight, \(\alpha_{t}\), is calculated using the below equation  
\[\alpha_{t}=\frac{\exp\big{(}e^{t}\big{)}}{\sum_{l}\exp\big{(}e^{l}\big{)}} \tag{13}\]  
\[e^{t}=U_{a}^{T}\ \text{tan}\ \text{h}\ (W_{a}\cdot h_{t}) \tag{14}\]  
Let \(U_{a}\in\mathbb{R}^{1\times n}\), \(W_{a}\in\mathbb{R}^{n\times 2m}\) be the parameter of the attention module, and weight \(\alpha_{t}\) stands for the coefficient which scores the matching degree among the recognition task and \(h_{t}\) hidden state. The invariant feature vector \(g\) incorporates the data at each time step based on the discrimination of the hidden state.