# Deep Learning Models for Yoga Pose Monitoring
## 3 Proposed System
### Pose Prediction  
The second phase concerns the creation of a deep learning model that aims at correctly classifying any real-time video to one of the six poses given in the dataset. Here, a hybrid model is incorporated, which is a mixture of CNN and LSTM, where CNN is used for feature extraction [13]. Specifically, CNN is a multilayered Artificial Neural Network (ANN) that is specifically created to work on images and is used for tasks like object recognition and image classification [28]. On the other hand, LSTM is useful for understanding the sequence of frames occurring in a particular yoga pose. Concretely, LSTM is a type of RNN that is equipped for learning and recollecting extremely long-term dependencies over long successions of input data [29]. In this work, a TimeDistributed layer along with CNN, which is particularly useful when working with video frames or time series data [30], is employed. Moreover, a Softmax layer, which uses a normalised exponentiation function, is used to find the probability of each yoga pose. The pose with the highest probability is predicted as the output. To be more specific, The first layer used in the pose prediction is a CNN layer with 16 filters and a window size equals to 3. The activation function used in this layer is ReLU, which is a piece-wise linear function that gives an output equal to 0 when the input  
Figure 3: MediaPipe Keypoints on Actual Humans. Figure 2: MediaPipe Keypoints. provided is less than 0, else it gives as output the given input [31; 32]. Equation (1) presents the activation function of \(ReLU\):  
\[ReLU(x)=max(0,x) \tag{1}\]  
where \(x\in R\). The second layer used is a Batch Normalization layer that solves the problem of internal covariate shift and makes easier the flow of data through different layers [33]. The next layer used is a Dropout layer, which is a regularization technique for preventing overfitting at the rate of 0.5 [34]. Afterwards, another Batch Normalization layer is used. The output obtained from this layer is then passed to a Flatten layer that converts the data to a one-dimensional array. The next layer utilized is an LSTM layer where its size is 20 units with forget bias set to True in order to return the output of every node. The output of LSTM is generally generated through a series of Gated operations, like the Forget gate, the Input gate and the Output Gate. The mathematical equations for an LSTM are presented in the following Equations (2)-(7). The state information of current and previous cell are stored in \(C\). The sigmoid function output in the forget gate indicates which information to retain or forget. The output mainly depends upon the previous state output vector \(h_{r-1}\) and current state input vector \(x_{r}\). If there is difference, then \(\sigma\) function does not allow the information to retain. Moreover, the forget bias gives a shift to the output either in the direction of 1 (retain) or 0 (forget); the model outperforms with forget bias value equal to 0.5. In the input gate, the \(\sigma\) function decides which value to update and \(\tanh\) function adds new values for the state. Regarding the output layer, the \(\sigma\) function determines the value to be selected as output \(h_{r}\).