## 4 Results
### Recurrence Plot  
In this model, we utilized the recurrence plot encoding method to obtain a grayscale image and then the CNN model to train and predict the classes. We used two types of final layer: SoftMax and L2-SVM layers. For the SoftMax layer, we apply a cross-entropy as the loss function in ConvNet. In the L2-SVM, we tuned the network with two optimizers, SGD and ADAM.  
Figure 24 (left) illustrates the accuracy graph of the model with the SoftMax layer, with a training accuracy of 86% and a test accuracy of 78%. Nonetheless, It also shows the loss of the model with the SoftMax layer with a cross-entropy optimizer. The model indicates a few signs of noise loss, meaning that the weight decay is not smooth. However, SoftMax utilizing cross-entropy achieves a lower error value because the cross-entropy uses the log to penalize each error. This penalty value depends on the value of the error. Therefore,  
\begin{table}
\begin{tabular}{c c} \hline \hline \multicolumn{2}{c}{**Hyperparameters Used**} \\ \hline
**Warping Distance** & \(k\)**Value** \\ \hline  & 5 \\
10 & 7 \\  & 9 \\ \hline  & 11 \\
15 & 13 \\  & 15 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Hyperparameters of the k-nearest neighbor dynamic time warping (kNN-DTW) network.  
Figure 23: The architecture of the long short-term memory (LSTM).  
the accuracy increases, leading to a minor penalty and helping the model achieve a lower error value.  
Figure 24 (right) represents the confusion matrix of the SoftMax CNN model. The prediction of the classes is significantly high in the first and fourth classes. The generalization gap in the error values of testing and training affects the accuracy of the third class. Here, the error is lower in the SoftMax loss, but its optimization and weight decay are not as smooth as those of the L2-SVM with SGD. The L2-SVM utilizes a kernel-regularizer, also called the weight decay factor, with a value of 0.01 in the final output layer.  
Figure 25 (left) illustrates the accuracy graph of the L2-SVM with SGD with 78% training. During testing, it reached 71% accuracy. The model shows the loss graph of the L2-SVM with SGD. It settles with less noise, but the loss value remains in the range of 0.9 to 0.8. One possible reason for this phenomenon is the low learning rate, which slows graph convergence. In addition, Figure 25 (right) illustrates the confusion matrix of the SGD-tuned RP model with L2-SVM. We utilized the squared-hinge loss as the loss function with the L2 kernel regularizer and the SGD optimizer. The squared hinge rapidly increases the error value by penalizing it quadratically. However, the SGD optimizer is slower than Adam; it converges slowly.  
Nevertheless, Adam seems to be a challenge because of its fast convergence. Therefore, if the value of the learning rate (\(\alpha\)) and beta1 (\(\beta\)1) is average or high, it will approach the convergence rapidly, jump the convergence point, and start overfitting. Hence, we utilized an early stopping technique to achieve the best convergence to avoid this issue. Figure 26 (left) shows the result of the RP model with L2-SVM, including model accuracy, model loss, and the confusion matrix.  
Figure 25: Model accuracy (**left**), loss (**middle**), and confusion matrix of the SGD-tuned RP model with L2-SVM (**right**).  
In the GADF-encoded CNN model, we achieved 77.6% and 73.2% accuracy in the training and test, respectively. Figure 28 (left) illustrates an accuracy graph of the GADF. There was no significant change in the training results because the Adam optimizer remained biased towards zero, whereas it also shows the loss of the GADF CNN model. The model achieves an error value of 0.64 in training and 0.72 in the test. The model generalizes well, as indicated on the confusion matrix in Figure 28 (right).  
Table 9 compares the accuracy values of the GASF and GADF techniques. The accuracy value of all the GADF model classes is better than the GASF model, except for the 1 st class.