# Optimizing the Parameters of Long Short-Term Memory Networks Using the Bees Algorithm
## 4 Results and Discussion
### Electrocardiogram (ECG) Dataset  
The novel hybrid BA-CNN-LSTM algorithm developed using the MATLAB platform can be designed to deal with classification problems as well. It was applied to the Electrocardiogram (ECG) benchmark images described in [44] to classify human ECG time series signals into three classes of cardiac arrhythmia (ARR), congestive heart failure (CHF), and normal sinus rhythms (NSR). The following Figure 11 shows an example of the three classes of ECG time series signals. There were 162 recordings with 96 observations from the ARR, 30 recordings from the CHF, and 36 from the NSR. Because the dataset was not large enough, the \"SqueezeNet\"  
Figure 11: An illustrative example of the three classes of ECG time series signals. pre-trained CNN network was used to extract the features in the images. The data were divided into 81 recordings for the training, 41 recordings for the validation set, and 40 observations for the testing set. The following Table 7 shows the values of the LSTM parameters for the evaluations of the BA. As can be seen in the previous table, the second evaluation yielded the minimum classification error for the validation set with a value of 0.1220, so the global learning rate of 0.00029 was adjusted in the forward side of the input gate by multiplying it by 1.0422, which resulted in a more optimum learning rate value of 0.0003. Similarly, the learning rate in the forward side of the forget gate was improved to 0.00031 using an adjustment factor of 1.0984. The new learning rate value for the forward cell candidate was 0.00028 after multiplying the global learning rate by 0.9703. The adjustment factor for the forward output gate was 1.0198, which resulted in a learning rate value of 0.00029. The new values for four backward parameters were 0.00031, 0.00031, 0.00027, and 0.0003 for the input gate, forget gate, cell candidate, and output gate, respectively. They were adjusted using factors of 1.0804, 1.0722, 0.9603, and 1.0578, respectively. Finally, the performance of the fully connected layer was improved as well by specifying a customized learning rate of 0.00027 after multiplying the global learning rate by 0.9413. The following Table 8 summarizes the new learning rate values for the LSTM parameters. The following Figure 12 shows the training progress for the proposed BA-CNN-LSTM algorithm using the new learning rate values stated above. The blue line represents the training progress, and the black line represents the validation set. \begin{table}
\begin{tabular}{c c c} \hline \hline
**LSTM Parameter** & **1** & **2** \\ \hline Learning rate factor for input gate (Forward) & 0.9483 & 1.0422 \\ Learning rate factor for forget gate (Forward) & 0.9808 & 1.0948 \\ Learning rate factor for cell candidate (Forward) & 0.9193 & 0.9703 \\ Learning rate factor for output gate (Forward) & 0.9264 & 1.0  
As can be seen from the figure, the training started with low classification accuracy and increased steadily until it reached a percentage value of 100%. In the validation set, the chart had almost the same pattern and reached a validation accuracy of 87.80% at the end of the chart. The following three Figures 13-15 show the confusion matrix for the BA-CNN-LSTM algorithm for all three sets:  
Figure 12: Training progress for the proposed hybrid BA-CNN-LSTM algorithm (ECG dataset). Figure 13: Confusion matrix for the training set of the proposed hybrid BA-CNN-LSTM algorithm. Since it is a classification problem, the \(t\)-test was not applicable, as the error is either zero or one. The following Figure 16 shows the validation accuracy for all four algorithms using 10-fold cross-validation. Figure 14: Confusion matrix for the validation set of the proposed hybrid BA-CNN-LSTM algorithm. Figure 15: Confusion matrix for the testing set of the proposed hybrid BA-CNN-LSTM algorithm.