# Optimizing the Parameters of Long Short-Term Memory Networks Using the Bees Algorithm
## 4 Results and Discussion
### Artificial Porosity Images  
The novel hybrid BA-CNN-LSTM algorithm was developed using the MATLAB platform. It was applied to the created artificial porosity images described in the previous section [38] to predict the percent of porosity in sequential layers of the SLM parts. The 30 sequences were divided into 18 sequences for the network training task, 6 sequences for the validation set, and the 6 sequences for the testing set. Since each sequence has 100 layers, 1800 slices were used for training, and 600 slices were used for each of the validation and testing sets. The following Table 3 shows the values of the LSTM parameters for the four evaluations of the BA. As can be seen in the previous table, the first evaluation yielded the minimum prediction error on the validation set with a value of 0.0115, so the global learning rate of 0.0101 [38] was adjusted in the forward side of the input gate by multiplying it by 1.0618, which resulted in a more optimum learning rate value of 0.0107. Similarly, the learning rate in the forward side of forget gate was improved to 0.0095 by using an adjustment factor of 0.9472. The new learning rate value for the forward cell candidate was 0.0092 after multiplying the global learning rate by 0.9151. The adjustment factor for the forward output gate was 1.0348, which resulted in a learning rate value of 0.0104. The new values for the four backward parameters were 0.0108, 0.0103, 0.0098, and 0.0103 for input gate, forget gate, cell candidate, and output gate, respectively. They were adjusted using factors \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**LSTM Parameter** & **1** & **2** & **3** & **4** \\ \hline Learning rate factor for input gate & 1.0618 & 0.9737 & 1.0629 & 1.0810 \\ (Forward) & 0.9472 & 0.9706 & 1.0812 & 1.0177 \\ Learning rate factor for cell candidate & 0.9151 & 1.0451 & 0.9254 & 0.9291 \\ (Forward) & 1.0349 & 1.0273 & 1.0827 & 1.0300 \\ Learning rate factor for input gate & 1.0684 & 1.0604 & 1.0265 & 0.9429 \\ (Backward) & 1.0258 & 1.0236 & 0.9195 & 1.0386 \\ Learning rate factor for cell candidate & 0.9749 & 0.9621 & 0.9557 & 1.0006 \\ Learning rate factor for output gate & 1.0253 & 0.9779 & 1.0094 & 0.9224 \\ Learning rate factor for fully & 0.9623 & 0.9477 & 1.0915 & 0.9931 \\ connected layer & 0.0115 & 0.0120 & 0.0134 & 0.0135 \\ Prediction error for the validation set & 0.0115 & 0.0120 & 0.0134 & 0.0135 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The values of LSTM parameters in the four evaluations of BA (artificial porosity images). of 1.0684, 1.0258, 0.9749, and 1.0253, respectively. Finally, the performance of the fully connected layer was improved as well by specifying a customized learning rate of 0.0097 after multiplying the global learning rate by 0.9623. The following Table 4 summarizes the new learning rate values for the LSTM parameters:  
The following Figure 8 shows the training progress for the proposed BA-CNN-LSTM algorithm using the new learning rate values stated above. The blue line represents the training progress, and the black line represents the validation set. As can be seen from the figure, the training started with a root mean square error (RMSE) of 0.2 and decreased significantly after the first 200 iterations, and then the chart experienced a steady state around an RMSE value of 0.02. In the validation set, the chart started with an RMSE value of 0.05, and it was alternating in the first 150 iterations before reaching an RMSE value of 0.0160 at the end of the chart. The following Table 5 shows the average porosity error (the difference between the actual and predicted percent of porosity) for the training, validation, and testing sets using 10-fold cross-validation. The novel hybrid BA-CNN-LSTM algorithm was compared with an existing algorithm that used Bayesian Optimization (BO) to optimize the same LSTM parameters (BO-CNN-LSTM) [46].