# Optimizing the Parameters of Long Short-Term Memory Networks Using the Bees Algorithm
## 3 Proposed Novel Hybrid BA-CNN-LSTM Algorithm
### LSTM Architecture  
The features were extracted from the artificial porosity images using the CNN to feed the LSTM, which retains the information for a long period so that it can remember long-term dependencies [7]. The following points are detailed mathematical explanations of the LSTM network. * Forget Gate  
Figure 3: The general framework for the proposed BA-CNN-LSTM algorithm. The first step in the LSTM cell is to decide if the information coming from the previous time scale is relevant to be remembered or irrelevant to be forgotten. It is based on the following forget gate equation [9]:  
\[\mathrm{f_{t}}=\sigma\times(X_{t}\times U_{f}+H_{t-1}\times W_{f}) \tag{1}\]  
where:  
\(X_{t}\) is the current time cycle input;  
\(U_{f}\) is the input weight;  
\(H_{t-1}\) is the previous time cycle hidden state;  
\(W_{f}\) is the hidden state weight matrix. Then, the following sigmoid function is applied, resulting in a \(f_{t}\) value between 0 and 1 [39]. \[\sigma\;(x)=(1+e^{-x})^{-1} \tag{2}\]  
The \(f_{t}\) is multiplied by the cell state of the previous time cycle:  
\[C_{t-1}\times ft=0\;(Forget\;everything) \tag{3}\]  
\[C_{t-1}\times ft=C_{t-1}\;(Forget\;nothing) \tag{4}\]  
* Input Gate  
This gate is used to quantify the importance of the new input information. It is based on the following input gate equation [9]:  
\[i_{t}=\sigma\times(X_{t}\times U_{i}+H_{t-1}\times W_{i}) \tag{5}\]  
where:  
\(X_{t}\) is the current time cycle input;  
\(U_{f}\) is the input weight matrix;  
\(H_{t-1}\) is the previous time cycle hidden state;  
\(W_{f}\) is the hidden state weight matrix. Similarly, the sigmoid function is applied, resulting in an \(i_{t}\) value between 0 and 1. Passing the information to the cell state is based on a function of the hidden state of the previous time cycle [24]:  
\[N_{t}=\tanh(X_{t}\times U_{c}+H_{t-1}\times W_{c})\;(new\;information) \tag{6}\]  
Using the \(\tanh\) function results in a \(N_{t}\) value between \(-1\) and \(1\). If it is positive, it will be added to the cell state, and, if it is negative, the information will be subtracted from the cell state, as shown in the following equation [24]:  
\[C_{t}=f_{t}\times C_{t-1}+i_{t}\times N_{t}\;(updating\;cell\;state) \tag{7}\]
* Output Gate  
In this gate, the cell transfers the updated information from the previous time cycle to the next time cycle. It is based on the following output gate equation [9]:  
\[O_{t}=\sigma\times(X_{t}\times U_{o}+H_{t-1}\times W_{o}) \tag{8}\]  
Likewise, the \(O_{t}\) value is between 0 and 1, because the sigmoid function has been applied. The current hidden state is a function of the long-term memory and the output, and it is calculated based on the following equation:  
\[H_{t}=O_{t}\times tanh(C_{t}) \tag{9}\]The output of the current time cycle is found using the SoftMax function, as shown in the following equation [23]:  
\[\mathrm{f_{t}}=\sigma\times(X_{t}\times U_{f}+H_{t-1}\times W_{f}) \tag{1}\]0  
The output with the maximum score is the predicted value. The above-described mechanism is used to provide a more accurate prediction of the porosity percentages appearing in sequential layers of artificial porosity images [38] that mimic CT scan images of parts manufactured by the SLM process. The created artificial porosity images in [38] consist of 30 3D cubes. Each one was sliced into 100 2D sequential slices, thus resulting in 3000 slices. Therefore, there are 30 sequences with 100 layers in each sequence. A total of 18 sequences were used for the network training task, and 6 sequences were used for the validation set, as well as 6 for the testing set. Reference [38] presented an example of artificial porosity images that mimic the real porosity images with a similarity index of 0.9967, as shown in Figure 4 [38].