## 4 Discussion
### LSTM  
Figure 13a,b illustrate the performance of the LSTM in terms of the number of hidden layers and the number of neurones per layer, respectively. In Figure 13a, the R\({}^{2}\) value of the model on the training set rises with the addition of hidden layers. This suggests that increasing the number of hidden layers enhances the model's ability to match the training data and capture more intricate relationships. The R\({}^{2}\) reaches its maximum with three hidden layers in the test data, signifying that the model enhances its generalisation capacity by extracting data features for improved prediction. The addition of a fourth hidden layer results in a decline in the LSTM's predictive performance on the test dataset, signifying that the model becomes excessively complex and begins to overfit the training data, hence impairing its generalisation capability. Consequently, we established the optimal quantity of hidden layers for the model at three. The subsequent analysis focused on the amount of neurones within the hidden layers. As illustrated in Figure 13b, the R\({}^{2}\) values for the training and test datasets varied with the number of neurones. While no definitive trend emerged, the optimal model performance was achieved with 90 neurones in each of the three hidden layers, resulting in R\({}^{2}\) values of 0.9987 for the training set and 0.8864 for the test set. Incorporating additional neurones beyond the optimal threshold results in overfitting and a reduction in model generalisation. The model may overfit the details and noise in the training data rather than identifying more pertinent features, resulting in diminished performance on the dataset.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Search Techniques** & \
\begin{tabular}{c} **Number of** \\ **Estimators** \\ \end{tabular} &
\begin{tabular}{c} **Maximum** \\ **Depth** \\ \end{tabular} & \(\mathbf{R^{2}}\) **Train** & \(\mathbf{R^{2}}\) **Test** \\ \hline GridSearchCV & 100 & 200 & 1.000 & 0.8277 \\  & 100 & 20 & 0.999 & 0.8135 \\  & 200 & 20 & 0.996 & 0.7923 \\  & 42 & 71 & 0.997 & 0.8153 \\ RandomizedSearchCV & 10 & 94 & 0.999 & 0.7216 \\  & 102 & 94 & 1.000 & 0.7897 \\ \hline \hline \end{tabular}
\end{table}
Table 11: R\({}^{2}\) values using various hyperparameters and search methodologies in forecasting hysteresis utilising RF.  
HKOA-LSTM  
In Figure 14a, the model's superior performance on the training set remains nearly the same as the number of hidden layers in the neural network varies. The R\({}^{2}\) of the model on the test set rises with the addition of hidden layers, peaks at three hidden layers, and subsequently declines upon expanding to four layers. Consequently, we select a network architecture with three hidden layers. Figure 14b examines the impact of the neurone count in these three hidden layers on model performance. As the quantity of neurones in the hidden layers escalates, the model's performance remains exceptional and nearly invariant within the training set. In the test set, R\({}^{2}\) progressively rises with an increase in the number of neurones in the hidden layer, achieving optimal model performance at 80 neurones. A compromise between the model tunability and generalisation capability is attained, resulting in R\({}^{2}\) values of 1 and 0.981 for the training and test sets, respectively.  
Figure 14: R\({}^{2}\) values of the HKOA-LSTM model for forecasting hysteresis based on (**a**) the number of hidden layers; (**b**) the number of neurones within the hidden layer.  
Figure 13: R\({}^{2}\) values of the LSTM model for forecasting hysteresis based on (**a**) the number of hidden layers; (**b**) the number of neurones in the hidden layer.