## 3 Numerical Results
### Metamodeling Results  
Metamodels described in the previous section are compared using the coefficient of determination R\({}^{2}\), using the Formula (11). The R\({}^{2}\) results for different neural network \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**No.** & **Symbol** & **Parameter Name** & **Min** & **Max** \\ \hline
1 & A & Population ratio R & 0.16 & 1.00 \\
2 & B & Crossover rate & 0.50 & 1.00 \\
3 & C & Mutation rate & 0.01 & 0.10 \\
4 & D & Penalty multiplier & 10.0 & 1000.0 \\
5 & E & Penalty exponent & 1.0 & 3.0 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Parameters for DOE performed on the GA and EVOL. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**No.** & **Symbol** & **Parameter Name** & **Min** & **Max** \\ \hline
1 & A & Number of islands & 2 & 4 \\
2 & B & Number of generations & 40 & 100 \\
3 & C & Migration rate & 0.05 & 0.20 \\
4 & D & Migration interval & 4 & 20 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Parameters for DOE performed on the MIGA. topologies were plotted in Figure 7, showing the influence of the number of hidden neurons in each layer on the final metamodel performance. Two types of metamodels are then compared in Table 3.  
\[R^{2}=1-\frac{\sum_{i=0}^{n_{t}}\left(y^{(i)}-\hat{y}^{(i)}\right)^{2}}{\sum_{i=0} ^{n_{t}}\left(y^{(i)}-\bar{y}\right)^{2}}=1-\frac{MSE}{Var(y)} \tag{11}\]  
where:  
\(n_{t}\)--the number of points for which the metamodel prediction is held out;  
\(y(i)\)--the \(i\)-th response value obtained using FEM;  
\(\hat{y}\)(\(i\))--the response value predicted with the metamodel;  
\(\bar{y}\)--mean value from all \(y(i)\) (FEM). As R\({}^{2}\) for training data is not assuring that the metamodel can predict unseen data correctly, only the R\({}^{2}\) values of validation data were used for comparison. Moreover, when the coefficient of determination is high for training data and, at the same time, low for validation data, it usually means that the model is overfitted, making it useless for any applications. The same studies were conducted for both sets of data: Big (5400 training, 1350 validation samples) and small (1350 training, 340 validation samples) to investigate the effect of sample size on the precision of surrogate model predictions. Furthermore, the metamodel training time is significantly reduced with the decreasing data size. Artificial neural networks with different topologies, activation functions, and/or regularization methods were compared, and the best-fitted ones were reported herein--see Table 3 for details. Each combination was run 10 times to partially overcome the effect of randomness, associated with the generation of initial weights and biases. As a result of the model fitting, the ANN used for FLMIT prediction, had 27-9-39-1 architecture, with the rectified linear unit (ReLU) activation function in both hidden layers. The influence of the network topology on the coefficient of determination (for validation data) R\({}^{2}\)V is shown in Figure 7.  
In general, the ANN models were better fitted once the bias and weight optimization (training) was performed incorporating the L-BFGS algorithm, rather than NADAM and SGD. Furthermore, L-BFGS converged significantly faster (40-100 iterations) compared to SGD (10,000-20,000 iterations). The only drawback is higher memory consumption associated with gradient history; however, the total usage is still orders of magnitude lower compared to the FEM calculations. The convergence history of the ANN training process is shown in Figure 8.  
On the other hand, the simple Kriging method was taken as a model of choice to be used for mass predictions, as it outperformed the neural networks (in predictive accuracy) for this particular output. Moreover, similar to ANNs, each Kriging fitting was repeated \(10\times\)--once with equal initial theta values, and 9\(\times\) with random initiation of theta vector.