# Reliability and Efficiency of Metamodel for Numerical Back Analysis of Tunnel Excavation
## 5 Results and Discussions
### Verification of the Metamodels  
To verify the metamodels, 243 datasets with parameter values that were not used in model training were generated by FLAC. Value numbers 11, 12, and 13 in Table 5 were used in the new test dataset, and these are the mean values of the numbers 1 and 2, 5 and 6, and 9 and 10 in Table 1, respectively. The numerical analysis method is the same as that noted in Section 3.  
The verification results are shown in Figure 5, and the MAPEs of Metamodel A to E are 9.5%, 6.1%, 4.9%, 0.8%, and 1.2%, respectively. The MAPE of Metamodel A shows the highest error (near 10%) among those of the five groups. From this result, it is assured that Metamodel A has an overfitting problem. That is, a metamodel developed by three values per parameter is unsuitable for computing datasets with new parameter values, which are not used in the training model, instead of the FLAC. The MAPE tends to decrease as the number of values per parameter increases and from Metamodel C, which was developed by six values per parameter, MAPE falls below 5%. Similar to predicting test datasets, Metamodels D and E developed by eight and ten values per parameter show very low MAPE of about 1%. These results imply that the generalization and performance of Metamodels D and E are excellent since they have been trained with a number of datasets. This is attributed to the MAPE converging to 1% as the number of values of each parameter increases.  
The MAPE trends in Figures 4 and 5 show a similar trend of decreasing as the number of values per parameter increases. In the trend, there are two different points. The ranks of the magnitude of the MAPE between Metamodels A and B and between D and E are reversed. Metamodel A shows a slightly lower error rate than B when predicting the test dataset in Section 5.1 due to the overfitting of Metamodel A. Metamodels D and E show low MAPE in both tests since they have a good generalization and good prediction performance as mentioned above. Therefore, the MAPE results are converged for the two tests for Groups D and E.  
Figure 4: Results of loss and MAPE of each metamodel.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Number** & \(\mathbf{\gamma}\) [kg/m\({}^{3}\)] & **UCS [MPa]** & \(\mathbf{m_{i}}\) & **GSI** & **K** \\ \hline
11 & 2431.2 & 105.3 & 7.5 & 15 & 0.625 \\
12 & 2716.6 & 187.5 & 19.5 & 55 & 1.625 \\
13 & 3002.1 & 269.7 & 31.5 & 95 & 2.625 \\ \(\Delta\) & 285.5 & 28.2 & 11.5 & 40 & 1 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Additional values of the five parameters to generate new test datasets.  
Based on the computing time to make the database of a metamodel of Group E, the computing times for Group A to D are expected about 0.2%, 1%, 8%, and 33%, respectively. These expectations considered only total numerical analyses time for database without metamodel development time using ANNs. The more datasets, the more time it takes to develop a model. If we allow about 5% or 1% error of a metamodel, then we can develop a metamodel in 8% or 33% of the time required to develop a metamodel using 10 values per parameter. These results will be helpful in efficiently generating a metamodel.