## 3 Results and Discussion
### Estimation of Bearing Capacity Using DNN  
To utilize an AI for predicting the bearing capacity, a DNN was applied to the same load test database. Overfitting (or overtraining) is a major problem in neural networks, and training conditions, such as the number of hidden layers, nodes, and epochs, also influence the prediction accuracy. The prediction accuracy for training data could be increased by adding hidden layers or increasing epoch values in the DNN. However, this did not lead to improving the prediction accuracy for the test data, but the prediction accuracy could be reduced due to the overfitting. Figure 9 shows the comparison of the observed and the predicted ultimate bearing capacity for cross-validation and all data in the DNN when the training is performed to minimize only the MAE for the training data. The predicted ultimate bearing capacity using all data in the DNN was found to be nearly the same as the observed values with an MAE of 3.4 kPa. However, when the same learning conditions were applied for cross-validation, the predicted values showed large errors: MAE, bias, and COV in  
Figure 8: Tornado diagram for the influence of the variables in the MLR model. Figure 9: Comparison of the predicted and the observed ultimate bearing capacity by the deep neural network (DNN) without considering cross-validation error. bias were 119 kPa, 1.01, and 29.2%, respectively. As described previously, the reason was that, although the prediction errors for the training data were reduced by adding hidden layers and increasing the epoch values, the prediction for the new condition had a large error due to the overfitting. The same process was applied to the ANN to compare the difference between the ANN and DNN for the overfitting problem. When using all data, the MAE for the ANN was 2.9 kPa, almost equal to the DNN, but the MAE, bias, and COV in bias for the cross-validation were 169 kPa, 1.03, and 35.7%, respectively, which were very large compared with the DNN. This means that the overfitting problem can occur significantly in the ANN, and that the DNN is better for predicting new data by compensating for the overfitting problem of the ANN. However, selecting an optimal structure that considers cross-validation errors is still crucial for a robust prediction of bearing capacity for new conditions, and the leave-one-out cross-validations were performed for various training conditions (324 cases), as summarized in Table 5.  
Seven variables excluding the ultimate bearing capacity in Table 1 were available as raw input data, but adding various forms of input values in advance could be more effective for training. Therefore, new input variables for combining \(s_{u}\) and \(a_{r}\) were added, and 10 variables were used as input data. Raw data were comprised of attributes with varying scales, and data preprocessing plays a very important part in many deep learning algorithms. Normalization is required so that all the inputs are at a comparable range, and many methods work best after the data have been normalized or standardized in practice. Therefore, min-max normalization was applied to rescale the raw data as follows:  
\[z=\frac{x-min(x)}{[\max(x)-\min(x)]} \tag{6}\]  
For the leave-one-out cross-validation, 36 load test data were used for training, and the excluded 1 load test datum was used for verification. This procedure was performed on all 37 data, and the prediction performance was evaluated through the average MAE. Figure 10 displays the change in MAE according to the number of epochs for the training and validation of the data. Figure 10: Change in accuracy according to epochs for training and cross-validation (hidden layer = 2, node = 10, batch size = 20). \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Hidden Layer** & **Batch Size** & **Node** & **Epoch** & **Drop Rate** \\ \hline \(1,2,3\) & \(10,20\) & \(5,10,15\) &
\begin{tabular}{c} 1000, 1500, 2000, 2500, 5000, \\ 7500, 10,000, 12,500, 15,000 \\ \end{tabular} & 0, 0.25 \\ \hline \hline \end{tabular}
\end{table}