# Machine Learning Algorithms for the Prediction of the Seismic Response of Rigid Rocking Blocks
## 4 Results
### Regression Results  
Similarly, to measure the performance of our regression models, the well-known metrics of root mean squared error (RMSE), mean absolute error (MAE), and coefficient of determination \(R^{2}\) are employed. Table 5 summarizes the employed regression metrics and their corresponding formulas, while Table 6 summarizes their computed values on the test set for both regression models examined here.  
We can readily observe that the gradient boosting regressor outperforms the regression tree, although not by a huge margin. Low MAE, MSE, or RMSE values show that the regression model has high accuracy, which is desired. On the other hand, \(R^{2}\) indicates how well the considered independent variables explain the variability of the dependent parameter. RMSE is a more robust index, indicating how well the model will fit with unseen data (generalization). The RMSE is almost double the MAE for both models, due to the presence of a few very large values in the models' errors. In Figure 7, the predictions \(\hat{y}_{test}\) of the two best-performing models, gradient boosting and ANN, versus the actual values \(y_{test}\) of the test dataset, are presented. We also present the thresholds for which we find that approximately 90% or more of the model's predictions lie within this distance from the true value.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline  & **Regression Tree** & **Gradient Boosting** & **Random Forest** & **ANN** \\ \hline RMSE & 0.09265 & 0.07754 & 0.08526 & 0.08169 \\ MAE & 0.04636 & 0.03977 & 0.04344 & 0.04245 \\ \(R^{2}\) & 0.84 & 0.89 & 0.87 & 0.88 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Regression metrics on the test dataset of employed models.  
Figure 6: Confusion matrices for the train and test dataset of the two examined classifiers. (**a**) ANN training confusion matrix. (**b**) ANN test confusion matrix. (**c**) Gradient boosting training confusion matrix. (**d**) Gradient boosting test confusion matrix.  
\begin{table}
\begin{tabular}{l c} \hline \hline
**Metric** & **Formula** \\ \hline RMSE & \(\sqrt{\frac{1}{N}\sum_{i=1}^{N}(y_{i}-\hat{y}_{i})^{2}}\) \\ MAE & \(\frac{\sum_{i=1}^{N}(y_{i}-\hat{y}_{i})}{\sum_{i=1}^{N}(y_{i}-\hat{y}_{i})^{2}}\) \\ \(R^{2}\) & \(1-\frac{\sum_{i=1}^{N}(y_{i}-\hat{y}_{i})^{2}}{\sum_{i=1}^{N}(y_{i}-\hat{y}_{i})^ {2}}\) \\ \hline \hline \end{tabular}
\end{table}
Table 5: Summary of the employed regression metrics. In the below, \(y_{i}\) and \(\hat{y}_{i}\) are the true values and model predictions, respectively, and \(N\) is the number of samples in the dataset.  
In Figure 8, we present the distribution of the errors in the models' predictions on the test dataset, along with its kernel density estimation. The errors are not normally distributed, as their distribution is leptokurtic, with kurtosis \(\approx 14.3\gg 3\), where 3 is the kurtosis of the normal distribution. This fact also means that the distribution has fatter tails and some extreme values exist. However, as was previously mentioned, more than 90% of the errors of the best-performing regressor are in the interval \([-0.1,0.1]\). Furthermore, it can be observed that the errors appear to be symmetrically distributed around 0, and indeed, both the mean and median of the errors are very close to 0, with values of \(3.09\times 10^{-4}\) and \(-1.21\times 10^{-3}\), respectively. That means that the model is unbiased, i.e., it does not systematically over or underestimate the true value.  
Finally, Figure 9 displays the evolution of the mean squared error of the two best-performing regressors. There is no significant gap between the loss on the training and validation datasets; thus, the algorithms did not overfit.  
Figure 8: Regression error distribution on the test dataset, along with its kernel density estimation, of the two best-performing regressors. (**a**) ANN. (**b**) Gradient Boosting.  
Figure 7: True values vs. predictions made by the best-performing regressors. (**a**) ANN. (**b**) Gradient boosting.