# Machine Learning Algorithms for the Prediction of the Seismic Response of Rigid Rocking Blocks
## 4 Results
### Classification Results  
In order to measure the performance of our classification models, some well-known classification metrics were computed, specifically [77]  
\[\text{Precision}=\frac{TP}{TP+FP}, \tag{7}\]  
\[\text{Recall}=\frac{TP}{TP+FN}, \tag{8}\]  
\[\text{F1-score}=2\frac{\text{Precision}\cdot\text{Recall}}{\text{Precision}+ \text{Recall}^{\prime}} \tag{9}\]  
\[\text{Accuracy}=\frac{TP+TN}{TP+FP+FN+TN}, \tag{10}\] \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Algorithm** & **Hyperparameter** & \multicolumn{2}{c}{**Value**} \\ \hline  & & Classification & Regression \\ \cline{3-4}  & neurons\_per\_layer & (35,) & (384, 384, 384, 384) \\ ANN & activation\_function & ReLU, \(tanh\) & ReLU, Linear \\  & learning\_rate & 0.1 & 0.1 \\ \hline  & & Classification & Regression \\ \cline{3-4}  & max\_iter & 1000 & 2000 \\  & n\_iter\_no\_change & 50 & 10 \\ Gradient Boosting & max\_leaf\_nodes & 127 & 384 \\  & learning\_rate & 0.1 & 0.1 \\  & min\_samples\_leaf & 20 & 100 \\ \hline Decision Tree & max\_depth & 25 & \\  & min\_samples\_leaf & 20 & \\ \hline Random Forest & n\_estimators & 1000 & \\  & max\_depth & 25 & \\  & min\_samples\_leaf & 20 & \\ \hline \hline \end{tabular} Note: ANN and gradient boosting were employed for both both tasks, while decision tree and random forest were employed for classification and regression, respectively.  
\end{table} 
Table 2: Hyperparameter configuration of the ML models. \begin{table}
\begin{tabular}{c c} \hline \hline
**Hyperparameter** & **Description** \\ \hline neurons\_per\_layer & The number of neurons per layer in the ANN. \\ learning\_rate & Shrinkage factor that controls the gradient of the loss function. \\ max\_iter, n\_estimators & Maximum number of trees in the ensemble. \\ n\_iter\_no\_change & Number of iterations before early stopping. \\ activation\_function & The activation function of each neuron in ANN. \\ min\_samples\_leaf & Minimum number of points to remain on each tree leaf. \\ max\_leaf\_nodes & Maximum number of leaf nodes allowed. \\ max\_depth & The maximum allowed tree depth. \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Hyperparameter descriptions. where in the above \(TP,\ FP,\ FN\), and \(TN\) are the number of true positive, false positive, false negative, and true negative instances, respectively. In addition to the above, we computed the so-called logistic loss (log-loss), also called binary cross-entropy, which is defined as  
\[\text{Log-loss}=-\sum_{i=1}^{N}[y_{i}\log\hat{y}_{i}+(1-y_{i})\log(1-\hat{y}_{ i})], \tag{11}\]  
where in the above \(N\) is the number of vectors in the dataset, \(\hat{y}_{i}\in[0,1]\) are the model's predictions, which correspond to the probability that a given vector belongs to the positive class and \(y_{i}\in\{0,1\}\) are the actual class labels. Finally, we computed the so-called area under the curve (AUC), which quantifies the balance between the false positive and true positive rates of the model [77]. Table 4 summarizes the above metrics for the two examined classifiers. The best-performing classifier, gradient boosting, achieved an overall accuracy of 95%. This fact indicates the overall success of the algorithm to accurately assess the safety of the rigid rocking block under seismic excitation. In addition, the classification metrics are approximately equal for both classes, enhancing the reliability of the predictions. Figure 3 presents the FP-TP rate curves (ROC curves) for ANN, while Figure 4 presents the corresponding curves for Gradient Boosting. It can be readily observed that a very high true positive rate can be achieved without a corresponding large false positive rate. Subsequently, Figure 5 shows the evolution of the training loss of the two examined classifiers. The training stops either when the maximum number of iterations is reached, or when the gain in the validation loss, i.e., the loss measured on a holdout subset of the training set, becomes too small compared to the gain in training loss, i.e., when there is the risk of overfitting.