## 3 Results
### Optimizing the Model
#### 3.1.4 Decision Tree  
The hyperparameters of the decision tree model include maximum depth, minimum number of samples split, maximum number of features, and criterion [25]. Maximum depth represents the maximum depth of the decision tree, and in this study, it was set to 5, 6, and 7. By controlling how deep the tree branches can extend, it helps mitigate the risk of overfitting. The minimum number of samples split refers to the minimum number of samples required to split a node. Nodes with fewer samples than this value will not be split. The default value in this research was set to 3, 4, and 5. The maximum number of features limits the number of features available for splitting and can help control the complexity and overfitting of the model. In this study, a square root of the total number of features and logarithm base 2 of the total number of features was chosen. Criterion is the function used to evaluate the quality of a node's split and mean squared error, Friedman mean squared error, and Poisson loss were applied in this research.  
The lowest RMSE and MAE values were achieved when CV values were 3 and 8. For CV = 3, the test set had an R\({}^{2}\) of 0.82, an RMSE of 7.03 MPa, an MAE of 5.46 MPa, and Delta RMSE was 0.98 MPa. For CV = 8, the test set had an R\({}^{2}\) of 0.82, an RMSE of 7.06 MPa, and an MAE of 5.49 MPa with delta RMAE as 1.01 MPa. Since the performance metrics for CV  
\begin{table}
\begin{tabular}{c c c} \hline
**Model** & **Best Hyperparameter** & **CV** \\ \hline \multirow{4}{*}{Decision tree} & Maximum depth: 7 & \\  & Minimum number of sample split: 5 & \\  & Maximum number of features: square root & \\  & Criterion: mean squared error & \\ \hline \multirow{4}{*}{Random forest} & Number of estimators: 200 & \\  & Maximum depth: None & \\  & Minimum number of samples split: 2 & \\  & Number of minimum samples leaf: 1 & \\ \hline \multirow{4}{*}{XGBoost} & Number of estimators: 200 & \\  & Learning rate: 0.1 & \\  & Maximum depth: 5 & \\  & Number of subsamples: 0.5 & \\ \hline \multirow{4}{*}{LightGBM} & Number of estimators: 200 & \\  & Learning rate: 0.1 & \\ \cline{1-1}  & Maximum depth: 7 & \\ \cline{1-1}  & Number of subsamples: 0.5 & \\ \hline \end{tabular}
\end{table}
Table 3: _Cont._values for 3 and 8 came out very similar, CV value 3 was chosen as the optimal model for the decision tree. The optimal hyperparameters for CV = 3 are as follows: the maximum depth is 7, the minimum number of samples split was set as 5, the maximum number of features as a square root of the total number of features, and the criterion is mean squared error, as summarized in Table 3.  
The advantage of the tree model is the ability to visualize the model [80]. Figure 2 represents the optimal decision tree model for CV as 3. Interpreting the figure, the first splitting criterion is 'AGE'. It uses the 'AGE' feature to perform the first split. If the 'AGE' value is less than or equal to 21.0, it branches to the left; otherwise, it branches to the right. \"Squared error\" indicates the mean squared error in the split, representing the average squared difference between the predicted and actual values in the split. In the first split, the mean squared error is 254.458. 'Samples' represents the number of data points included in the split, which is 541 in this case. The value of 541 is obtained by multiplying the training set ratio (0.7) by the total number of data points (752). Finally, 'value' denotes the average value of the target variable predicted within the split, which is 36.753 for the first split, representing the average of the target variable values for the data points belonging to the first split.