# Data-Driven Pavement Performance: Machine Learning-Based Predictive Models
## 7 Discussion  
In this research, different machine learning algorithms have been used to perform performance predictions of asphalt pavement in terms of fatigue damage and rutting. Furthermore, the relationship and dependency between input and output variables have been analysed for 350 data points. K-fold optimization technique has been used to optimize the hyperparameters and the corresponding number of folds for each machine learning algorithm used. The machine learning algorithms used are linear regression, decision tree, random forest, gradient boosting, K-nearest neighbour, SVR, LightGBM and CatBoost. As observed from the results, CatBoost and LightGBM outperform other machine learning models in terms of reduced MSE and R\({}^{2}\) values. Furthermore, the hyperparameters for LightGBM and CatBoost can be optimized easily with fewer numbers of folds, leading to time efficiency in performance predictions. The least performance is exhibited by linear regression and in some instances by KNN. In the case of a higher number of passes used both for fatigue damage and rut depth, scatter in the results for SVR increases significantly. KNN and linear regression are also affected by a higher number of passes, where scatter in predicted values increases beyond 1.2 \(\times\) 10\({}^{6}\) passes, as shown in Figures 8c and 8d, respectively. Decision tree and random forest show better overall performance than the aforementioned models. CatBoost leads to the use of the minimum number of folds required to reach the minimum MSE of 0.001 in cases of rut depth at 1.6 million passes. Furthermore, LightGBM also exhibits a closer performance to CatBoost in terms of reduced MSE. The inclusion of categorial boosting and arrangement of hyperparameters for both of these models leads to higher prediction efficiency when compared to other models. Predictive performance of LightGBM is followed by gradient boosting and random forest. CatBoost performs best in all scenarios, whether it is the number of passes to fatigue and rut damage or fatigue and rut life reduction in number of years. Both LightGBM and CatBoost are specific implementations of the gradient boosting machine (GBM) algorithm, which builds trees sequentially by correcting the errors of the previous trees. This iterative correction process helps in reducing bias, leading to better performances as observed in Figures 7g and 7h, respectively. Moreover, LightGBM and CatBoost employ advanced techniques like histogram-based approaches and categorical feature handling (CatBoost), which significantly improve training speed and accuracy. These models are less prone to overfitting compared to simpler models like decision trees because they include regularization techniques like shrinkage (learning rate), early stopping and pruning. Regularization ensures that the models do not memorize the training data but generalize well to unseen data, which is crucial for achieving lower MSE and higher accuracy. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Machine Learning** & **MSE** & **R\({}^{2}\)** \\
**Algorithm** & 0.463450 & 0.9491 \\ Decision Tree & 0.017459 & 0.9981 \\ Random Forest & 0.011836 & 0.9987 \\ Gradient Boosting & 0.000901 & 0.9999 \\ KNN & 0.499082 & 0.9452 \\ Support Vector Regression & 0.030690 & 0.9966 \\ LightGBM & 0.008 & 0.999 \\ CatBoost & 0.001 & 1.000 \\ Linear Regression & 0.463450 & 0.9491 \\ Decision Tree & 0.017459 & 0.9981 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: MSE and R\({}^{2}\) values of different models for rut depth at 1.6 million passes. Linear regression assumes that there is a linear relationship between the input features and the target variables. However, in cases like predicting fatigue and rutting damage, the relationship is likely complex and non-linear with high scatter, as shown in Figure 9d. Linear models are unable to capture this complexity, leading to lower accuracy, higher scatter in predictions and increased MSE. This explains why linear regression performs poorly in this context.