# Data-Driven Pavement Performance: Machine Learning-Based Predictive Models
## 7 Discussion
K-nearest neighbour works by finding the closest neighbours to a given data point and predicting the output based on their values. While this method can work well for simple datasets, it struggles with high-dimensional or large datasets, particularly when there is noise or irrelevant features. The model has no internal mechanism to handle feature interactions or non-linearity, and its predictions can be highly sensitive to the choice of k (the number of neighbours) and the distance metric. Moreover, KNN can suffer from issues such as overfitting or underfitting, especially if the data have many irrelevant or noisy features, as observed in Figure 10c where the scatter increases with the increase in rut depth magnitude. This leads to reduced prediction accuracy and high variance in its performance. Decision trees can model complex relationships and non-linear data patterns. However, they have a tendency to overfit the training data if not properly pruned or regularized with less scatter in terms of fatigue life reduction in number of years, as shown in Figure 7a, and rut depth at 1.6 million passes, as shown in Figure 10a. While decision trees provide interpretable results and perform well in capturing feature interactions, their performance often degrades when there is noise in the data. SVR is designed to model non-linear relationships by mapping the input data to a higher-dimensional space via a kernel function. While SVR can capture complex data patterns, it is highly sensitive to hyperparameters such as the choice of kernel, regularization parameters and the trade-off between bias and variance, leading to further deterioration in prediction performance with very high scatter with higher magnitudes of numbers, as shown in Figures 8f and 9f. If these parameters are not carefully tuned, SVR can underperform, especially in the presence of noisy data or with large datasets. SVR also requires careful preprocessing, like feature scaling, to perform optimally. Random forest generally performs better than a single decision tree because it averages multiple trees to reduce variance and overfitting. However, it does not take into account the errors of previous trees in the way that gradient boosting, LightGBM and CatBoost do, leading to less refinement in predictions, which can be observed in Figure 9e, in terms of number of passes to reach rut depth of 6 mm. While random forest is robust, less prone to overfitting and more interpretable than other models like SVR, it still cannot reach the accuracy of boosting-based methods in terms of predictive power, particularly in complex, high-dimensional datasets. Random forest benefits from the fact that multiple trees are averaged, reducing variance, but it does not have the advantage of boosting, which can systematically focus on difficult-to-predict cases. Gradient boosting models (like LightGBM and CatBoost) tend to perform better because they correct errors from the previous trees, which allows them to make more refined and accurate predictions. These models can be used for a variety of applications based on the availability of input parameters as used in this research; however, the effect of temperature variations as input variables and their correlation with output variables and the resulting fatigue cracking and rutting damage has not be considered as part of this research, including the use of statistical significance of each model type and error distribution in the prediction evaluation for each model which shall be included in the future research.