# Influence of Social Overhead Capital Facilities on Housing Prices Using Machine Learning
## 3 The Proposed Scheme
### Modeling Process
#### 3.3.2 Optimal Model Determination  
Upon addressing multicollinearity, our next step involved evaluating the efficacy of eight distinct machine learning models using cross-validation. A concise overview of these models and their unique characteristics is provided in Table 4.  
Model validation is an integral component of the modeling process. Cross-validation, a foundational technique in machine learning, gauges a model's performance by partitioning the dataset into multiple training and test subsets. This approach is instrumental in preventing overfitting. However, our dataset presents a unique challenge due to its inherent time series nature. Resorting to standard cross-validation methods would be problematic, as it poses the risk of unintentionally leveraging future data to forecast past or present events. To navigate this intricacy, we adopt the Time Series Nested Cross-Validation (TS-Nested CV) strategy [67]. This specific validation approach is tailored for data with temporal dependencies. It ensures that the training dataset always precedes the validation set in chronological order. As such, instead of the conventional k-fold techniques, the TS-Nested CV is preferred. We implemented this process using the Time Series Split tool available in the scikit-learn library. A schematic depiction of the approach can be found in Figure 5. \begin{table}
\begin{tabular}{l l} \hline \hline
**Model** & **Description** \\ \hline Linear Regression & \begin{tabular}{l} A general linear regression learning model that reflects the correlation \\ between explanatory and dependent variables. \\ \end{tabular} \\ \hline Ridge Regression & \begin{tabular}{l} A linear regression learning model with L1 regularization. \\ \end{tabular} \\ \hline Lasso Regression & \begin{tabular}{l} A linear regression learning model with L2 regularization. \\ \end{tabular} \\ \hline ElasticNet Regression & \begin{tabular}{l} A linear regression learning model that combines both L1 and \\ L2 regularization. \\ \end{tabular} \\ \hline Decision Tree & \begin{tabular}{l} A tree-based learning model that branches in the direction of lower \\ impurity and learns to minimize this impurity. \\ \end{tabular} \\ \hline Random Forest & \begin{tabular}{l} Utilizing Bagging, this tree-based learning model selects variables \\ randomly, preventing overfitting typically seen in Decision Trees. \\ \end{tabular} \\ \hline XGBoost & \
\begin{tabular}{l} An ensemble tree-based learning model that utilizes boosting techniques. \\ It inputs the loss of the previous model into the learning data and uses \\ the gradient method to correct errors. \\ \end{tabular} \\ \hline LightGBM &
\begin{tabular}{l} A tree-based learning model that minimizes error loss by employing \\ methods like GOSS, EFB, and Leaf Wise. \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Overview of the eight machine learning models employed in the study, highlighting their unique characteristics and features. Figure 5: Nested cross-validation process: illustration of the step-by-step splitting of training and test datasets, emphasizing the hierarchical structure of hyperparameter tuning within the inner loop and performance evaluation in the outer loop. Choosing the right evaluation metric is crucial for assessing a model's performance accurately. These metrics offer quantitative insights into a model's predictive accuracy. In this study, the target variable 'SMUP' is derived from the housing sale price variable 'DLNG_AMOUNT'. As depicted in Figure 2, there exists a notable discrepancy exceeding a two-fold difference between the highest and lowest prices, indicating potential outliers that could significantly influence the sale price. When the Root Mean Squared Error (RMSE) is employed as a loss function, its inherent sensitivity to outliers is augmented, as it squares the residuals between the actual and predicted values. This can obscure whether predictions systematically underestimate or overestimate the actual values.