# Hybrid Approach to the Cost Estimation of External-Fleet Full Truckload Contracts
## 4 Estimation Case Study  
The data used to evaluate and test the method originated from the databases for selected Polish transportation companies [35]. The original records for each contract included dozens of fields and features. The proposed approach aimed to incorporate into the algorithm the knowledge of the process; thus, the data fields (contract features) were filtered to leave the most influential ones (as described in the previous section). The original orders database consisted of approximately 583,000 orders. After pre-processing the 414,000 record remains, the own fleet contracts (fixed-price costs) and the erroneous entrances were removed. The data considered cover the time period from 1 January 2016 to 30 April 2022. These records were considered the training data. Records from 1 May 2022 till 1 August 2022 were considered as the validating dataset. It comprised 15,000 orders obtained after removing the own-fleet and erroneous ones from the original size of 25,000 records. The residuum analysis for the obtained custom kNN modeling was compared with the simultaneously prepared and tuned reference ML eXtreme Gradient Boosting (XGBoost) model [36]. Actually, four different kNN models were compared with the XGBoost model:  
**kNN(1)raw--**: the raw kNN model evaluated with the weights not tuned, i.e., for all \(i\), \(w_{i}=1\) and considering all the routes, for which there was at least **one** historical shipping record. **kNN(1)opt--**: the raw kNN model evaluated with all the weights optimized and taking into consideration all the routes, for which at least **one** historical shipping happened. **kNN(5)raw--**: the raw kNN model evaluated with the weights not tuned and taking into consideration all the routes, for which at least **five** historical shipping happened. **kNN(5)opt--**: the raw kNN model evaluated with all the weights optimized and taking into consideration all the routes, for which at least **five** historical shipping happened. **XGBoost--**: the XGBoost model. The residuum analysis started from the evaluation of the main model fitting indicators, i.e., the integrals and statistical factors of the prediction error. They are shown in Table 1. We see that the tuning of the model's hyper-parameters improved the model's quality. The model **kNN(5)opt** reached the best quality in terms of all indexes. The percentage index **pIAE** also indicates that even the **kNN(1)opt** model behaved better than the **XGBoost** one. One has to be aware that such simple and one-dimensional residuum analysis does not explain the nature of the model and the causes for the model's misfits. The analysis of the error histograms and their properties may deliver further information. Sample histograms for two models, **kNN(1)raw** and **kNN(5)opt**, are presented in Figure 5.  
They show the best and the worst model, respectively. We clearly may notice that normal distribution is not an appropriate approximation of the error stochastic properties. Robust estimator is better. Therefore, normal mean and standard deviations should not be \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **ISE** & **IAE** & **pIAE** & \(\sigma\) & \(\sigma_{\mathbf{R}}\) \\ \hline
**kNN(1)raw** & 1,207,139 & 561.7 & 16.54 & 1099 & 489.8 \\
**kNN(1)opt** & 1,111,817 & 492.3 & 14.94 & 1053 & 385.9 \\
**kNN(5)raw** & 921,361 & 490.0 & 13.01 & 960 & 456.5 \\
**kNN(5)opt** & 821,574 & 416.1 & 11.33 & 904 & 353.8 \\ \hline
**XGBoost** & 902,040 & 456.2 & 16.18 & 944 & 400.1 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The residuum analysis performed for all data taken as they are. Figure 5: Sample histogram plots with the fitted normal and robust Gaussian distributions. used. Following this line of thinking further, the MSE should also not be used, since, as [PERSON] has already shown, it is equivalent to the standard deviation. Thus, the IAE measure (absolute or percentage) and robust standard deviation are acceptable alternative measures of model quality.