# Applying Machine Learning to Preselective Weighing of Moving Vehicles
## 4 Vehicle Weight Estimation Model Using Machine Learning Algorithm
### Development of a Model for Vehicle Weight Estimation Using the XGBoost Algorithm  
The authors aim to develop a machine learning model capable of estimating the total weight of a vehicle using the characteristics presented in Table 1. To achieve this, the XGBoost algorithm was utilized. This algorithm is known for its advanced and efficient  
Figure 5: Histogram of the distribution of vehicle-speed values. Figure 6: Histogram of the distribution of countries in which vehicles were registered. implementation of gradient boosting, designed for scalability and high-speed processing of large datasets, as discussed in Section 2 of this article. Our primary objective was to develop a machine learning (ML) model capable of estimating the total weight of a vehicle using only the features highlighted with blue, green, and white backgrounds in Table 1. For this regression task, we chose to employ Extreme Gradient Boosting (XGBoost), a powerful algorithm that applies Gradient Boosting to decision tree regressors. XGBoost works by constructing an ensemble of weak models, each trained to predict the target variable. The final prediction is obtained by taking a weighted sum of the predictions from these weak models (Figure 7). The weights are determined during the XGBoost training process. The weak models, which together form the XGBoost model, are trained using the Gradient Boosting algorithm. The taxonomy of the XGBoost model is illustrated in Figure 7. Since its release in 2014, the XGBoost algorithm has become one of the most efficient ML models for regression and classification tasks. It is widely used in both academic and industrial contexts, as highlighted in Section 3. The algorithm has gained significant recognition in numerous machine learning and data mining competitions, including those hosted by Kaggle. For instance, among the 29 winning solutions published on the Kaggle blog in 2015, 17 solutions employed XGBoost. Eight of these relied solely on XGBoost to train the model, while the others combined XGBoost with neural networks in team solutions. The second most popular method, deep neural networks, was used in only 11 solutions. XGBoost's success was further demonstrated at the 2015 KDDCup, where all top-ten winning teams utilized the algorithm. To create the ML model, a Python implementation of the XGBoost regressor from the xgboost package [28] was used. All model parameters were kept at their default values, with the maximum decision tree depth set to 10. Two separate models were trained and tested using subsets of data from the Grodziec and Osowiec locations. The models were evaluated on unseen data using standard regression metrics:  
* Root Mean Squared Error (RMSE);
* Mean Absolute Percentage Error (MAPE);
* R\({}^{2}\) (coefficient of determination). The results, along with the number of training and test samples, are presented in Table 2. \begin{table}
\begin{tabular}{c c c c c c} \hline \multirow{2}{*}{**Location ID**} & **Number of** & **Number of** & \multirow{2}{*}{**R\({}^{2}\)**} & \multirow{2}{*}{**RMSE**} & \multirow{2}{*}{**MAPE [\%]**} \\  & **Training** & **Samples to be** & & & \\  & **Samples** & **Tested** & & & \\ \hline Grodziec & 153,914 & 38,479 & 0.941 & 8.289 & 12.5 \\ Osowiec & 200,080 & 50,020 & 0.956 & 3.123 & 11.7 \\ \hline \end{tabular}
\end{table} 
Table 2: Performance results of the XGBoost algorithm in the machine learning model, including the number of training and test samples [own elaboration]. Figure 7: The taxonomy of the XGBoost used in the model for estimating the weight of vehicles in traffic. The results in Table 2 indicate the strong performance of both machine learning models in estimating the weight of passing vehicles based on the features in Table 1. The R\({}^{2}\) values, which measure the quality of the model's fit to the data, are high for both the Grodziec and Osowiec locations.