# Applying Machine Learning to Preselective Weighing of Moving Vehicles
## 2 Basic Classification and Characteristics of Machine Learning Algorithms
While the Gradient Boosting model is highly flexible, it requires proper tuning to maintain generalization capability. One technique to optimize the Gradient Boosting method while preserving generalization capability is LightGBM (Light Gradient Boosting Machine) [19]. Developed by Microsoft, LightGBM implements Gradient Boosting with advanced optimization techniques [20]. The LightGBM algorithm is distinguished by its use of the Leaf-wise Growth technique. Unlike classical decision trees, which are built level by level, this technique selects and splits the leaf with the greatest potential to reduce error, regardless of its level in the tree. This approach creates trees that can be more asymmetric and deeper in areas where data patterns are more complex. Another technique employed by LightGBM is Gradient-based One-side Sampling (GOSS). This method reduces the number of samples used to evaluate gradients, significantly speeding up calculations in the decision model without compromising accuracy. The final technique in the LightGBM algorithm is Exclusive Feature Bundling (EFB). EFB combines features to reduce data dimensionality, accelerating computations further. In general, the decision tree structure in the LightGBM algorithm takes the following form:  
\[\mathrm{H_{T}(x)}=\sum_{\mathrm{t}=1}^{T}\mathrm{H_{t}(x)}\quad\mathrm{H_{t} \epsilon I} \tag{5}\]  
In Equation (5), \(\mathrm{H_{T}}\) represents the t-th learner, and I is the space of the set of all learners. A concrete implementation of the LightGBM algorithm is as follows: LightGBM continuously improves the learner's performance through multiple iterations, using the learner to obtain a mapping function from the input space \(\mathrm{X_{s}}\) to the gradient space. It is assumed there is a training dataset \(\mathrm{x_{1}}\ldots\mathrm{x_{n}}\) with a data volume of n, where \(\mathrm{x_{i}}\) is a vector of the ith dimension s in the \(\mathrm{X_{s}}\) space. If the learner obtained in the previous iteration is \(\mathrm{H_{(t-1)}(x)}\), and the loss function is \(\mathrm{L(y,H_{(t-1)}(x))}\), the goal of this iteration is to find the learner \(\mathrm{H_{t}(x)}\) that minimizes the loss function for this iteration:  
\[\mathrm{h_{T}(x)}=\underset{\mathrm{h\epsilon H}}{\mathrm{argmin}}\mathrm{L(y,H_{t-1}(x)+h_{t}(x))} \tag{6}\]  
The negative gradient of the loss function is calculated to approximate the current value of the loss function iteration, which can be expressed as follows:  
\[\mathrm{r_{t}}=\frac{\partial\mathrm{L(y,H_{t-1}(x))}}{\partial\mathrm{H_{t-1 }(x)}} \tag{7}\]  
Using the squared difference approximation method to fit the function \(\mathrm{h_{t}(x)}\), we obtain the following:  
\[h_{t}(x)=argmin\sum(r_{t}-h_{t}(x))^{2} \tag{8}\]  
The final strong learner in the tree is as follows:  
\[H_{t}(x)=h_{t}(x)+H_{t-1}(x) \tag{9}\]  
As shown above, only the leaf (learner) in the tree that contributes most to improving the model's quality (the strong learner in the tree) is developed. This technique results in trees that are more asymmetrical and complex in structure. However, creating such deep, asymmetrical trees can lead to overfitting, where the model becomes too tailored to the training data and fails to generalize well on test data. A more scalable and efficient machine learning algorithm is XGBoost (eXtreme Gradient Boosting). XGBoost is one of the most advanced and efficient implementations of Gradient Boosting, designed with scalability and speed in mind. The algorithm primarily uses a training set to predict future changes and trends in target variables [21]. The essence of XGBoost lies in constructing multiple decision trees. Each tree predicts independently, and the final prediction value is obtained by combining the results of all the trees. Using a decision tree as the basic learner, XGBoost constructs multiple weak learners and trains the model iteratively along the decreasing gradient direction. The operation structure of the XGBoost decision model is depicted in Figure 2 (self-created based on [19]).