## 4 Methodology
### Machine Learning Models
#### 4.1.2 GBDT Model  
GBDT is a common basic machine learning algorithm based on boosting iterative decision trees that can be used for classification, regression, and other problems. The steps of the algorithm first train a decision tree with an initial sample, and then train the next decision tree based on the residuals of the previous decision tree, thus gradually reducing the iterative residuals, thus improving the iterative results and finally constructing a new combinatorial model. When all the convergence criteria are satisfied, all the base models are obtained. In the GBDT classification algorithm, the final output is discrete categories rather than continuous values, so we cannot fit the category output error directly from the output categories. To solve this difficulty, we fit the loss function using a method similar to the log-likelihood loss function used in logistic regression. The dataset of this experiment has a relatively large number of features, and the traditional linear model cannot achieve a better classification effect. GBDT, as one of the representative models of integrated learning, consists in training multiple weak learners and synthesizing their classification results to form a strong learner. Figure 18 shows the flowchart of the construction of the GBDT model for the binary classification of the traffic black spot recognition.  
Figure 18 consists of multiple weak learners. X represents the feature attributes, Y represents the prediction results of the weak classifiers, and the results of multiple weak learners are combined to obtain the final classification results.  
The traffic black spot recognition model is constructed using the GBDT model, which is an integrated learning method based on a decision tree, and its main idea is to construct a strong classifier by combining multiple weak classifiers. When building the GBDT model, it is necessary to select the hyperparameters such as the depth of the decision tree and the number of leaf nodes, as well as to adjust the parameters such as the learning rate and the loss function. The selection and description of model parameters are shown in Table 15.  
\begin{table}
\begin{tabular}{c c c} \hline
**Parameter Name** & **Hidden Meaning** & **Usage** \\ \hline loss & Loss function & Deviance log-likelihood function \\ \hline max\_depth & Growth depth of each decision tree & Exponential loss function \\ \hline n\_estimators & Number of iterations of the weak learner & A positive integer \\ \hline learning\_rate & Learner learning step size & A positive integer \\ \hline \end{tabular}
\end{table}
Table 15: Important parameters of the GBDT algorithm model.  
Figure 18: GBDT traffic black spot identification classification model construction process.  
Gradient Boosting Classifier in the machine learning Sklearn toolkit was used to model the GBDT algorithm, with loss as the loss function, and then 2519 data points were randomly divided into test and validation sets, and the parameters of the algorithmic model were appropriately tuned using the grid search method, including the parameters of n_estimators, max_depth, and so on.  
As shown in Table 16, the values of the loss parameter are set to deviance and exponential, respectively, and it can be seen that the model is slightly more accurate when the value of the loss parameter is set to deviance.  
According to Figure 19, the accuracy of the model is higher when the n_estimators parameter in the GBDT model is set to 23.  
Moreover, the results of the grid search method show that the accuracy of the model is better when the other parameter, max_depth, is set to 7 and when min_samples_leaf is set to 1. Table 17 shows some of the parameter settings of the GBDT model, on the basis of which the GBDT model is used for the identification study of traffic black spots.