# A Predictive Pedestrian Crash Model Based on Artificial Intelligence Techniques
## 2 Data and Methods
### Methodology  
#### 2.2.1 The CatBoost Algorithm  
Gradient boosting is a decision-tree based algorithm that was developed in 1990 by [PERSON] [34] to obtain a strong learner as a combination of weak learners in an iterative way. Gradient boosting belongs to a family of powerful machine-learning techniques that have shown considerable success in a wide range of practical applications in terms of a non-parametric regression or classification model from the data [35]. Here, we adopted an open-sourced algorithm named CatBoost, which was developed by Yandex [36] within the Gradient Boosting algorithm family. CatBoost's name comes from the two words Category and Boosting. Specifically, CatBoost was selected because of its proficiency in working with data containing categorical variables [37; 38], such as those contained in the ISTAT dataset. CatBoost aims to reduce the prediction shift that occurs during training by dividing training and test instances to obtain the solution [37; 39]. The ith model of the mth iteration is trained on the first instances of the permutation and it is used to estimate the gradient of the (i + 1)th instance for the (m + 1)th boosting iteration. In order to be independent, the process starts with the use of s-different random permutations. The response variable is predicted via a recursive partitioning of the dataset (root node of the tree). After the first partition of the input space into sub-regions, the splitting continues. Each time the sub-regions are split, a decision node (or internal node) is created in the tree. When a sub-region is not further partitioned, it represents a final node of the tree (commonly called the leaf of the tree). A response variable value or a function corresponds to each leaf. In this application, only decision trees with constant values at the final nodes were used. An example of a \"binary and symmetrical\" decision tree of depth 2 is shown in Figure 2.  
A weak learner has a predictive power just slightly better than chance. Decision trees can be considered weak learners and are widely used in the field of computer science. For classification tasks, the response variable can be interpreted as the probability of belonging to a certain class. To overcome this weakness, in gradient boosting, multiple decisions trees are fitted in series. After the first tree is fitted, the residuals between the target variables and the predicted values are computed and a new tree is fitted using the residuals as a target. This procedure continues until the stopping criterion is met. The stopping criterion used in this research was the number of trees to be fitted (1000). A visual representation of the process is shown in Figure 3.  
In CatBoost, categorical variables are automatically substituted by a numeric feature that assigns a \"default\" value to each category (without any data pre-processing). In order to avoid over-fitting the training data, these values would ideally need to be computed using a different dataset. The procedure proposed in the CatBoost model for each permutation is similar to the one followed for building the models. Therefore, for each random permutation, the information of instances < i is used to compute the value assigned at step i. Then, after several permutations, the process allows the final average value to be obtained. Some characteristics of the model were usually adjusted by the modeler, for example, the number of learning trees, the depth of each tree, and the number of samples in each leaf. These take the name of hyper-parameters. The tuning of the hyper-parameters strongly  
Figure 3: Series of decision trees. Figure 2: Representation of a general binary decision tree. influences the performance of the model, and it is essential to guarantee an optimal fit. The tuning of the hyper-parameters can be performed manually by the modeler through a grid search or using an algorithm that performs a random search. In this research, the manual tuning of three hyper-parameters was performed on the learning rate, the tree depth, and the number of trees.