## 3 Fault Diagnosis of the Protection System Based on Gradient Boosting Decision Tree (GBDT)
### Principle and Training Steps of Gradient Boosting Decision Tree  
GBDT intelligent algorithm belongs to the ensemble learning algorithm. The ensemble learning algorithm is a hot topic in the field of engineering applications. It is a method to improve the learning ability through the combination of multiple weak learners [16]. Compared with conventional methods, it has a good performance in terms of accuracy and generalization ability. Bagging and Boosting algorithms are two typical ensemble learning algorithms. The schematics are shown in Figures 1 and 2. The bagging algorithm generates \(n\) training sample sets from the total sample library according to the random sampling method with playback. Each sample set trains a weak learner and uses the sample set to train \(n\) weak learners. The weak learners run in parallel. According to different combination strategies, \(n\) weak learners are combined to generate strong learners. The boosting algorithm is an inherited algorithm, in which the weak learners operate in a serial manner. The data weight in the training set of each iteration is changed by the learning results of the weak learners. The learning results are fitted according to the residuals, and then \(n\) weak learners are combined according to different combinations to generate strong learners [17]. The gradient boosting decision tree (GBDT) intelligent algorithm is one of the most widely used boosting algorithms in the engineering field, and it combines the sampling idea of the bagging algorithm, allowing sampling samples and features to increase the independence between weak learners. The GBDT intelligent algorithm does not change the sample weight in the iteration process, but continuously learns the negative gradient of the loss function, generates multiple new weak learners, and combines multiple weak learners into strong learners. Compared with the traditional machine learning algorithm, the GBDT intelligent algorithm can achieve higher accuracy in many of the application scenarios, and has a faster operation speed, stronger generalization ability, and lower requirements for parameter adjustment. Regardless of whether the GBDT intelligent algorithm performs regression tasks or classification tasks, its core idea is \"gradient boosting\", and the negative gradient of the loss function in the iterative process is shown in Formula (10):  
\[\widetilde{y}_{i}=-\frac{\partial L(y_{i},F(x_{i}))}{\partial F(x_{i})},\;i=1, 2,\ldots,\mathrm{K} \tag{10}\]  
Figure 1: General schematic diagram of bagging algorithm. Figure 2: General schematic diagram of boosting algorithm. In the above formula, \(\widetilde{y_{i}}\) is the negative gradient of the current loss function, namely the fitting target of the next iteration, \(L(y_{i},F(x_{i}))\) is the current loss function, \(y_{i}\) is the learning target value of the current weak learner, \(F(x_{i})\) is the output value of the current weak learner, \(x\) is the input variable (refers to the fault feature information in this paper), K is the number of training samples, and \(i\) is the current training sample. GBDT multi-classification is an organic combination of the GBDT binary classifier. In the training process, the idea of 'one positive class, multiple negative classes' is adopted. There are 12 kinds of fault types in the fault set. In the training process of a single sample, when the sample is the \(i\)-th type of fault (\(i=1,2,\ldots,12\)), it is assumed that the fault type of this sample is from 1 to 12, and each time it is assumed that the other 11 types of fault are unified as the negative samples of this sample, and 12 binary classifiers are trained to generate independent classification. Then, according to the real fault type of the sample, only the output result of one binary classifier (assuming that the i-th class is a positive class) is correct, and the rest are errors, then the final classification result is the fault type corresponding to this binary classifier.