# Fault Diagnosis for Wind Turbines Based on ReliefF and eXtreme Gradient Boosting
## 2 Introduction to the Relieff and XGBoost Algorithms
### The Principle of the XGBoost Algorithm
When selecting the bifurcation feature, the XGBoost algorithm uses the \"greedy strategy\" to select the feature with the smallest target function value at the current moment as the bifurcation feature by enumerating the target function value. When calculating the predicted score of each leaf node, the XGBoost calculates the minimum value of the objective function, and the minimum value point is the predicted score of the leaf node. Because the XGBoost algorithm uses a gradient boosting strategy internally, when constructing a classification regression tree, not all the trees are obtained at once, but a new tree is added each time, and the previous test results are continuously patched while adding new trees. Assume that the predicted value after generating t trees is \(\hat{y}_{i}{}^{(t)}\). The derivation process is shown as  
\[\left\{ \begin{array}{l}\hat{y}_{i}{}^{(0)}=0\\ \hat{y}_{i}{}^{(1)}=f_{1}(x_{i})=\hat{y}_{i}{}^{(0)}+f_{1}(x_{i})\\ \hat{y}_{i}{}^{(2)}=f_{1}(x_{i})+f_{2}(x_{i})=\hat{y}_{i}{}^{(1)}+f_{2}(x_{i}) \\ \vdots\\ \hat{y}_{i}{}^{(t)}=\sum\limits_{k=1}^{t}f_{k}(x_{i})=\hat{y}_{i}{}^{(t-1)}+f_ {t}(x_{i})\end{array} \right.. \tag{6}\]  
Therefore, the objective function of each layer is shown as  
\[ \begin{array}{l}\text{Obj}^{(t)}=\sum\limits_{i=1}^{n}l(\hat{y}_{i}{}^{(t) },y_{i})+\sum\limits_{i=1}^{t}\Omega(f_{i})\\ =\sum\limits_{i=1}^{n}l(\hat{y}_{i}{}^{(t-1)}+f_{t}(x_{i}),y_{i})+\Omega(f_{t}) \end{array} . \tag{7}\]The purpose of Equation (7) is to find a way to minimize the objective function. The Taylor expansion of the objective function at \(f_{t}=0\) is shown in Equation (8)  
\[\text{Obj}^{(t)}=\sum\limits_{i=1}^{n}\Bigl{[}l(\hat{y}_{i}^{(t-1)},y_{i})+g_{i }f_{t}(x_{i})+\frac{1}{2}h_{i}f_{t}^{2}(x_{i})\Bigr{]}+\Omega(f_{t}), \tag{8}\]  
where \(g_{i}=\partial_{\hat{y}_{i}^{(t-1)}}l(y_{i},\hat{y}_{i}^{(t-1)})\) is the first derivative and \(h_{i}=\partial_{\hat{y}_{i}^{(t-1)}}^{2}l(y_{i},\hat{y}_{i}^{(t-1)})\) is the second derivative. Delete the constant term and get the objective function of step t as shown as  
\[\text{Obj}^{(t)}=\sum\limits_{i=1}^{n}\Bigl{[}g_{i}f_{t}(x_{i})+\frac{1}{2}h _{i}f_{t}^{2}(x_{i})\Bigr{]}+\Omega(f_{t}). \tag{9}\]  
Put \(\Omega(f_{t})\) into Equation (9) and the object function is shown as  
\[ \begin{array}{l}\text{Obj}^{(t)}=\sum\limits_{i=1}^{n}\Bigl{[}g_{i}w_{q}(x _{i})+\frac{1}{2}h_{i}w_{q}^{2}(x_{i})\Bigr{]}+\gamma T+\frac{1}{2}\lambda\sum \limits_{j=1}^{T}w_{j}^{2}\\ =\sum\limits_{j=1}^{T}\Bigl{[}G_{j}w_{j}+\frac{1}{2}(H_{j}+\lambda)w_{j}^{2} \Bigr{]}+\gamma T\end{array} , \tag{10}\]  
where \(G_{j}=\sum_{i\in I_{j}}g_{i}\), \(H_{j}=\sum_{i\in I_{j}}h_{i}\), \(I_{j}=\{i|q(x_{i})=j\}\) represents the sample set belonging to the j-th leaf node. The minimum value of the objective function of the aforementioned leaf node score \(w\) can be obtained by making its derivative zero. The minimum value point is the optimal leaf node prediction score and is shown as  
\[w_{j}^{*}=-\frac{G_{j}}{H_{j}+\lambda}. \tag{11}\]  
Take Equation (11) into the objective function, and the minimum value of the objective function is shown as  
\[\text{Obj}^{*}=-\frac{1}{2}\sum\limits_{j=1}^{T}\frac{G_{j}^{2}}{H_{j}+ \lambda}+\lambda T. \tag{12}\]