# Fault Detection for Pitch System of Wind Turbine-Driven Doubly Fed Based on IHHO-LightGBM
## 2 Light Gradient Boosting Machine  
The gradient boosting decision tree (GBDT) is one of the most widely used machine learning models in industry and it aims to iteratively train the weak classifier (decision tree) [33], to achieve an optimized model with a good training effect and hardly any over-fitting. On the contrary, GBDT will take much more time to reach the same training effect when faced with massive amounts of industrial data. The original idea that was put forward by LightGBM was to improve the performance of GBDT with massive amounts of data, to make the GBDT more suitable for industrial practice.  
Since the light gradient boosting machine (LightGBM) [34] was released by Microsoft Corporation in 2017, its performance has immediately attracted wide attention. LightGBM is a framework for implementing the GBDT algorithm. On the basis of XGBoost [35], LightGBM proposes a decision tree algorithm based on Hisgram, adopts gradient-based one-side sampling, uses the leaf growth strategy with depth restriction, and supports efficient parallel training, which makes LightGBM have a better training speed, less consumption, and better accuracy, making it more suitable for processing massive amounts of data. Figure 1 is a schematic diagram of a decision tree algorithm based on Hisgram, adopts GOSS. Figure 2 is a comparison of two tree growth methods.  
When given a supervised learning dataset \(X=\left\{(X_{i},y_{i})\right\}_{i=1}^{N}\), LightGBM needs to find a mapping relationship \(F(\hat{x})\) to approximate the function \(F(x)\), to minimize the expected value of the loss function \(\Psi(y,F(x))\).  
\[\hat{F}=\ \operatorname*{argmin}_{F}E_{y,x}\Psi(y,F(x)), \tag{1}\]  
At this time, the regression tree \(\sum_{t=1}^{T}f_{i}(x)\) is used to approximate the final model.  
\[F_{T}(X)\ =\ \sum_{t=1}^{T}f_{i}(x), \tag{2}\]The regression tree can be expressed in another way, which is \(w_{q(x)},\in\ \left\{1,\ 2,\ldots,J\right\}\), in which \(J\) represents the quantity of leaf nodes, \(q\) represents the decision rules of the tree, \(w\) represents the sample weight, and the objective function \(Obj^{(t)}\) could be represented as follows:  
\[Obj^{(t)}=\sum_{d=1}^{n}\Psi(y_{i},F_{t-1}(x_{i})+f_{t}(x_{i}))+\sum_{k}\Omega (f_{k}), \tag{3}\]  
where \(\Omega(f_{k})\) represents a regular item.  
LightGBM apply the Newton-Raphson method to quickly approach the objective function, and the following can be obtained:  
\[Obj^{(t)}\cong\sum_{i=1}^{n}\left[g_{i}f_{t}(x_{i})+\frac{1}{2}h_{i}f_{t}^{2} (x_{i})\right]+\sum_{k}\Omega(f_{k}), \tag{4}\]  
Among them, \(g_{i}\), \(h_{i}\) represent the loss function of first order and second order, respectively, i.e., the following:  
\[g_{i}=\partial_{F_{t-1}(x_{i})}\Psi(y_{i},F_{t-1}(x_{i}), \tag{5}\]  
\[h_{i}=\partial_{F_{t-1}(x_{i})}^{2}\Psi(y_{i},F_{t-1}(x_{i}), \tag{6}\]  
The sample set of leaf \(j\) can be represented by \(I_{j}\), Equation (6) can be simplified as follows:  
\[Obj^{(t)}\cong\sum_{i=1}^{n}[g_{i}f_{t}(x_{i})+\frac{1}{2}h_{i}f_{t}^{2}(x_{ i})]+\sum_{k}\Omega(f_{k}), \tag{7}\]  
Figure 1: A decision tree algorithm based on Hisgram, adopts GOSS.  
Given the tree structure \(q(x)\), the limit value of the best weight sum \(w_{j}^{*}\) and \(L_{T}^{*}\) of each leaf node can be obtained by quadratic programming, as follows:  
\[w_{j}^{*}=-\frac{\sum_{i\in I_{j}}g_{i}}{\sum_{i\in I_{j}}h_{i}+\lambda}, \tag{8}\]  
\[L_{T}^{*}=-\frac{1}{2}\sum_{j=1}^{J}\frac{\left(\sum_{i\in I_{j}}g_{i}\right)^ {2}}{\sum_{i\in I_{j}}h_{i}+\lambda}, \tag{9}\]  
The calculation formula of gain is as follows:  
\[G=\frac{1}{2}\left[\frac{\left(\sum_{i\in I_{j}}g_{i}\right)^{2}}{\sum_{i\in I _{j}}h_{i}+\lambda}+\frac{\left(\sum_{i\in I_{B}}g_{i}\right)^{2}}{\sum_{i\in I _{B}}h_{i}+\lambda}-\frac{\left(\sum_{i\in I_{j}}g_{i}\right)^{2}}{\sum_{i\in I _{B}}h_{i}+\lambda}\right], \tag{10}\]  
Figure 2: Comparison of level-wise tree growth and leaf-wise tree growth.  
Obviously, the LightGBM algorithm is better for processing high-latitude data than the GBDT algorithm.