# Fault Diagnosis for Wind Turbines Based on ReliefF and eXtreme Gradient Boosting
## 2 Introduction to the Relieff and XGBoost Algorithms  
### The Principle of the Relieff Algorithm  
If the dimension of the data sample is too large, the performance of the algorithm will be negatively affected by irrelevant or redundant features, resulting in problems such as dimensional disaster. Therefore, before the large-scale data operation, we had to select the characteristics of the data to reduce the sample dimension. The Relief algorithm, which was first proposed by [PERSON] in 1992, is a classic feature selection algorithm and it is mainly suitable for feature selection of binary classification [21]. The Relief algorithm draws on the ideas of the nearest neighbor learning algorithm and is widely used [22]. The Relief algorithm is a supervised feature evaluation method. Its core idea is to give weight to each feature parameter to be selected in the sample to represent the correlation degree between the parameter and the category. After calculating the weight, the parameter with larger weight is selected as the feature parameter to form the output feature set. The implementation principle of the algorithm is as follows. First, select a sample R randomly from the sample set, then select the nearest sample A from the same class set of sample R, and select the nearest sample B from different class sets of sample R. Then, calculate the distance between sample R and sample A and the distance between sample R and sample B in each feature dimension, respectively. If the distance between R and A in a feature dimension is less than the distance between R and B, it means that the feature is meaningful to distinguish the same samples and different samples, so the feature weight is increased; on the other hand, the feature weight is decreased. Finally, the total weight is obtained by accumulating the weights calculated by multiple iterations. The Relieff algorithm is an improvement on the Relief algorithm and is mainly used for feature selection of multi-class labels [18; 19]. The core idea of the Relieff algorithm is to judge the correlation between observed features and categories by calculating the weight of each feature. The larger the weight, the more sensitive it is to multi-class classification. The specific implementation process of the algorithm is as follows: Suppose you want to sample the sample set m times, and there are n observation features in the data set. In each sampling, first select sample R randomly from the sample set, then select k nearest samples from the same kind of samples of R, then select k nearest samples from the rest of each kind of samples, and then use Equation (1) to calculate the weight of each feature circularly. Since m times are sampled, the above process iterates m times, and finally gets the average weight of each feature. A feature subset is output by preserving the features whose weight is greater than the threshold value. The flow chart of the Relieff algorithm is shown in Figure 1.  
The equation to calculate the weight of each feature circularly is shown as  
\[ \begin{array}{l}\begin{array}{l}{\rm W}{(T)}_{i}=W{(T)}_{i-1}-\sum\limits_{j=1}^ {k}diff(T,R,H_{j})/(mk)\\ +\sum\
olimits_{C\in\text{class}(R)}\big{[}\frac{p(C)}{1-p(class(R))}\sum\limits_ {j=1}^{k}diff(T,R,M_{j}(C))\big{]}/(mk)\end{array}^{\text{,}}\end{array}  \tag{1}\]  
where \({\rm W}{(T)}_{i}\) represents the weight of the t-th feature in the i-th iteration, and the initial value is 0, \(H_{j}\) represents \(k\) nearest neighbor samples of the same kind as \(R\), \(M_{j}(C)\) represents \(k\) nearest neighbor samples different from \(R\), \(p(C)\) is the probability of class \(C\), \(diff(T,R,H_{j})\) represents the distance between sample \(R\) and sample \(H_{j}\) on feature \(T\), and \(diff(T,R,M_{j}(C))\) represents the distance between sample \(R\) and sample \(M_{j}(C)\) on feature \(T\). The distance equation is shown as  
Figure 1: The flow chart of the Relieff algorithm. \[diff(T,R_{1},R_{2})=\left\{