## 4 Random Forests Theory
### REF Algorithm  
**Definition 1**.: _The random forests model \(f\) is a set of decision trees {h(X, \(\theta_{k}\)), \(k\) = 1, 2,... \(N_{true}\), and the classifier \(h\)(X, \(\theta_{k}\)) is an unpruned decision tree constructed with a decision tree algorithm._  
\(\theta_{k}\) is a random vector independent and identically distributed with the \(k\)th decision tree, representing the growth process of the tree; the final classification value of the random forests is obtained by the majority voting method.  
**Definition 2**.: _For the input vector \(X\), it contains at most \(J\) different categories, let \(Y\) be the correct classification category, and for the input vector \(X\) and output \(Y\), define the edge function as:_  
\[\begin{array}{ll}K(X,Y)=&a_{k}I(h(X,\theta_{k})=Y)\\ &-\underset{j\
eq Y}{\max}a_{k}I(h(X,\theta_{k})=j)\end{array} \tag{3}\]  
In the formula: \(j\) is one of the \(J\) categories; \(I(.)\) is the indicator function; \(a_{k}\) is the average function; \(k=1\), \(2\), \(\ldots\)\(n\). The larger the edge function, the higher the confidence of the correct classification. The generalization error of random forests is thus defined as:  
\[E^{*}=P_{X,Y}(K(X,Y)<0) \tag{4}\]  
In the formula: \(P_{X,Y}\) is the classification error probability function for a given input vector \(X\). When the number of decision trees in the forest is large, the following theorem is obtained by using the law of large numbers:  
**Theorem 1**.: _When the number of decision trees is large enough, for all sequences \(\theta_{k}\), \(E^{*}\) converges almost everywhere:_  
\[\begin{array}{ll}\underset{k\rightarrow\infty}{\lim}E^{*}=P_{X,Y}(P_{ \theta}(h(X,\theta)=Y)-\\ \underset{j\
eq Y}{\max}P_{\theta}(h(X,\theta)=j)<0)\end{array} \tag{5}\]  
In the formula: \(P_{\theta}(c)\) is the probability of satisfying the condition \(c\) for a given sequence \(\theta\). This theorem shows that the generalization error of random forests will not cause overfitting as the number of trees increases, but will tend to an upper bound.  
**Theorem 2**.: _The upper bound of the random forests generalization error is:_  
\[E^{*}\leq\frac{\rho(1-s^{2})}{s^{2}} \tag{6}\]  
In the formula: \(\rho\) and \(s\) are the average correlation coefficient and average strength of the tree. It can be seen from Theorem 2 that with the decrease in the correlation of decision trees and the increase in the strength of a single decision tree, the upper bound of the generalization error of random forests will decrease, and its generalization error will be effectively controlled.  
Guaranteed by the law of large numbers, random forests has a high classification accuracy without overfitting, which is extremely suitable for power source fault diagnosis. From the above analysis, there are two main ways to improve the diagnostic accuracy performance of random forests fault diagnosis model, namely, reducing the correlation \(\rho\) of decision trees and improving the fault diagnosis performance \(s\) of a single decision tree. In addition, an important feature of random forests is out-of-bag estimation. When a training subset is generated by bagging, for each decision tree, nearly 37% of the samples in the original sample set \(S\) will not appear in the other trees. In the training subset, these samples are called out of bag (OOB) samples. OOB samples can be used to estimate the generalization error of RFs and also to calculate the importance of each feature. The above theorems are the key factors to reduce the fault misdiagnosis in this paper.