# Fault Diagnosis of Intelligent Production Line Based on Digital Twin and Improved Random Forest
## 2 Random Forest and Its Improvements
### Hierarchical Clustering Algorithm Rescreening Decision Trees  
The hierarchical clustering algorithm performs hierarchical decomposition according to the similarity between data points to create a nested clustering tree with a hierarchical structure. Whether a hierarchical clustering algorithm is agglomerative or divisive mainly depends on the process of hierarchical decomposition. The bottom-up hierarchical decomposition corresponds to the agglomerative method and the top-down hierarchical decomposition corresponds to the divisive method. Agglomerative hierarchical clustering has lower complexity than divisive hierarchical clustering [27]. The specific steps of agglomerative hierarchical clustering are as follows:  
1. Treat each dataset as a cluster, calculate the distance between each pair of data points, and finally obtain the distance matrix. 2. Find the two closest clusters according to the distance matrix, and then merge them. Update the distance matrix. 3. If the data are in one cluster or reach the preset number of clusters, stop. Otherwise, repeat step 2. In this paper, the linkage criterion used is average linkage, which can be expressed as follows:  
\[d_{\text{avg}}\big{(}C_{i},C_{j}\big{)}=\frac{1}{|C_{i}||C_{j}|}\sum_{x\in C_ {i}}\sum_{y\in C_{j}}\text{Dis}(x,y) \tag{1}\]  
where \(C_{i}\) and \(C_{j}\) are given clusters; Dis is the similarity calculation formula between two points, and the specific formula will be given below. The accuracy of the random forest algorithm is related to the base classifier. The existing conclusion shows that [28] the higher the prediction accuracy and the greater the difference between the base classifier, the better the ensemble effect. Conversely, the  
Figure 2: Typical structure of random forests. lower the accuracy and difference of the base classifier, the worse the final result will be. The higher the classification accuracy of each base classifier, the higher the confidence of the final voting results and the accuracy of the random forest algorithm. If the similarity between two base classifiers is high, there will be repeated voting, which will lead to a decrease in the accuracy of random forest. Therefore, we improve the traditional random forest algorithm through a hierarchical clustering algorithm to select a base classifier with high accuracy and low similarity. The algorithm flowchart is shown in Figure 3.  
If the initial data are \(X=\{x_{1},x_{2}\ldots x_{K}\}\), where sample \(x\in R^{m}\) and \(K\) is the number of samples, \(N\) training datasets \(\{X_{1},X_{2}\ldots X_{N}\}\) are obtained by bootstrap resampling. The corresponding decision tree is \(\{T_{1},T_{2}\ldots T_{N}\}\). The decision tree of this paper adopts C4.5, and the classification attribute of nodes adopts the attribute with the largest information gain rate. The generated classification rules are easy to understand and have a high accuracy rate. After the completion of RF construction, the accuracy of each tree is calculated through the test set. The difference between each pair of trees will be calculated to form a matrix S:  
\[\text{S}=\left[ \begin{array}{cc}a&b\\ c&d\end{array} \right] \tag{2}\]  
where \(a\) and \(d\) represent being classified correctly and incorrectly, respectively; \(b\) and \(c\) represent accurate classification by only one of the base classifiers at the same time. The difference between a pair of trees (\(T_{i}\) and \(T_{j}\)) is calculated by the following formula:  
\[\text{Dis}_{i,j}=1-\frac{a+d}{a+b+c+d} \tag{3}\]  
Figure 3: Flowchart of the algorithm. The distance matrix \(\mathrm{D}\) in hierarchical clustering is represented by the similarity between trees:  
\[\mathrm{D}=\left[ \begin{array}{ccc}Dis_{1,1}&\cdots&Dis_{1,N}\\ \vdots&\ddots&\vdots\\ Dis_{N,1}&\cdots&Dis_{N,N}\end{array} \right] \tag{4}\]  
The decision tree in the random forest is taken as a single sample point, and the dissimilarity between any two trees is calculated. According to the hierarchical clustering algorithm, the two trees with the smallest distance are found.