# Fault Diagnosis of Intelligent Production Line Based on Digital Twin and Improved Random Forest
## 4 Experiments and Analysis
### Results Analysis  
[PERSON] [28] proved the effectiveness of screening irrelevant high-performance trees for classification tasks in random forests. [PERSON] [28] used AUC accuracy as an indicator to evaluate the performance of each tree, then selected good decision trees from a large number of trees according to the value of high AUC accuracy. The selected trees are clustered according to the correlation between the trees. As a result of clustering, each cluster contains similar or highly correlated group of trees. We refer to the settings in [28]. In the initial state, \(M\) decision trees are generated for classification. Then, through the hierarchical clustering algorithm, the decision tree with the highest accuracy in each cluster is selected to form an improved random forest, in which there are \(N\) decision trees. \(M\):\(N\) = 3:1. For example, if there are 100 trees in the IRF, then there are 300 trees in the initial random forest. Table 2 shows the comparison between the RF and DT + IRF algorithms using the dataset in this paper. Figure 6: Fault diagnosis system. Next, we used the DT + IRF algorithm for the fault diagnosis analysis of the automobile rear axle assembly line. After data preprocessing, 300 classification trees were generated [40; 41]. Figure 7 shows the relationship between the accuracy of random forest and the number of decision trees. It can be seen from Figure 7 that, when the number of decision trees is greater than 100, even if the number of decision trees is greatly increased, the accuracy change is very small. The complexity of constructing a random forest is proportional to the number of decision trees. There are not as many decision trees as possible. With the increase in the number of decision trees, the computational efficiency will continue to decline, and the time spent will continue to increase. When the number of decision trees is 300, the accuracy rate is 90.8%, and then it fluctuates slightly. At the same time, when the number of decision trees reaches a certain level, the interpretability of the random forest will be weakened. Therefore, we set the number of initially generated decision trees to 300, and the number of decision trees to 100 after hierarchical clustering. According to Section 3.2, the training set was used to train the model to form a random forest through bootstrap sampling, and the accuracy of each base classifier was calculated through the test set. Figure 8 gives the accuracy of each classification tree in the random forest. It can be seen that the classification accuracy of each base classifier is roughly between 65% and 75%, and the accuracy of individual classification trees can reach about 80%. There are also individual classification trees whose accuracy rate is low, and the classification accuracy rate is only about 60%, which is not accurate enough for fault diagnosis. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Datasets Trees** & \multicolumn{2}{c}{**Physical Shop Floor + Digital Twin Shop Floor**} \\  & **RF** & **DT + IRF** \\ \hline
10 & 61.4\% & 74.9\% \\
20 & 78.5\% & 84.7\% \\
30 & 84.0\% & 90.3\% \\
40 & 84.8\% & 92.9\% \\
50 & 86.1\% & 94.2\% \\
60 & 86.9\% & 94.9\% \\
70 & 87.2\% & 95.7\% \\
80 & 88.1\% & 96.5\% \\
90 & 88.4\% & 97.1\% \\
100 & 88.7\% & 97.8\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of prediction accuracy between DT and DT + IRF algorithms. Figure 7: Accuracy of different decision trees in random forest. Therefore, we used the proposed method to cluster the trained classification trees according to the similarity of the classification results through hierarchical clustering, and then selected the batch of classification trees with the highest prediction accuracy from each category to participate in the integration. We set the cluster of clustering to 100 by validation [40; 41].