# Overlap in Automatic Root Cause Analysis in Manufacturing: An Information Theory-Based Approach
## 5 Evaluation
### Evaluation Results
#### 5.2.2 Stochastic Simulation  
This section focuses on evaluating the performance of the proposed solution as a whole. The root causes included in the simulated data are depicted in Figure 6. Each dataset corresponds to a period of four months. On the left side of the figure, we can see the temporal distribution of the different root causes. The first two root causes are located at the beginning of the simulated period, while the last two are at the end. This concentration of root causes was (i) to mirror the imbalance between normal products (the vast majority) and problematic products (a small minority) that exist in real data, and (ii) to simulate the simultaneous occurrence of root causes, which may also occur in real datasets. The right side of the figure explains in more detail each root cause and the reason why it was used. RC1 is a failure in part of equipment or a machine. RC2 is a standard malfunction in a single machine when performing a single function (or step). RC3 is a simultaneous failure that occurs at the same time as RC4. This last root cause is a failure that happens when a machine is malfunctioning in all its functions/steps.  
Table 5 presents the results of the RCA algorithms used, divided by dataset, and each moment where it is possible to detect a problem. Each column represents the root causes detected, and in brackets, the position in the ranking of variables. The code of the root causes is explained in Figure 6. Ties among positions are signaled with \"\". False positives in the DT column are signaled with an FP. Each time a root cause was correctly detected, it is presented. In the case of CO and PMI, the ranking of the factor is also presented. In the ranking, ties between factors may occur, which means that the root cause was not the only factor identified as having the same importance, meaning that the root cause was not isolated. As such, we signal the ties with \"\". For the DT classifier, we indicate the false positives/incorrect detection, in addition to the root causes. These false positives are presented as \"FP\". The reasons for doing so is (i) due to the classifier not providing a ranking and (ii) to have an improved understanding of the false detections that occurred.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Dataset** & **Moment** & **Overlap\({}_{PMI}\)** & **CO** & **PMI** & **DT** \\ \hline
1 & 1 & 4.20\% & RC2 (1 st); RC1 (12 th) & **RC2 (1 st); RC1 (4 th)** & RC2 \\ \hline
1 & 2 & 0.05\% & **RC2 (1 st)** & **RC2 (1 st)** & RC2; FP; FP \\ \hline
1 & 3 & 16.11\% & **RC3 (1 st) \({}^{*}\)** & **RC3 (1 st) \({}^{*}\)** & FP \\ \hline
1 & 4 & 0.16\% & **RC3 (1 st); RC4 (2 nd) \({}^{*}\)** & RC3 (1 st) & RC3; FP;FP \\ \hline
2 & 1 & 14.24\% & RC2 (6 th) \({}^{*}\) & **RC2 (2 nd) \({}^{*}\)** & FP \\ \hline
2 & 2 & 0.10\% & **RC2 (1 st)** & **RC2 (1 st)** & RC2; FP \\ \hline
2 & 3 & 0.00\% & **RC3 (1 st)** & **RC3 (1 st)** & RC3; FP; FP \\ \hline
2 & 4 & 0.67\% & **RC3 (1 st)** & **RC3 (1 st)** & **RC3** \\ \hline
2 & 5 & 0.22\% & RC3 (6 th) & **RC3 (1 st)** & FP; RC3 \\ \hline
3 & 1 & 3.60\% & RC2 (1 st) & **RC2 (1 st); RC1 (10 th)** & RC2 \\ \hline
3 & 2 & 0.00\% & **RC2 (1 st)** & **RC2 (1 st)** & RC2; FP \\ \hline
3 & 3 & 18.38\% & **RC3 (1 st) \({}^{*}\)** & **RC3 (1 st) \({}^{*}\)** & FP \\ \hline
3 & 4 & 0.61\% & **RC3 (1 st) \({}^{*}\)** & **RC3 (1 st) \({}^{*}\)** & RC3, RC4; FP \\ \hline \hline \end{tabular}
\end{table}
Table 5: Table with the results of the RCA algorithms (FP—False Positive).  
Figure 6: Root causes defined in the simulated datasets [32].  
When considering the performance of the algorithms, it is possible to see that the PMI algorithm has, overall, a better performance than the CO algorithm. Examples of this improved performance are dataset 1, moment 1; dataset 2, moments 1 and 5. In these examples, the PMI algorithm is able to detect what is detected by the CO algorithm but does so while placing the true root causes in better positions.