# Overlap in Automatic Root Cause Analysis in Manufacturing: An Information Theory-Based Approach
## 4 Proposed Methodology  
### Proposed Measure  
A measure of overlap is necessary to ensure that we can correctly measure its presence and its effects on the performance of ARCA solutions. As mentioned before, overlap is present when the values of two columns in a dataset such as the example in Table 1 are synchronized (in the example, \"Machine M_1\" with \"Machine M_3\"). As overlap has a pernicious effect on the analysis using DM and ML algorithms, it is better to consider other perspectives to measure overlap. As such, and to include resilience to noise, we measure overlap from a probabilistic and information theory perspective. Noise is understood as random elements that can affect the final quality of the product. Noise can affect the label, as a small percentage of normal products are labeled as faulty and vice-versa. Considering overlap from a probabilistic perspective, an overlap between two tuples represents the very high probability of a product going through a certain machine given that we know that it went through another machine in another step. Formally, a high overlap between two tuples occurs when  
\[P(Y=y|X=x)\geq Th, \tag{1}\]  
That is, overlap occurs when the probability of going through machine \(y\) in step \(Y\), given that a product goes through machine \(x\) in step \(X\), is above a certain threshold \(Th\) (we propose 0.9 to be the default, as it considers cases of very high overlap while leaving some margin for errors caused by noise). From an information theory perspective, if we consider each tuple as a random variable, it is possible to say that knowing the value of one of the variables provides a high amount of information about the other variable. This information \"shared\" by both variables is quantified in [43] as mutual information or  
\[I(X;Y)=\sum_{x}\sum_{y}p(x,y)log\frac{p(x,y)}{p(x)p(y)}, \tag{2}\]  
It varies between 0 and 1, where 1 means the highest association between variables. Alternatively, mutual information can be defined in terms of entropy,  
\[I(X;Y)\equiv H(X)-H(X|Y)=H(Y)-H(Y|X), \tag{3}\]  
where \(H(X)\), \(H(Y)\) represent the entropy of \(X\) and \(Y\) (two random variables), respectively. Mutual information has been widely used in fault detection applications (see [44] as an example). This measure aligns with the intuition developed for overlap, except that it considers both positive and negative associations. In order to improve the measure, we propose the use of Positive Mutual Information (PMI), defined in [33], which takes into consideration only the positive (normalized) mutual information between the two binary variables:  
\[PMI(x,y)=\frac{2}{H(x)+H(y)}\left(p(x_{p},y_{p})log\frac{p(x_{p},y_{p})}{p(x_{ p})p(y_{p})}+p(x_{n},y_{n})log\frac{p(x_{n},y_{n})}{p(x_{n})p(y_{n})}\right), \tag{4}\]  
where \(x_{p}\) and \(y_{p}\) represent positive values (in this specific case \"went through machine\" and \"problematic product\", respectively), and \(x_{n}\) and \(y_{n}\) represent negative values (\"did not go through machine\" and \"normal product\", respectively). Therefore, in the case of overlap between two tuples, PMI only takes into consideration the cases where the product goes through either both tuples or neither of them. PMI aligns better with the intuition developed for overlap than mutual information. To compute the effect of overlap on a dataset such as the one in Table 1, first one-hot encoding of the factors representing steps is performed in order to obtain binary variables representing the tuples, and then the PMI between all the new tuples is computed. We then count the number of interactions that have a PMI above a certain threshold and divide by the number of interactions with a valid value of PMI. The number of valid interactions is used, because the PMI on factors that have no positive association is undefined, and considering the total number of interactions would \"dilute\" the presence of overlap, diminishing the perception of the issue. Expression (5) measures the amount of overlap on a dataset, based on the concept of PMI.