# Hi-RCA: A Hierarchy Anomaly Diagnosis Framework Based on Causality and Correlation Analysis
## 4 Experiment
### Evaluation of Hi-RCA  
In this subsection, we evaluate the effectiveness of our method by comparing it with five baseline methods (RS, Loud, Cauchy, MicroDiag, MicroDiag-V1) and ablation experiments.  
**Baseline methods.**  
* _MicroDiag_[30]: MicroDiag locates the culprit metric based on the metric's causality graph. It identifies the potential propagation among components first, considers different methods of anomaly propagation, and uses two types of causal inference methods to construct the propagation graph among different components and metrics. The root cause metric is localized based on the PageRank algorithm of the causality graph.  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{3}{c}{**Dataset A**} & \multicolumn{3}{c}{**Dataset B**} & \multicolumn{3}{c}{**Dataset C**} \\ \cline{2-10}  & **PR@1** & **PR@2** & **PR@3** & **PR@1** & **PR@2** & **PR@3** & **PR@1** & **PR@2** & **PR@3** \\ \hline Quantile & 0.25 & 0.29 & 0.37 & 0.26 & 0.42 & 0.48 & 0.27 & 0.31 & 0.35 \\ Level-shift & 0.66 & 0.76 & 0.78 & 0.82 & 0.84 & 0.90 & 0.71 & 0.80 & 0.85 \\ Cauchy & 0.73 & 0.86 & 0.90 & 0.84 & 0.96 & 0.96 & 0.82 & 0.87 & 0.89 \\ Kalman filtering & 0.88 & 0.95 & 0.98 & 0.96 & 0.98 & 1.0 & 0.93 & 0.96 & 0.98 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Effectiveness of Kalman filtering with ablation experiments.  
Figure 3: Precision of anomaly microservice location in different datasets.  
* _MicroDiag-V1_: Since MicroDiag directly locates the root cause metric from diverse components, to eliminate the distractions of diverse microservices, we implement it in a simplified scenario where the anomalous microservice has been located, to compare the effectiveness of MicroDiag.
* _Loud_[8]: Loud localizes the culprit metrics based on a propagation graph of anomalous metrics. The graph is constructed by the Granger causality test of metrics. To implement Loud, the anomalous metrics are chosen by the Cauchy Detector [48], the propagation graph is built using the Granger test, and the root cause is located based on the PageRank result of the propagated graph with weighted edges.
* _Cauchy_[48]: An anomaly detector that can quantify the time series abnormality and detects an anomaly by comparing the current value and the history-smoothed value in a sliding window. We compute all metrics' Cauchy anomaly scores and locate the metric with the highest anomaly score as the root cause. Cauchy is chosen as a baseline to show the result where metrics are observed in isolation, regardless of the association between them.
* _Random Selection (RS)_: Random selection is a method engineers use when lacking domain-specific knowledge of the system. Every time, they randomly select an unchecked metric to investigate until the root cause is found.  
We apply Hi-RCA and five baseline methods to all anomaly cases and obtain their performance in terms of \(PR@1\), \(PR@2\), and \(PR@3\), as shown in Table 2. We can see that none of these methods can pinpoint the culprit metric in the top one of the ranked list. However, compared to the best baseline method MicroDiag-V1, our Hi-RCA achieves 48%\(\sim\)64% precision improvement in \(PR@1\).