Robust Root Cause Diagnosis using In-Distribution Interventions
Authors: Lokesh Nagalapatti
Ashutosh Srivastava
Sunita Sarawagi
Amit Sharma
Venue: ICLR 2025 Poster (2025)
Keywords: Root Cause Diagnosis
Causal Inference
Interventional RCD
Primary area: causal reasoning
TL;DR: Identifying root cause of anomalies using interventions rather than counterfactuals estimated from a learned SCM

Abstract: Diagnosing the root cause of an anomaly in a complex interconnected system is
a pressing problem in today’s cloud services and industrial operations. We propose In-Distribution Interventions (IDI), a novel algorithm that predicts root cause
as nodes that meet two criteria: 1) Anomaly: root cause nodes should take on
anomalous values; 2) Fix: had the root cause nodes assumed usual values, the
target node would not have been anomalous. Prior methods of assessing the fix
condition rely on counterfactuals inferred from a Structural Causal Model (SCM)
trained on historical data. But since anomalies are rare and fall outside the training distribution, the fitted SCMs yield unreliable counterfactual estimates. IDI
overcomes this by relying on interventional estimates obtained by solely probing the fitted SCM at in-distribution inputs. We present a theoretical analysis
comparing and bounding the errors in assessing the fix condition using interventional and counterfactual estimates. We then conduct experiments by systematically varying the SCM’s complexity to demonstrate the cases where IDI’s interventional approach outperforms the counterfactual approach and vice versa.
Experiments on both synthetic and PetShop RCD benchmark datasets demonstrate that IDI consistently identifies true root causes more accurately and robustly than nine existing state-of-the-art RCD baselines. Code will be released
at https://github.com/nlokeshiisc/IDI_release.

Published as a conference paper at ICLR 2025
ROBUST RoOT CAUSE DIAGNOSIS USING DISTRIBUTION INTERVENTIONS
IN-
Lokesh Nagalapatti', Ashutosh Srivastava", Sunita Sarawagi', Amit Sharma' IIdian Institute of Technology Bombay
IInternational Institute of Information Technology Hyderabad
3 Microsoff Research India
ABSTRACT
Diagnosing the root cause of an anomaly in a complex interconnected system is pressing problem in today's cloud services and industrial operations. We pro- pose In-Distribution Interventions (IDI), a novel algorithm that predicts root cause as nodes that meet two criteria: 1) Anomaly: root cause nodes should take on anomalous values; 2) Fix: had the root cause nodes assumed usual values, the target node would not have been anomalous. Prior methods of assessing the fix condition rely counterfactuals inferred from a Structural Causal Model (SCM) trained on historical data. But since anomalies are rare and fall outside the train- ing distribution, the fitted SCMs yield unreliable counterfactual estimates. IDI overcomes this by relying on interventional estimates obtained by solely prob- ing the fitted SCM at in-distribution inputs. We present theoretical analysis comparing and bounding the errors in assessing the fix condition using interven- tional and counterfactual estimates. We then conduct experiments by systemat- ically varying the SCM's complexity to demonstrate the cases where IDI's in- terventional approach outperforms the counterfactual approach and vice versa. Experiments on both synthetic and PetShop RCD benchmark datasets demon- strate that IDI consistently identifies true root causes more accurately and ro- bustly than nine existing state-of of-te-of-thh-art RCD baselines. Code will be released at https: /ttp:://gtttttbb.
IDI IDI_reloase
INTRODUCTION
In recent years, cloud services have become increasingly popular due to their advantages in resource sharing, scalability, and cost efficiency (Newman 202T) These systems consist of multiple inter- connected nodes operating over complex topology (Ashok et al. 2024 Hardt et al., 2024 Gu et al.] 2024). To ensure the health of cloud systems, each node continuously monitors key performance indicators (KPIs), such node latency, request counts, CPU utilization, and disk 1/O (Meng et al.) 2020). However, given the inherent complexity of these environments, faults are inevitable. These faults often manifest as citiaaliei-iinniiiaaee deviations from normal behavior- but can propagate across the system, impacting neighboring nodes and potentially compromising the entire application (Lomio et al. (2200). Timely identification of the root cause of such anomalies crucial to minimizing downtime and reducing operational costs. While cloud systems can detect anomalies by identifying deviations in KPI patterns, automated root cause diagnosis (RCD) remains a significant challenge (Chen et al. 2019a).
rare
We define root cause nodes that satisfy key conditions: (1) Anomaly condition: the root cause node exhibits anomalous behavior even while its causal parents are operating usually; and (2) Fix condition: had the root cause nodes assumed their usual values, the anomaly at the target node would not have occurred, While many prior RCD methods address the anomaly condition (Chen)
et al. 2014 Lin et al. 2018 Liu al. 2021 Li al. 2022). the fix condition is often overlooked, with some notable exceptions (Budhathoki et al. 2022b Okati et al. 2024 Budhathoki et al. 2022a) that assess it via counterfactuals. Counterfactual estimation relies on learning a Structural Causal
Model (SCM) (see Sec. A). Given a causal graph linking the system's nodes, an SCM learns a set of functional equations that model the generation of each node as a function of its causal parents
* *Correspondence to: Lokesh Nagalapatti <n1 1 okeshi i scÉ i 1 com>. Work done by L. N. and A. S.
during an internship Microsoft Research India.

---

Published as a conference paper at ICLR 2025
#
#1
CRC
S
(a) Root Causc sample
(b) True Counterfactual
(c) Est. Counterfactual
(d) Est. Intervention
This figure demonstrates how abduccion errors in counterfactuals impact RCD performance. In panel (a), an instance shows anomaly at the purple target node with the rool cause being the gold node 21, which is affected by abnormal intervention. Hence æı takes OOD values, and influences its downstream nodes also to take on OOD values. Latent exogenous nodes are shown grey. Panel (b) illustrates the true CF x (()) obtained by applying the fix. Panel (c) presents the estimated CF for the fix, using exogenous estimates involving abduction that requires SCM evaluations in OOD regions. Finally, panel (d) shows the interventional estimate, which uses sampled è, yiekding resulting x. that conforms to the usual regime. Our theory in Scc|captures this rigorously.
Figure 1:
and latent exogenous variables. Such causal graphs can be derived from inverted call graphs that are easily available for cloud deployments Hardt et (2024).
A key challenge with SCMs is that they are typically trained on historical data collected under usual operating conditions. As result, while SCMs can produce accurate counterfactuals for in-distribution data, they become unreliable for anomalies that lie outside the training distribu- tion (Okati et al., 2024). Our work reveals that these reliability issues stem from the first step in counterfactual estimation abduction, where the functional equations in the SCM need to be in- verted to estimate exogenous variables. Fig. provides an illustration.
If the oracle values of latent exogenous variables were available, counterfactuals would be an ideal method for RCD. In practice, however, exogenous variables are never observed and must be esti- mated from causal mechanisms learned from observational training data. These causal mechanisms are trained on data predominantly representing usual behavior, leading to significant estimation er- rors in exogenous variables compared their oracle values. To address this, introduce IDI (In- Distribution Interventions). which evaluates the fix condition using in-distribution interventions. A key feature of IDI is that, when assessing the fix condition for the true root cause, it requires in- ference of the SCM only in in-distribution regions, ensuring a robust diagnosis. Our theoretical analysis compares IDI with counterfactual methods, demonstrating that the latter's error scales with the total variation distance between the usual training distribution and the rare anomalous distribu- tion, which can be high for outliers. In contrast, IDI's error bounded by the standard deviation of the latent exogenous variables. Experiments using bloud-based synthetic SCMs and a widely used RCD benchmark dataset with known causal graph (PetShop) demonstrate that IDI achieves greater robustness and accuracy in RCD compared to eleven baselines.
2 RELATED WORK
Several anomaly detection methods (Akoglu 2021
Chandola et al. 2009) use an anomaly scoring function g and a threshold to detect anomalies when an instance x has g(x) > т. Our focus in this work is on RCD, and we group existing RCD approaches into Correlation and causal methods.
Non-causal RCD methods typically rely on correlation based analy- ses (Pham et al. 2024. cor 2024 Chen et al.) 2019b] Zhang et al.) 1996. Chen et al. 2022 Luo jet al., 2014] Ma et al. 2020 2019 Yu et al., 2023) to assess relationships between a candidate root cause and the anomalous target node. Sometimes, spurious correlations make these methods predicting nodes that are not even causal ancestors of the target, leading to mislcading conclusions.
Correlation Approaches.
Causal Approaches. Causal methods use causal graph & to limit root cause search ancestors. Some methods propose learning the graph G. and may yield better results than the Correlation ones. In cloud deployments, it is best to leverage the easily available call graphs as learning a causal graph from training dataset requires some strong, and untestable assumptions (Glymour al.) 2019)
2

---

Published as a conference paper at ICLR 2025
Training Data [e.E:
Anomaly Detectors 2ot
Tost oase Input
Anomaly condition
Fix condition
Anomaly Threshokss
(X
on( +
$I 1
SCM Traning
000
X P(X./x.)
Graph
A(Pug:)) oi E 11lt-Aithh,,
Reazd (4
Doc ((f Hi Usual observations
Roct Cause
Training
Inference
Figure 2: IDI's pipeline: During training. we use samples Dun. a causal graph, and anomaly thresholds t. to learn anomaly detectors Φ., and structural equations f. as part of SCM training. During inference, we are given root cause test case. The example illustrates two nodes X1, X4 with abnormal interventions. The edge weights indicate the strength of parent's influence (c.g., fa (x2, x4) = 22 + 2.) While applying the anomaly condition, IDI discards X2 in Rand because of its anomalous parent X1. Finally, the fix conditions excludes X4 because is insufficient to restore Xs to its usual value. IDI declares X1 as the root cause.
Causal Anomaly Approaches. Some methods appraoch RCD using just the anomaly condition
to identify nodes with disrupted local causal mechanisms (Chen et al. 2014 Lin et al. 2018 Liu
al.
2021 He al. 2022; Meng et al.) 2020; Xin et al. 2023 Yang et al. [002]] Okali et al.
2024
Yu et al. [021 Wang et 2018 Li et al.
2022
Shan et al. 2019). They predict nodes Tj
with low empirical sampling probabilities PX. (z)
Paæ. conditioned on their causal parents as root
causes. Some methods (Budhathoki et al.) (2022a) attribute such abnormalities to latent exogenous
disturbances, and they aim to infer abnormalities in the latent variables Ej. Causal Fix Approaches. Sometimes, an abnormal ancestor may have a negligible causal effect on the target, disqualifying as root cause. The fix condition becomes necessary to disregard such nodes, Prior methods assess
the fix condition via soft interventions (Jaber et al. 2020: Ikram et al. 2022). interventions (Okati 2022b a.. SAGE (Gan et al. 2021 is promi-
et al. 2024), and counterfactuals (Budhathoki al.
nent method in this category that leverages conditional variational autoencoders to model SCMs and estimates counterfactuals by sampling from latent distribution predicted by the encoder. However, in the RCD context, the encoder itself is conditioned on OOD inputs during inference. Most meth- ods probe their trained models with OOD inputs. In contrast, our approach, IDI, aims for RCD by probing its trained SCM on in-distribution inputs, thereby leading to a robust RCD approach.
3 PROBLEM FORMULATION
3.1 TRAINING SETUP
Our training dataset comprises the causal graph G. and a set of N samples {xr 1i=1 where each x' E R". These samples are assumed to be drawn from an oracle SCM, S = (X = (X1...., X,), samples that are obtained as follows from the Oracle SCM 1) sample e' i.i.d. from pam and 2) compute x' using the structural equations F. Thus, most of the samples in Dum exhibit usual be- havior. To assess the anomaly level of each node x; E x, we train a set of anomaly detectors Pp = {(g) TI); (g2, .2),..., (gn, ). Each detector Pi consists of a scoring function gi : Xx + R+ and an anomaly threshold T; > O. Thus, (a(x;) = (m.) is the binary indicator for be- ing anomalous. These detectors can be trained unsupervised on Dun using algorithms such as Z- score (Eidleman 1995). isolation forest (Liu et al.] 2008). or IT score (Budhathoki et al. 20226). Given a test instance æ = (.1,..., Tn.) where an anomaly detected at Fn, our goal is to trace the root cause of this anomaly to nodes in the system that caused it. We defer a detailed discussion on
fe,,...,,,, ), F (f1..... f.). p!m). Our training dataset D'm encompasses
Structural Causal Models (SCMs) and the process of computing counterfactuals and interventions using SCMs to Appendix
3.2 QUALIFYING CRITERIA FOR ROOT CAUSE
For of exposition, let assume there is unique root cause xy. and establish the criteria that X, needs to meet to qualify as a root cause. First, X, must be actionable; i.e., these must exist fix value fix such that applying it to X, avoids anomaly at h,. Furthermore, the abnormality at the root cause node should not have originated from any of its ancestors; otherwise, resolving the anomaly
3

---

Published as a conference paper at ICLR 2025
at E, may also require fixing other nodes upstream of zy. More formally, root cause should satisfy the following two criteria:
Anomaly Condition: 2; must be anomalous, while its parent nodes are not, i.e. (y((2)) = 1 and Pp(x,) 0 for any parent node p E Pax,.
2. Fix Condition: Setting Xj to its fix value aiJt should have resolved the anomaly Zn. This implies the counterfactual x: xCF(j)) obtained by intervening on X; 2xfx, should exhibit usual behavior at CF(5) O.
Xn; i.e., Pn (2 (Im (2.
Defining the Root Cause Distribution. We define the distribution (C(() that governs unique root cause samples æ, where 2j is the root cause of anomaly at Tn Such a distribution will have e-j> representing all exogenous variables except C,. drawn from their usual distribution. Whereas, Ej must be sampled strong enough to induce an anomaly at both X..
We define the anomalous distribution Qx (x())) for unique root cause at Xj as: Qec(i( = (RCC) (E_)) (esle))))))))))
Definition
(1)
These two factors are defined as:
QRCWi) (E-s) = Pet
(2)
Q (c P (cgle-), = 1, (In) = 1)
(3)
We denote the distribution induced by ((ci)) on the observed X through the SCM S RCU))
We determine the fix value i to be applied to the root cause node by sampling from the conditional distribution (X, Pazz) ). reflecting how humans would naturally adjust X, in practice. This approach also aligns with prior research (Budhathoki et al. (022b),, which samples an exogenous fix y fi to induce usual values in Xj as a consequence. To estimate the causal effect of propagating x/
downstream to the target node Xn, we consider two approaches: the counterfactual estimate In cFF(()
from Budhathoki al. (2022b) and an alternative interventional estimate zin((j) that we propose. We begin with a theoretical analysis to compare the errors associated with these two methods.
4 INTERVENTIONS VS. COUNTERFACTUALS FOR RCD
We begin with the definitions used in our theoretical analysis.
Definition For two distributions P, Q defined over a space X. the total variational dis- tance Redko et al. 2019) between them is defined as: Pv)( 1 Ssex ((zz))Q(z))dz.
Definition We say that an SCM S is an additive noise model when the structural equations are of the form = (Pax,) + 6,, where Sü is a deterministic function, and 6, is the exogenous variable.
Let pim denote the estimate for latent exogenous distribution Pmm obtained from validation dataset Dv. For additive noise models, pim simply the empirical distribution of the error residuals x;; f. (Paa,) computed for each æ E Dy. We need to use validation set to avoid the overfitting bias in estimation, that otherwise arises from using the train set (Chernozhukov et al. 2018).
Definition Consider an additive noise model S, and let S be its estimate learned from training data (Dins G). For a fix fi applied to the node X,, let CF(i) be the true counterfactual from S, and @CF(j) @im(() be the estimated counterfactual and intervention from S. Then, the following equations show how these quantities are derived:
Fjt1 (1+ + cj+1 where (j+1 2j+1 - f3+1(1r))
CF(J)
(4)
CCFU) = S5ti(1 + E5tı where ēj+1 xj+1 - f551(1z))
fj+l
(5)
(/1l = -5yt111 + 22+1 where ēst1 Pey+1
(6)
This procedure tterates for j + 2,..., in topological order.
4

---

Published as a conference paper at ICLR 2025
Since assessing the fix relies on yp (Ein (In derived from the Oracle SCM S using In
CF()
we bound the error incurred in estimating the true In .FF() from a fitted SCM S.
CFG)
Theorem Suppose the oracle SCM S is additive noise model over a chain graph G = X1 - Xw with structural equations of the form fi(2:-1) + C, where each e, has bounded vari- ance o², and each function J& is K K-Lipschitz. Consider the hypothesis class H = {H, ) ii= where each Hi comprises bounded K K-Lipschitz functions, resulting in losses bouned by M
Let S be the SCM fitted on training data Din- with estimated functions {feli Then, for test sam- ples drawn from the root cause distribution (x with Xj as the unique root cause; for fix for ~ -Pir (())))))) sampled from its empirical distribution, the estimated counterfactual In. CF() computed from S satisfies:
E (cs) Q
[1 CF()) - 2n
SCF()
1 Σ
Km-i
gn-i+1p
E>j
+ M Mn
1 (x(L)) (vd(Px (xx Prr x
Proof Sketch: The main issue stems from abduction in Eq. 12 where fj+1 is inferred at an OOD input zj. This introduces tvd( P, Q) terms in the bound. The exogenous error at a node propagates downstream to their descendants, further compounding the error at the target node. Please refer Fig. for an illustration.
(R(C)) terms that arise from abduction. In geometric series, the leading term dominates the sum. Here, the leading term involves tvd(PX), Qx(() where (x,) Thus, for the root cause X, the above discrepancy lies between the usual training distribution and the anomalous test distribution. Since this discrepancy is substantial, <CF() is
Remark: We first discuss the tvd Q
poor estimate of xn CF()) Other terms in the bound reflect discrepancies between i.i.d. train, test errors, and can be shown to reduce with increasing training size Dmml using generalization bounds like VC-dimensions, etc. (Redko et al. 2019).
CFG) Next, we conduct similar analysis to assess the error in using In as an estimate for 2n
Theorem 6 Under the same conditions laid out in Theorem [ the error between the true counter- CF() and the estimated intervention mit(f) admits the following bound: factual Fa
RC()) x~ R(())
|
CF(() - 1
<
Σ 123
K n-
B
x1-1 [1((())(())) - f CFG))
+ 1 tvd pK Pxi-13 Pim pim ximxx.) + std(e;)
+ std(E)) + E[e]] E[ēil
(8)
where std denotes the standard deviation.
sint(f) does not need abduction. Instead, Proof Sketch: Unlike in the CF case, the estimation of Ix samples Ej+1 as shown in Eq. [ and the enror from abduction is limited by the standard deviation of the oracle Cj- A key advantage of this is that the challenging tvd( (, Q) terms are eliminated from the bound, and these samples ēj+1 result in only in-distribution inputs the fitted SCM.
Corollary 7 Suppose in Theorem 6 the exogenous variables & zero-mean in addition having bounded variance for any E [n], then the two terms std(ē,) and [[(c]] [[2,]] can be dropped from the bound in Theorem 6
5

---

Published as a conference paper at ICLR 2025
Remark:
All terms, except std( can be simplified using generalization bounds. Importantly, interventional estimates remain stable despite the drift between the training distribution Px and the anomalous distribution ( Thus, when exogenous variables have low variance, interventions provide a more robust method for estimating Zn (CF)) and evaluating the fix condition.
We defer all proofs Appendix
OUR APPROACH: IDI
IDI uses interventions motivated by the analysis in Sec. as they offer robust approach to RCD. We illustrate the training and inference procedure of IDI Fig. [ IDI first applies the anomaly condition to filter the promising root cause candidates into a set Reand Then, it applies the fix condition to nodes Reand to diagnose the root cause. We lay down the procedure for unique root cause and multiple root cause scenarios separately. For multiple root cause diagnosis, we need the following assumption to be met:
Assumption At most root cause exists in every simple path that leads the target Xn in G.
Step 1 of IDI: Assessing the anomaly condition. The goal of this step is to identify nodes whose exogenous variables are abnormal. IDI's approach is straightforward; it iterates over all ancestors of Xn, in G. adding node X; to Reand if X; shows an anomaly but none of its parents do. Theorem 4.5 in (Okati et al. 2024) proves that this approach is sound for chain graphs. To assess anomalies, we use the Z-Score, defined for X, as Z-score(r;) = [ze- where μ4; and d; are the sample mean and standard deviation computed for the th node in the training data. IDI easily accommodate any other methods proposed for the anomaly criterion (Chen et al. 2014 Li et al. 2022 Okati et al. 2024).
X,
X,
Figure 3: A chain graph with more than one root cause simple path. Dotted lines denote directed path.
Next, We illustrate the need for assumption using chain graph in Fig. that comprises two root cause nodes, X; and Xj. along a simple directed path to X,, Since X; is a root cause, it is anomalous and is likely influence its downstream nodes to take on anomalous values, including Xj 's parent. As result, IDI may discard Xj incorrectly.
Step of IDI: Assessing the fix Condition. We first describe the procedure for the simpler of unique root cause, and then generalize to multiple root causes.
Unique root cause: this case, IDI applics the fix condition on nodes in Reand iteratively. Suppose X, is the true cause, fix value æ²fix applied to X, suppresses the only abnormal 6j that caused the anomaly Σg. Since IDI samples all other exogenous variables downstream usual distributions, the fix condition for root cause is assessed by probing the learned SCM in-distribution inputs. Therefore, y (1 (1)))())) would evaluate to and IDI correctly predicts X,.
X, from their
Multiple causes: In this case, require set-valued fixes to avoid OOD evaluations. We justify this need with an example in Fig. 4 Let a" (X,, X,) be two root cause nodes. Since they intersect at common descendant Xx. a fix applied to X; leads to an OOD evaluation at X₃ due the influence of its anomalous ancestor Xj. This can potentially causing errors in ssin(s) inference that propagate to x sin(s) However, when both X; and X, are fixed simultaneously, all evaluations will be in-distribution.
X;
X,
X
X,
Another issue with interventions is that any superset of a* may appear to resolve the anomaly, leading to over-prediction of root causes. For instance, applying fix to C a* suppresses the abnormalities in both the root causes a" and also some redundant nodes a* \ a. To address this, we leverage Shapley values (Shapley) 1953) from cooperative game theory. Given set of m items and a utility function & 2 + R+ that assigns a score to each subset a U, Shapley values provide fair method to distribute credit among the m players.
Figure 4: An exam- ple graph with more than root
A player i that achieves high score of a(a U {i}) Φ(a) across multiple a C. U receives a high Shapley value. We treat Reand the items and define the utility function for C Reand as

---

Published as a conference paper at ICLR 2025
Validation
Test (RC)
Counterfactual
Intervention
Linear A00o 0:51 (Low)
0.5) (Low)
(Medium)
Non-Lin A10. (Madium)
Ener TLog Seal)
Emt (L0g Scali
Ener 1Log Soll
25
50
100
1000
23
50
1.3
2000
25
50
100
1000
50
1.20
2300
Data
Trsiiing
(a) Low Variance
(b) Medium Variance level
-Mb.2
Non-Lin (11gg))
High)
Non-Lin ./0,
30"
Eror Log Scale)
Eror 0.00 Scal
221
Eror 3.00 State
227
10-
(c) High Variance
(d) Very High Variance
Figure 5: Assessing the impact of variance of 6 using four variable additive noise toy dataset.
the reduction in raw anomaly scores at X, after fixing a: gn(x,) yn (m) Any subset that includes a reduces the raw anomaly scores significantly thereby achieving high Shapley values.
The pseudocode for the multi-root-cause diagnosis algorithm is presented in Alg in the Appendix.
EXPERIMENTS
We conduct a toy experiment on a four-variable dataset to showcase when interventions outper- form counterfactuals and vice versa. We then test our approach on PetShop and multiple synthetic datasets, comparing it against a broader set of baseline RCD methods.
6.1 TOY EXPERIMENTS
We consider a causal graph where the root nodes X1, X2, X3 each have an edge to a common child X4. We instantiate two Oracle SCMs: linear additive noise model and non-linear one. We sample the linear weights w1, W2, W3 from Ao 1) and define the non-linear model as X4 /oixx/////xx/eexp(--X
64. We draw the exogenous variables , 22, e3 from ( 1) and define structural equations for root nodes as X, = fe())) e; for E 21, create root cause test samples, we assign one of X2, deviation shift that induces an abnormality at X,. We generate n E {25, 50, 100, 1000} training samples, along with 100 validation and 100 test samples, each with unique root cause. Since consider a simple SCM with a single function over 3 variables and the linear model is estimated in closed form, small training sample sizes are chosen to study SCM fitting errors. Learning Š reduces to fitting fa: we fit the linear model using closed-form regression and train the non-linear model as lhree-layer MLP with 10 hidden nodes and ReL.U activations via gradient descent. For the toy experiment, sample zifix from its true distribution (0, 1). We assess the following errors:
3
value from U[3, 10), ensuring least 3-standard
1. Validation Error: Measures the accuracy of fa on in-distribution data; [
1.33 fa (x)
2. Test (RC) Error: Measures the prediction accuracy on OOD samples; (x
(#tS)
3. Counterfactual Error: Quantifies the error arising from using CFs; | 84 .CF))
CF(j)
CF() 34 2im(1)
4. Interventional Error: Measures the error for using interventions:
[24

---

Published as a conference paper at ICLR 2025
We defer detailed description of the toy setup and additional experiments to Appendix D Figure presents results across different é variance levels. Each panel the figure compares linear (left) and non-linear (right) settings. We make the following key observations.
Low Variance: Linear models generalize well to the OOD root cause distribution, leading to
small CF errors (Fig. Sa Non-linear models, however, overfit, resulting in high CF errors.
Medium Variance: Trends remain consistent, confirming that CF error correlates with OOD test error whereas interventional error remains bounded by the standard deviation of e4.
High Variance: Interventions outperform CFs, especially in low-data regimes, as high e4 vari- ance destabilizes the learning f4 leading to very poor OOD generalization.
Very High Variance: CFs slightly outperform interventions, but only because the additive noise assumption holds. This fails when the assumption is violated, as we will see in Sec. (RQ3).
This simple toy experiment shows that while interventional error bounded by the standard deviation of 64, CF error closely aligns with the test error on OOD root cause samples. Thus, when the learned SCM deviates from the true SCM on distributions unseen during training. interventional estimates provide more reliable approach to root cause diagnosis.
Summary:
6.2 COMPARISON WITH BASELINES
We compare IDI against following baselines:
Correlation This class includes c-diagnosie (Shan et al.] 2019). which identifies root causes by testing for significant behavior changes during anomalies, and ranked correlation (Hardt et al. 2024). which assigns root cause to nodes correlated with the target.
Lin Causal Anomaly This class encompasses traversal methods (Chen al., 2014; Lin al.] 2018 Liu et al., 2021 Meng et al. 2020), broadly grouped by (Okati et al. 2024), including the smooth traversa_ method introduced by them. These methods implement version anomaly condition. CTRCA (Li at. 2022) identifies nodes connected to the target through fully anomalous paths, while random wa k methods (Yu et al. (202T) use heuristics.
Causal Fix This includes Hierarchical RCD (HRCD) (Ikram et al. 2022), which predicts root causes as nodes that suffered local mechanism changes affecting the target. TOCA (Okati al. 2024) implements the fix condition jointly over all nodes. The CF Attribute on (Budhathoki al. 2022b) method uses Shapley values to perform CF contribution analysis on all n nodes.
We implemented IDI in the RCD library released by PetShop (Hardt al. 2024 PetShop uses Dowhy (Sharma & Kiciman, 2020) and gem (Blöbaum et al. 2022) for causal inference and PyRCA (Liu et al.) 2023) for root cause analysis. To ensure fair comparisons, we applied the same experimental settings for IDI as those used for the baselines in PetShop.
Evaluation Metric. We assess root cause prediction accuracy using Recall k (Irram et al. 2022 Let a* X be the ground truth root causes, and á the ordered list of predicted root causes, where a(1] is the most prominent. For any k >
Σ la" ++--1 Lafilear
Eis1
Recall@k(ā,a")
lajjea = 1 à E
|a*|
For & = that Recall @ (á, = 1 iff every node in a* is present in á(1 k), while for larger k. we need a to be present the first + predictions of â. We assess using h: = 1,3.
6.3 EXPERIMENTS ON PETSHOP: RCD IN A DEPLOYED CLOUD SYSTEM
PetShop HHardt et 2024) is a recent dataset designed for benchmarking RCD methods in the cloud domain, featuring call graph G that causally links key performance indicators (KPis). The baseline methods in the PetShop library use a linear additive noise model S. which also adopted for IDI. This dataset encompasses three types of latency issues: low, high, and temporal, with many methods successfully identifying the temporal issues. Overall, IDI outperforms other methods in most settings, except for Recall@3 in high latency, where CIRCA emerged as the best performer. CIRCA identifies root causes nodes connected to the target through all anomalous nodes, and
https: http//ttthhhhh.o amwm ama sci wwtp::sccccccccecceeeeeeeeeeeeee
http://www.....ooooooo////o/eeaa cause anal y
8

---

Published as a conference paper at ICLR 2025
Low
High
Temporal
Conrelation
Random Walk 2021
0.00
0.33
Ranked Correlation Hardi
2024
0.40
0.60
0.70
0.90
0.50
0.67
-Diggnusis Shan
2019
0.00
0.00
0.00
0.17
0.17
Cansal Anomaly
Cinca (Li
2022
0.60
0.50
0.60
1,00
0.67
1.00
Traversal
2014
0.80
Smooth Traversal OKati et al.
2024
0.40
0.60
0.00
0.60
0.50
1.00
HRCD [Kram
0.25
Causal Fix
TOCA [Okali all 2014
0.40
0.40
0.20
0.20
0.00
0.00
CF Attribatinn (Budhathoki al.
2022b
0.40
0.50
0.40
0.70
0.00
0.50
[D] (Ours)
0.90
0.90
0.90
0.90
1,00
1.00
Table I: Diagnosing root causes of latency issues in PetShop, a cloud-based microservices dataset. Methods are categorized into Correlation, Causal Anomaly, and Causal Fix.
possibly high latency test cases showed such favorable behavior. Otherwise, Traversal proved to be a strong contender against IDI. The improvements observed in IDI across various settings can be attributed solely to its robust implementation of the fix condition. In contrast, the gains seen in other methods assessing the fix condition are less pronounced compared to Causal Anomaly methods, as their evaluations involve applying S to out-of-distribttinn inputs.
6.4 EXPERIMENTS ON SYNTHETIC SCMs
Next, we design synthetic experiments to answer three key research questions:
RQ1 Linear SCM: How effective is RCD when the Oracle SCM S has linear structural equations, so that an S fit using samples from the usual distribution also generalizes OOD?
RQ2 Non-Lin Lin Invertible SCM: How effective is RCD when & closely approximates non-linear S within the usual regime, and S allows abduction, meaning fi invertible with respect to e,?
RQ3 Non-Lin oon-Invertible SCM: What are the implications for root cause identification when $ closely matches non-linear S in the usual distribution, but S does not support abduction?
We evaluate each option under unique and multiple root cause scenarios. For multiple root causes, we ensure they follow the assumption [ and later perform ablations under its violations in Table 5 Our synthetic setup involved a single anomalous test sample for RCD, so we did not run two baselines: 1) e didignnnii is method, which requires multiple anomalous samples for conducting two-sample tests, and 2) HRCD which learns the causal graph solely from anomalous samples.
We randomly sample a causal graph G using the Networkx li- brary [Hagberg et al. (008). We select the node with the most ancestors the anomaly target, and the root cause nodes a arbitrarily. Since cloud KPIs, such as node latency and CPU utilization, are typically positive (Meng al. 2020). ensured that all nodes in the synthetic data assume posi- tive values. Exogenous variables follow a uniform distribution 6 ~ UÍO. 1] making their standard deviation std(G) 0.3. We explore other choices for c; in Appendix G Finally, We generate the training dataset Dmm by sampling exogenous variables and then use S to generate the observed nodes x in topological order. Each node in is assigned a local causal mechanism as follows:
Generating the Oracle SCM S:
For linear SCMs in RQ1, we define the functional equations as linear, with random coefficients as: x4 = w. Paz, where wi ~ |00.5, 2].
For non-lin invertible SCMs in RQ2, we set each local mechanism fa as an additive noise three layer ELU activation based MLP.
For non-lin non -nvvererti SCMs, we use the same MLP architecture, but without additive noise. In this case, each MLP f; receives both the parents Paz, and the noise as inputs.
For linear SCM, we fit a linear additive noise SCM 3, whereas for both RQ2 and RQ3, we fit an additive noise MLP-based SCM S. We show some example graphs in Appendix
Test case generation: For the root cause set at to cause anomaly at In. we first sample e from U0, 1), then apply a grid search over e,- so that (Ear, <-a-) together lead the SCM to induce anomaly Zn.- We also introduced some irrelevant anomalies at nodes that have weak functional relationships with I, to assess the impact on methods that ignore the fix condition. We repeat each experiment 10 times and report the average values of Recall @ (1,3).

---

Published as a conference paper at ICLR 2025
RQI Linear
RQ2 Non- Lin Invertible
RQ3 Non-Lin Non- Non-Invertible
Number Tool causes
Unique
Multiple
Unique
Multiple
Umique
Multiple
Recall60
k
k
k
k
k
Random Walk
0.00
0.00
0.03
0.03
0.10
0.30
0.27
0.27
0.10
0.30
0.27
0.27
Ranked Correlation
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
Traversal
0.50
1.00
0.40
0.67
0.00
1.00
0.03
0.48
0.00
0.80
0.00
0.33
Smooth Traversal
0,50
1.00
0.52
0.90
0.50
1.00
0.57
0.83
0.40
0.80
0.40
0.73
CIRCA
0.80
1.00
0.33
0.33
0.60
0.80
0.38
0.48
0.40
0.70
0.03
0.03
TOCA
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
CF Attribution
1.00
1,00
0.97
1.00
0.30
0.70
0.20
0,40
0.00
0.20
0.00
0.23
IDI (ours)
1.00
1.00
1.00
1.00
1.00
1.00
0.83
0.97
0.60
0.80
0.63
0.83
Table 2: Recall@k=1 and 3 values for experiments on synthetic SCMs. We present the recall values averaged across ten runs. Best methods are highlighted in green, while the second best methods are shown in yellow. Overall, IDI demonstrates the highest recall across all settings.
Table summarizes our findings. Correlation methods struggle across all settings as root cause nodes correlate all descendants with the target X,. In linear SCMs, many methods detect the unique root cause. However, with multiple root causes, the number of anomalous downstream nodes increases significantly, leading to more false positives. Among Causal Anomaly methods, smoot h traversal performs best. CF Attribution ion excels only in linear SCMs due to accurate abduc- tion. TOCA evaluates all nodes indiscriminately, causing numerous OOD SCM evaluations, while IDI efficiently focuses on Reand- Overall, IDI achieves the best performance.
Results:
6.5 EXPERIMENT ON IDI VS. IDI (USING CFS)
Synthetic Oracle SCM
PetShop Lateney
Linear Unique Multiple
Non-I Lin Inv
Non-l Jin Non Inv Unique Multiple
low
high temporal
Unique Multiple
CF Attribution
1.00
0.97
0.30
0.20
0.00
0.00
0.40
0.40
0.00
IDI (CF)
1.00
1.00
1.00
0.77
0.60
0.12
0.70
0.70
1.00
IDI
1.00
1.00
1.00
0.83
0.70
0.63
0.90
0.90
1.00
Table 3: Recall@ @1 I for CF Attribution baseline method (Budhathoki al. 2022b), IDI that uses CF step-2, and IDI. IDI performs the best.
IDI (CF) version of
In this experiment, we run IDI in counterfactual (CF) mode, denoted as IDI (CF). IDI (CF) first filters root cause candidates Reand and then applies Shapley analysis on estimated counterfactuals instead of interventions. As a baseline, we compare a CF attribution method that skips Reand filtering and applies CF Shapley analysis to all nodes. Table 3shows that CF attribution underperforms IDI (CF), confirming that filtering Reand isolates promising candidates for in-distribution SCM evaluations. While IDI (CF) excels in linear settings, its performance drops elsewhere compared to IDI, solely due to errors in abduction, highlighting interventions as the more effective approach.
7 LIMITATIONS AND CONCLUSION
Limitations: (1) IDI's performance can degrade when Assumption I is violated. (2) In additive noise models, when the training size is large enough for the estimated SCM to approach the oracle SCM, errors in using interventional estimates of counterfactuals plateau at the std. deviation of the exogenous variables (Appendix G). Whereas, counterfactual estimates converge to the true values.
Aaniiiions-Anomalyyy and Fix- to identify root causes of anomalies. While prior methods effectively addressed the anomaly condition, the fix condition often relied on probing trained models with OOD inputs. To address this, we proposed IDI, a novel in-distribution intervention method that ensures the fitted SCM is probed using in- distribution inputs while evaluating the potential of true root cause nodes to resolve the anomaly. Unlike previous methods that required a unique root cause assumption, IDI operates under the more relaxed condition of at most one cause per path. We showed theoretically that IDI's intervention method is superior to counterfactual approaches for additive noise chain SCMs. Our experiments with arbitrary SCMs reaffirmed IDI's capability to deliver robust and accurate RCD.
Conclusion: In this paper, we introduced two key conditions Anomaly
10

---

Published as a conference paper at ICLR 2025
8 ACKNOWLEDGEMENTS
We thank Avishek Ghosh and Abhishek Pathapati for their help in reviewing the proofs. We also thank Srinivasan Iyengar and Shivkumar Kalyanaraman from Microsoft for providing motivating real-world examples and feedback on our solutions.
REFERENCES
Automated root cause analysis with watchdog rea, 2024. URL https: www www.dataddggg. com/ bwtgg//aaadgggaaaaaaaaaaaa
automatteetttaeee root cause- ana lysis/
Leman Akoglu. Anomaly mining: Past, present and future. In Proceedings of the 30th ACM Inter- national Conference on Information & Knowledge Management, pp. 1-2, 2021.
Sachin Ashok, Vipul Harsh, Brighten Godfrey, Radhika Mittal, Srinivasan Parthasarathy, and Lar- isa Shwartz. Traceweaver: Distributed request tracing for microservices without application modification. In Proceedings of the ACM SIGCOMM 2024 Conference, ACM SIGCOMM $24, pp. 828-842, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400706141. doi: 10.1145/365 011553365189033772254 3651890.
URL https:://cc.orgg/e/e/ 1rr.:i...rr/0....1111.
Patrick Blöbaum, Peter Götz, Kailash Budhathoki, Atalanti A. Mastakouri, and Dominik Janzing. Dowhy-gem: An extension of dowhy for causal inference in graphical causal models, 2022.
Kailash Budhathoki, George Michailidis, and Dominik Janzing. Explaining the root causes of unit- level changes, 2022a. URL anttps://arxtit httppp////aaatt.oorr//////////// 12986
Kailash Budhathoki, Lenon Minorics, Patrick Bloebaum, and Dominik Janzing, Causal structure- based root cause analysis of outliers. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 2357- 2369. PMLR, 17-23 Jul 2022b.
Varun Chandola, Arindam Banerjee, and Vipin Kumar. Anomaly detection: survey. ACM com- puting surveys (CSUR), 41(:111-58, 2009.
Junjie Chen, Xiaoting He, Qingwei Lin, Yong Xu, Hongyu Zhang, Dan Hao, Feng Gao, Zhangwei Xu, Yingnong Dang, and Dongmei Zhang. An empirical investigation of incident triage for online service systems. 2019 JEEF/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), pp. 111-120. IEEE, 2019a.
Pengfei Chen, Yong Qi, Pengfei Zheng, and Di Hou. Causeinfer: Automatic and distributed per- formance diagnosis with hierarchical causality graph in large distributed systems. In IEEE IN- FOCOM 2014 IEEE Conference on Computer Communications, pp. 1887-1895, 2014. doi: 10.1109/NFO0CMM.204..66688
Yujun Chen, Xian Yang, Qingwei Lin, Hongyu Zhang, Feng Gao, Zhangwei Xu, Yingnong Dang, Dongmei Zhang, Hang Dong, Yong Xu, Hao Li, and Yu Kang. Outage prediction and diagnosis for cloud service systems. In The World Wide Web Conference, WWW '19, pp. 2659-2665, New York, NY, USA, 2019b. Association for Computing Machinery.
Zhuangbin Chen, Jinyang Liu, Yuxin Su, Hongyu Zhang, Xiao Ling, Yongqiang Yang, and Michael Lyu. Adaptive performance anomaly detection for online service systems via pat- tern sketching. In Proceedings of the 44th International Conference on Software Engineering, pp- 61-72, 2022.
Victor Chernozhukov, Denis Cherverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/dehiased machine learning for treatment and structural pa- rameters, 2018.
Gregory Eidleman. 7. scores-a guide failure prediction. The CPA Journal, 65(2):52, 1995.
11

---

Published as a conference paper at ICLR 2025
Yu Gan, Mingyu Liang, Sundar Dev, David Lo, and Christina Delimitrou. Sage: practical and scalable ml-driven performance debugging in microservices. ASPLOS '21, pp. 135-151, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 978 4781433333 (145/3445814.34440
doi: 10. URL httpt:ttttssett http:/:::/ddi..rgg///.... wrt/ooog///oo 11w5/ 3443344488811 111.0111114444444411 3446700
Clark Glymour, Kun Zhang, and Peter Spirtes. Review of causal discovery methods based on graph- ical models. Frontiers in genetics, 10:524, 2019.
Xuhang Gu, Qingyang Wang, Jianshu Liu, and Jinpeng Wei. Grunt attack: Exploiting execution dependencies microservices. 2024 54th Annual IEEE/FIP International Conference on Dependable Systems and Networks (DSN), pp. 115-128, 2024. doi: 10.1109/DSN58291 22224
00025.
Aric Hagberg, Pieter Swart, and Daniel A Schult. Exploring network structure, dynamics, and function using networkx. Technical report, Los Alamos National Laboratory (LANL), Los Alamos, NM (United States), 2008.
Michaela Hardt, William Roy Orchard, Patrick Blöbaum, Elke Kirschbaum, and Shiva Ka- siviswanathan. The petshop dataset - finding causes of performance issues across microser- vices. In Francesco Locatello and Vanessa Didelez (eds.), Proceedings of the Third Confer- ence on Causal Learning and Reasoning, volume 236 of Proceedings of Machine Learning Research, pp. 957-978 PMLR, April 2024. URL v236/ ht16/hardt24.. 24 html
.ml...mm.r.... press/
Zilong He, Pengfei Chen, Yu Luo, Qiuyu Yan, Hongyang Chen, Guangba Yu, and Fangyuan Li. Graph based incident extraction and diagnosis in large-scale online systems. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering (2)) pp- 1-13, 2022.
Azam Ikram, Sarthak Chakraborty, Subrata Mitra, Shiv Saini, Saurabh Bagchi, and Murat Kocaoglu. Root cause analysis of failures in microservices through causal discovery. Advances in Neural 2022.
Information Processing Systems,
Amin Jaber, Murat Kocaoglu, Karthikeyan Shanmugam, and Elias Bareinboim. Causal discovery from soft interventions with unknown targets: Characterization and learning. In Advances in Neural Information Processing Systems (NeurIPS' volume 33, pp. 9551-9561, 2020.
Mingjie Li, Zeyan Li, Kanglin Yin, Xiaohui Nie. Wenchi Zhang, Kaixin Sui, and Dan Pei. Causal inference-basee root cause analysis for online service systems with intervention recognition. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3230-3302000 3240, 2022.
JinJin Lin, Pengfei Chen, and Zibin Zheng. Microscope: Pinpoint performance issues with causal graphs in micro-service environments. In International Conference on Service Oriented Comput- ing. 2018. URL Utthhtts::
semant cscholar. wwwphhholr.rrr///ooooooo Crtp.oooooooooo::3333311111
Chenghao Liu, Wenzhuo Yang, Himanshu Mittal, Manpreet Singh, Doyen Sahoo, and Steven CH Hoi. Pyrca: A library for metric-based root cause analysis. arXiv preprint arXiv:2306. 11417, 2023.
Dewei Liu, Chuan He, Xin Peng, Fan Lin, Chenxi Zhang, Shengfang Gong, Ziang Li, Jiayu Ou, and Zheshun Wu. Microhecl: high-efficient root cause localization in large-scale microservice systems. In Proceedings of the 43rd International Conference on Software Engineering: Software Engineering Practice, ICSE-SEIP '21, pp. 338-347. IEEE Press, 2021. ISBN 9780738146689
Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. In 2008 Eighth IEEE Interna- tional Conference on Data Mining, pp. 413-422 2008. doi: 10. 109/CDMM2008.11
Francesco Lomio, Diego Martínez Baselga, Sergio Moreschini, Heikki Huttunen, and Davide Taibi. Rare: labeled dataset for cloud-native memory anomalies. In Proceedings of the 4th ACM SIG- SOFT International Workshop on Machine Learning Techniques for Software Seffware-Quaalty Quality Evaluation, pp. 19-24, 2020.
12

---

Published as a conference paper at ICLR 2025
Chen Luo, Jian- MaanGuang Lou, Qingwei Lin, Qiang Fu, Rui Ding, Dongmei Zhang, and Zhe Wang. Correlating events with time series for incident diagnosis. In Proceedings of the 20th ACM SIGKDD international conference Knowledge discovery and data mining, pp. 1583-1592, 2014.
Meng Ma, Weilan Lin, Disheng Pan, and Ping Wang. Ms-rank: Multi-metric and self self-adaptive root cause diagnosis for microservice applications. In IEEE International Conference on Web Services (1CWS'19) pp. 60-67, 2019.
Meng Ma, Jingmin Xu, Yuan Wang, Pengfei Chen, Zonghua Zhang, and Ping Wang. Automap: Diagnose your microservice-ban based web applications automatically. In Proceedings of The Web Conference, WwW'20, pp. 246 258, 2020.
Yuan Meng, Shenglin Zhang. Yongqian Sun, Ruru Zhang, Zhilong Hu, Yiyin Zhang, Chenyang Jia, Zhaogang Wang, and Dan Pei. Localizing failure root causes in a microservice through causality inference. In 2020 IEEF/ACM 28th International Symposium on Quality of Service (IWQoS), pp- 1-10. IEFE, 2020.
Sam Newman. Building microservices. O'Reilly Media, Inc.", 2021.
Nastaran Okati, Sergio Hernan Garrido Mejia, William Roy Orchard, Patrick Blöbaum, and Dominik Janzing. Root cause analysis of outliers with missing structural knowledge, 2024. URL https:
// //arxivv. //arxiv.org//a/ooo arb/ 2406 05014
Judea Pearl. Sufficient causes: On oxygen, matches, and fires Journal of Causal Inference, 7(2), 2019.
Luan Pham. Huong Ha, and Hongyu Zhang. Baro: Robust root cause analysis for microservices via multivariate bayesian online change point detection. Proceedings of the ACM on Software Engineering, (FFSE):21142233) 2024.
levgen Redko, Emilie Morvant, Amaury Habrard, Mare Sebban, and Younes Bennani. Advances in domain adaptation theory. Elsevier, 2019.
Huasong Shan, Yuan Chen, Haifeng Liu, Yunpeng Zhang, Xiao Xiao, Xiaofeng He, Min Li, and Wei Ding. s-diagnosis: Unsupervised and real-time diagnosis of small-window long-tail latency in large-scale microservice platforms. In The World Wide Web Conference, pp. 3215-3222, 2019.
Lloyd S Shapley. value for n-person games. Contribution to the Theory of Games, 2, 1953.
Amit Sharma and Emre Kiciman. Dowhy: An end-to-end library for causal inference, 2020.
Ping Wang, Jingmin Xu, Meng Ma, Weilan Lin, Disheng Pan, Yuan Wang, and Pengfei Chen. Cloudranger: Root cause identification for cloud native systems. In 2018 18th IEEE/ACM In- ternational Symposium on Cluster, Cloud and Grid Computing (CCGRID), pp. 492-502. IEEE, 2018.
Ruyue Xin, Peng Chen, and Zhiming Zhao. Causalrea: Causal inference based precise fine-grained root cause localization for microservice applications. Journal of Systems and Software, 203: 111724, 2023.
Wenzhuo Yang. Kun Zhang, and Steven CH Hoi. A causal approach to detecting multivariate time- series anomalies and root causes. arXiv preprint arXiv:2206. 15033, 2022.
Guangba Yu, Pengfei Chen, Hongyang Chen, Zijie Guan, Zicheng Huang, Linxiao Jing, Tianjun Weng, Xinmeng Sun, and Xiaoyun Li. Microrank: End-to-end latency issue localization with extended spectrum analysis in microservice environments. In Proceedings of the Web Conference 2021, pp. 3087-3098, 2021.
Qingyang Yu, Changhua Pei, Bowen Hao, Mingjie Li. Zeyan Li, Shenglin Zhang, Xianglin Lu, Rui Wang, Jiaqi Li, Zhenyu Wu, et al. Cmdiagnostor: An ambiguity-awan root cause localization approach based on call metric data, In Proceedings of the ACM Web Conference 2023, pp. 2937- 2947, 2023.
Tian Zhang, Raghu Ramakrishnan, and Miron Livny. Birch: an efficient data clustering method for very large databases. ACM sigmod record, 25(2):103-114, 1996.
13

---

Published as a conference paper at ICLR 2025
A PRELIMINARIES ON STRUCTURAL CAUSAL MODELS
Notation. Let X = X2. Xn represent the random variables denoting the Key Per- formance Indicators (KPIs) for each node a system. We use æ = (Zi,....22) In) to denote their realizations. These nodes interconnected via topology defined assumed to be directed and acyelic. An edge (X;, X3) E E indicates that changes to X; affect the values X3 can take, but not vice versa. We denote the parents of node in the graph as Pax., and the corresponding values assigned to these parents in æ as Paz,- We assume that each observed instance x; generated from its causal parents and node specific latent exogenous variable e, via a structural causal model (SCM) as outlined below.
graph g = (X. E) that is
Structural Causal Models (SCM). An SCM (Pearl 2019) is a four-tuple S = (X, 6,F, Pc), where Pe represents the distribution from which the exogenous variables are sampled. The observed variables X; E X are determined by a set of structural equations F = -ff Jn where fi Pax, 6 + X,, This implies that, conditioned on its immediate parents, no other variable can exert a causal influence on X principle known as modularity in causal inference. In summary, an SCM models a data- generating process where an observed instance produced by sampling the latent exogenous variables e ~ P. first, and then subsequently computing the node values by applying the structural equations topological order defined over the causal graph G.
Counterfactuals. Given an observed instance x (21...., In), we may want to generate hypo- thetical instance CFF() representing what would have been Xj had taken the value x' instead of zz. This hypothetical instance, called a counterfactual, is computed using an SCM S in ihree steps:
Abduction: For each i, estimate é, by inverting the function f: so that 2; = (P <) holds in the SCM S. (F() = x; for any X; that is not a descendant of Xj in g. For X,, set 2j CF()) = x'j- 2. Action: Set v;
3. Prediction: For each descendant X; of X,. in topological order, set xr CF()) = (i CRO) G)), using obtained during the abduction step.
Interventions. An intervention mn) represents the distributional effect of setting X, to z'y on its descendants in the causal graph G. They are sampled follows:
1. Sample é; from its marginal distribution P...
Steps 2 and are the same as for counterfactuals, except that interventions use the sampled value E in place of the abducted é; during prediction; i.e., a;;
int(f) (paa (.).
Note that the abduction step in counterfactuals necessitates each fs to be invertible with respect to E,. whereas interventions do not need this requirement. Further, while x CF() a point estimate, int()) is a random variable due to the randomness in é,
B IDI ALGORITHM
We provide the pseudocode for IDI in Alg.
C PROOFS
Definition 8 (Reekko al. 2019)) For a loss function & and hypothesis class H, we define the discrepancy distance between two distributions P, Q,Q as follows:
disc 34 (P.
sup EH
Ex~ ((((p(())))))))))))))))))))()))) N' (x))) - (E==)((((()))
Lemma For bounded loss functions E( $ M, where M 0, we have:
disc 2 (P.Q)= $ Mtvd(P,QQ Q)
See (Redko al. 2019) (Proposition 3.1) for the proof.
14

---

Published as a conference paper at ICLR 2025
Algorithm IDI Algorithm
Require: Data Dm. DAG G, anomalous instance æ, anomaly config A
1: Ensure π
4 predicted root causes 4 A is part of problem spec; default is Z-Score anomaly condition
2: {yi, g t Irainnkoomalyens((Du.. A)
3: Rami
(x(x)) = 1, and (p(1p) = 0 V e Pax,)
4: 8 < FiSSCM(Dm. G)
Fits structural equation fi for each æs input Paz, using La loss
5: for a E 2 do
6: de < TopologicalSon(aa_ 9)
4 To apply the fixes in topological order Follow order de Intervene on a using the SCM S < Shapley utility for the subset
7: Set P Paz, )tj ae
8: @im(()) <- Intervene a. fix)
9: dla] f (an)) Elgn (2)m)())
10: end for
11: R +- ShapleySor((d, desc)
4 Compute Shapley values and Sort R descending
12: return R
Theorem [ Suppose the true SCM S is an additive noise model defined over a chain graph X1 + Nn. with structural equations of the form f((x;-1) + G. where c has bounded variance 6² and fi is K K-Lipschitz. Let H {H4 be the realizable hypothesis class for S, with each Hi containing bounded functions that are K K-lipschitz. Let S denote the SCM learned from training data Dim- S encompasses the estimated functions {fr)ī= For any let 2 denote the distribution of samples with unique root cause at Xj. Then, for a sampled from Q RCL) with fix J for Pir [X11 z)-1) applied the root cause X,, the error in estimated counterfactual
at the target node Xn admits the following bound:
E (F())
SCF
In
S Σ
Km-f
2 B x.-1 -
i2)
(10)
+ 141
(nd((PT....))) QKLLi) + aakKP Pxr Px...)
Proof For the intervention anJ applied on Xy, let us denote the true counterfactual values obtained
from the SCM S as CFF() and the estimated counterfactual from the learned SCM S as CFF))
Then, at j + 1, we have:
CF()) = ()+ (ix) + 6j+1 where 6+1 Ij+1 f1+1(1;)) jtt
(11)
* CFU) = (z + Ez+t where ēÿ+1 Ij+1 - (1z)
(12)
Now, us bound the error in the true CF CF(j) and the estimated CF (CF() using S index j i+1 1. Taking difference of Eqs. 12
b0 #+1
CF() CC(()
15z+1 (2
( - J3+1 (1. + (1;((;;)
(13)
$ 1
(2 (z,) - f1,]
(14)
E |1
CF())
-
< cCF())
E
m
E
Beginit (X)1),11)
- -
term
+ E (onno))ll(ss [155+((5))) f
x (Ct)
(15)
temm 2
Let us bound term above.
15

---

Published as a conference paper at ICLR 2025
Recall that S is obtained by training from Dua- fy+ is learned using N samples obtained i.id. from
PX-- Further, the loss is assessed only for X5+13 we have x [15++ f)-1(1,) =
E Reti (zf)+1(1;)) - fz)((1))]
Qx
Therefore, have:
E 4 pcu) (1f - (x
(16)
s aco) (z,) 10000-1--0 + (1)+11()))) - f+1(1,))
x,
Eny [15)+1(1;)) f5+1(1,)11
< RR [15+1110;)) (-1(0))0 + / (54+(1z)) f041(1,)) (x)) (x)
(xy))da,
(17)
S Pu (1f (
(18)
[ [M(arg)) N (xx²) (x,) 2x2 (xz))do;
sup
hWEEH++
Px [15++((1))) (z,)) disc Hit1 PKT (zy 2x00) (x²))
(19)
The last inequality is valid as long as fJ+1 E Hj+1 which true because Hj+1 is realizable. Further since Hj+1 encompasses bounded fuunctions, we can bound the L1 loss E using a constant M >> 0. Therefore,
E
a- (enoooll1 - fz)((1;)) s Px [1z+)(2)) 1 + Mtvd (zz)) Q RR5 (xx,)
term 1
(20)
where term represents a classical divergence between empirical risk and true risk, and can be bounded using classical VC, Rademacher complexity based generalization bounds. Term 2 is more interesting as it captures the divergence hetween the standard training distribution and the root cause distribution that governs when an anomaly occurs.
Combining Eqs.
15 20
for the error at X5+1, we get
E 2
acG) [|
CF()
< E x (
E P (x)123-1) [U
(21)
term
E Pg (15+1((;))) (1,(l,)) + M tvd PXy Qx1
(C(())
Now, we reduce the first term above. For fix affe Pie, (X,l the following holds:
agg ree (x)115-1)
-
[
lz+ (a))))]
(22)
E
m- Q
Ro(s)
E Bappin
[ (atfin)) ( (af
# RC(3)
E Bag
(1t1 fot1 f
B RCG)
B (x,
( fo+1
< Pag.
[1z+ (+z) M ivd( PTT
(23)
16

---

Published as a conference paper at ICLR 2025
This follows from the definition of QRC()) First observe that for the indices till {1, j- 1}. both root cause distribution (C()) and the training distribution Px agree; i.c., Qx (X, [ x = ( x--1). The only index that changes is j at which the root cause occurs. But since,
RC())
while applying fix, we sample it from Px (X (zy=)) the marginal distribution of the fix simply reduces to PX
Finally, combining the inequalities in 21
23 get:
E s mco)
[1
-CF() j+1
S 2E 2x [1fs+1(zz)) - f
(24)
+ M [Lvd(P 2x + tvd xxp
Now, let carry forward these arguments to 2.
CH(j) 1 [+] (H(S)) + fj+2 where cjt = 4j+2 15+2(18+1)
(25)
j+2
CF()) f)+2(0 + E342 where ēμ42 2j42 f
(26)
j+2
(27)
Now for the estimated counterfactual at X,+2, we have the error
1242 - Fj42 = 15 j+l cT()) - 1z+2(15+1) ( (FU) + f3 13=2(1)41)
CF()
(28)
< 1f+2(2 CF() - ff+2(1 (CF(()
- - f0+2(2)+1)
(29)
term
2
The term above admits the same proof technique as we derived for Xj+1-
E o (13+2((1+41 f)+2(15+1)1 S E E (155+2(1z+1) - (3+2(1+1111
(30)
+ M -M-ww Mvdd 2x5())
Now, let analyze the first term.
Since fs+2 is K K-Lipschitz, have - CF(0)) K| - -x++1
CFC)) Therefore,
scb) [155+((2 CF()) - fz+2 cF() 2
(31)
( 21 (F(i)) f (2(()((() + f f (P()) f
E
m
z
E
j+1
CF()) f CH()) + 인 pct) 2 (2
cF()) - f)42(2
(32)
<
E
[5522(2 CFU) f1+2(1 || + KT
(HU)
o ecu) 1a 54 3++1
x(F
(33)
Using the same arguments as used in Eq. 23 it can be shown that:
E
an a 1o (FO)
(FO) I]] < E P [15+2((1)11) - f
(34)
+ M aod Pxpt))
Finally, we have:
17

---

Published as a conference paper at ICLR 2025
E a
| j++z
CFG)
- - 2B ,+ , [110+2(05+1) (+2(+)++11
(35)
xx
Qx 806) (1 -#j+1
KE
x
M
tvd Lvd X, 0x tvd (xdlP
Pim
X,+1
Now, extend this result to assess the enror X, as follows:
E
Qx RcUs 1 (+(())
20 S Σ Km-
2n- Px -
121
(36)
+ Mn-it1 (Ivd( (x.L1) + tvd( PX;
X
The above inequality holds because from definition of RRC() have E
2 Rco 1
CFU)
2(F() || for any j. Further Xy, because of performing intervention have
CF() CF()
2j Ij =/
Several prior works (Chen et al. 2014 Lin al. 2018 Liu et al. 2021 defined root cause as a node that is anomalous and is connected to target node X,, through a chain of anomalous nodes. i.e., they expect (p(())) 1 for all j. and in such case, we that tvd (Pmm (RCG)) is large for any i > j. Nonetheless, the leading term in the above geometric progression has tvd Pi Q RC()) for which we know kイイ(;)) = 1 by root cause definition, and therefore, in practice @CF() is a poor estimate for CFG), All the other terms can be bounded using classical generalization bounds, and they go down with the size of training data [Duml.
Remark:
Theorem Under the same conditions laid out in Theorem [ the error between the true counter- factual x CFi)) and the estimuted intervention fimit) admits the following bound:
w~ ACis) | - x:m s2in((i11
<
Σ 1>j
K ni
Emp-I~ PM X.-
[1((( - f ]
+ M n-+1 tvd (pmr pim x..1) + std(Ei)
st.(E,) + Eles] 1odl-EEj E[E,]] -E([]]
(37)
where bias(f)) = Ex (f((x;;)((((;)))))))))))))) f((x))) For an unbiased learner, bias is zero.
Proof For the intervention 2ffh applied on X,, let us denote the true counterfactual values obtained from the SCM S as CF()) and the estimated intervention from the learned SCM S fim(()) Then, at j + 1, we have:
# ij1 = ff+1(1))) + Ej+1 where Ej+1 = IJ+1 - fz+1(z;)
CF())
(38)
1 mim())) = (x)))
+ ( where ē,+1 pun E
(39)
j+1
cy+1 represents the empirical distribution of the marginal pm obtained from validation dataset Dy Dun- For additive noise models, this is simply the empirical distribution of the error residuals, Iz+1 - f141(1,) for Dy-
where, Ptm
18

---

Published as a conference paper at ICLR 2025
Now, let us bound the error in the true CF CCG) and the estimated intervention Eim(O) using S
index j + 1. Taking difference of above Eqs.
CT() anaetr M0() |15+11 2 E
E Pug (x, my 1) [10+1((0]) - b+1(1)91]] 2 x
E
(40)
term
an (C()) Bg P [1ejt1 - Ef+1ll
We use Eq. 23 to reduce term to Px
+ M. tvd( Xx) X,² PXT))
Now, let analyze term 2. Since X, is the unique root cause, we have Q Poy Therefore,
E
So+1 acu) B B 9+1 [le/+1 - B[e,+1]l] = 1e P.y- E B [1/t11 - 115
(41)
Now,
bj+1 -6+1| = 1ej+1 - E[ej=1] Ble [[ + [[ej+1] Ej+1l
(42)
S |6 - E[ee= + [ BI] + [B[ē++1] - Es+1l
(43)
using la + bl S |a| + 16]. Note that [B[ej+1] [[] is constant.
Taking expectation on both sides, we get:
Be 5+1 E 1 [|ss+1 1|1
(44)
< Besnirpy 544 [1++] B[ej-1]]] + B Pos [B[rj+]] Ef+1 [B[e++1] B[a
$ VV B Bejtirr [[ej+] - [[()+1])3]]+ +
B Bess (1[()+1)-
- [y+1) [B(++]]
(45)
= =std(E)+1) + (()+)))) + [B[c - E[Es+1]l
(46)
where (t)4()))) is the standard deviation of the latent noise Ej+1-
Remark: It is common in causal literature to assume that e, are zero-mean in addition to having bounded variance. When zero mean assumption holds, we can use ēj+1 = 0, and in that case, we will have E ( E Pyt [|e+t1 - E[2 S a(est1).
Finally, for the estimated intervention at Xz, have:
2)m(((11 s E E [10+1(0))- (z)((
E 2 moEn |
(47)
+ M tvd( Ptt PiT)) + std(e+1) + std(E,+1) + [B(,+1]] - E(E
For the estimated intervention at Xj+2, we have:
cF(j) fj+2(1 (0, (2) (ejm((()))) + E[ē,+2] where En+2 pim
CF(3) + cj+2 where Cj+2 = 2j+2 8542(1541)
x 4j42
(48)
a sim(() f)+2(2
(49)
Therefore, the error at j + is:
CF(f) sinl(()) || = E Keon [1)+2(1 cr(i) -f f ( b]] [1 i2+
(50)
term [lezt2 E E25
+ E.
term
19

---

Published as a conference paper at ICLR 2025
We know that can bound E El2
== (cG)
1c5+2 B[c std(c)+2) std(E)+2) + [[(e)2]]
To bound the first term, we use Lipschitz property.
(11++(00++)))) f542(1541)
CFG) I|I I LL en 1f3+2 ((())) 1 (
(51)
KE e 1 CFU) -1/4111
< B [|]42(())1)) f)+2(15+11]]
(52)
+ M
pm
Xyt: Pri
Finally, we have:
E pc5) 11
CF()) - -x 2
s 2 111)+2(1 (P()) - f)+2(0 (FU)))
(53)
KE m 11 (T()) -1/-1111 std(e)+2) std((5+2) [B[cs+2] B[2j+2]
We can extend this result assess the error at X,, follows:
E w RCG)
[
x
Σ Km
E ExiLIN Xx 1-1
[f CF(1)) ((
+ Mn lvd pi Px-=1 pim pim x..xx..) std(Es)
+ std(?;) + E[G,] E(ε]]]
(54)
Corollary 7 Suppose in Theorem 6 the exogenous variables 6, are zero-mean in addition to having bounded variance for any E [n]. then the error between interventions and counterfactuals admits the following bound:
E 0 (c0) (1() CP(J) s
< Σ Kn-i
E Ey PM
Xi-1 [11(0,9910] ((
+ Mn+++1 tvd Pir., pie x std((a))
(55)
Proof If know that the latent exogenous variables are zero-mean, can pim 6511
in Eq. to to
the dirac-delta distribution 8(ēz+1 = 0).
Then we can see from the remark below Eq. 46 that the proof simply follows.
D TOY EXPERIMENTS
Section 6.1 outlined the experiments on toy datasets. Here, provide detailed description of the dataset generation process and conduct additional experiments to further highlight the differences between interventional and counterfactual estimates for RCD.
20

---

Published as a conference paper at ICLR 2025
Xs
N
Aum Ailt Nx. Kot Landiyy (iCheeeihh
(a) Linear
(b) Non-Linear
Figure 6: This figure illustrates toy causal graph with four variables: root nodes X,, X2. and Xs. and child node X4, which is connected all three root nodes. Panel (a) depicts a linear SCM where the root nodes are direct copies of the exogenous noise terms e, while X, is a linear function of the root nodes. Panel (b) presents non-linear version. where X4 is the average of three non-linear functions applied to the root nodes. In both panels, the SCM follows an additive noise model.
Figure 7: Four variable toy SCMs.
X1
X2
X3
X4
Training x
N(O,1)
N( 1)
N(0, 1)
fa(2))))) +_0(0,0²)
Validation val
10 1)
N(O,1) 1)
+N(0,,²
Test &C
1)
U/3. 10
((
fu(mhi))
+ +N(0,00
Counterfactual CP())
1
W/ 1)
RCC
f +
Intervention 2
11
W/ 1)
RC
Table 4: This table outlines the process for generating each sample in our toy simulation study. The node X, draws its from normal distribution (00 ("), with o² varied between 0.5 and 5 in our experiments. For the test sample shown here, the root cause lies X2, and the corresponding é2 is sampled from uniform U/3, 10). The fix uz is sampled from (0, 1). For counterfactuals, c4 represents the abducted e. while for interventions, éa corresponds randomly sampled error residual obtained from the validation data.
We illustrate the variaber toy SCM for both linear and non-linear cases in Figure presents the equations used generate the toy dataset samples.
7 Table
We now provide a detailed analysis of the key observations from the results shown Figure 5 in main paper.
Low Variance: (a) A linear model, being convex, generalizes well across both training and test domains. As result, the abducted values of closely approximate the values. This trend is evident in Fig. 5a where the interventional error plateaus around the standard deviation 0.5, while the counterfactual (CF) error decreases with more training.
(b) In contrast, for the linear model, interventional estimates significantly outperform CF estimates (note the log scale on the Y-axis). Nonlinear models are non-convex, leading to local minima overfitting during training. Consequently, while validation error decreases rapidly, the test (RC) error remains high training increases. This results in poor CF accuracy when fa is evaluated on -iitr--diitiiittion (OOD) inputs.
Medium Variance: Similar trends are observed in this setting. Figs. 5a and 5b together show that CF error correlates strongly with OOD test (RC) error, while interventional estimates remain bounded by the standard deviation. These results confirm the tightness of our theoretical bounds in practice.
High Variance: In the linear dataset, the trends remain consistent. However, in the nonlinear dataset, interventional estimates show more pronounced advantages over CFs, especially in low- data regimes. This is because, in such scenarios, high variance <4 destabilizes fa training, leading to high variance in parameters. Only with sufficient training samples (e.g., 1000) does the validation error approach zero, causing the CF error to be comparable with the interventional error.
Very High Variance: In this regime, we observe the first instance where CF enror outperforms interventional error for the nonlinear model. One interpretation of the extreme variance in 6 is that the features X1, X2, X3 collected in the training data insufficiently rich to explain the variance of Xg. This means that certain additional features must be collected, and thereby reduce the variance 2² Moreover, CFs perform better in this scenario only because their assumption of
21

---

whether the fix suppresses the root cause at the target. Consequently, both interventions and CFs Published as a conference paper at ICLR 2025
additive noise holds. We will demonstrate in the subsequent experiment that CFs perform very poorly when this assumption is violated.
Summary: While interventions may appear less favorable for linear models, observed them to be adequate for in the root cause diagnosis problem. This is because, even if the estimated a.mi(2) deviates from the true 24 CF()) it suffices to infer the correct signal (1 regarding 24
achieve near wear-perfect recall in our linear experiments, as reported in the main paper. However, for non-linear models, interventions surely emerge as a better choice.
D.1 ADDITIONAL EXPERIMENTS ON LINEAR TOY DATASET
Validation
Test (RC)
Counterfactual
Intervention
0,01
p 0.05
0.1
0.5
101
107
30'
Enwr (Lop Scaidl
Em
EnV L0
10-1
12-2
Training Data Size
Training Data Size
Trairing Data Size
Faining Data Sise
Traiiing Data Siae
(a) Linear toy dataset with multi collinear features in the high training size regime and very high variance regime
331
Emor Stalel
Eng [Log Scile
200
Eng Log
10P
32*
57
Thanning Data Size
Trannin Data Size
Trining Data Size
Trlninng Data Site
Taining Data Size
(b) Linear toy dataset with multi collinear features the low training size regime and very high variance regime
Figure& 8: Assessing the impact of variance of using four variable toy dataset.
We extended the four-variable linear toy dataset above in the very high variance setting to examine scenarios where counterfactuals (CFs) and interventions perform differently. We considered low training size with 50 samples and high training size with 1000 samples to study the impact of such correlated features on the intervention and counterfactual errors. The input features, X, were ex- panded to six dimensions. The first three features, X(: 0 3), were sampled i.i.d. from a standard Gaussian distribution as before. The remaining three features were designed be correlated with the first three follows:
X|: 4| X|: +]+ + (O,,p) X|:55] X|: + X|: X|: + (N, p)
Here, P takes values in (0.01, 0.1, 0.5, 13. When P 0.01, the fourth column is heavily correlated with the first, while for 1, the correlation coefficient is less pronounced.
The results shown in Fi 8b
8a reveal the following observations:
For high training size and high variance settings:
When is small, interventions outperform CFs because multicollinearity induces high error in the abnormal root cause regime for CFs. Validation error remains close to zero across all p settings, but CF test error blows up in the abnormal regime.
22

---

Published as a conference paper at ICLR 2025
As p increases (e.g., P > 0.5), the effects of multicollinearit diminish, and CFs regain their dominance over interventions.
For low training size across all p values:
Interventions consistently outperform CFs. CF error escalates to as high as 10", while interven- tions remain robust with their error bounded by the standard deviation of
General Insights: The central thesis of our work is that while there are specific scenarios where counterfactuals (CFs) outperform interventions- such as when the abducted values are accurately estimated and closely align with the oracle values (e.g., when f: fi)-these cases exceptions rather than the norm. When consolidating the performance across the diverse SCMs examined in our study, interventions surely emerge as the superior approach for RCD compared CFs.
E DESCRIPTION OF DATASETS
E. PETSHOP
The PetShop application is microservices-basee pet adoption platform deployed on Amazon Web Services (AWS). It allows users to search for pets and complete adoption transactions. The system is built using multiple interconnected microservices, which include storage systems, publish-subscribe systems, load balancers, and other custom application based logic. These services were container- ized and deployed using Kubernetes. The main focus of this dataset is to diagnose the root cause for anomalies that occur a target node called PetSite the front-end interface where users interact to browse for the pets displayed in the website. An anomaly is triggered at Petsite upon a viola- tion of the service level objective (SLO), such as when the website's 's response time exceeds 200 microseconds.
Causal Graph. The service dependencies the PetShop application are captured in a directed graph, with edges indicating the call relationships between services. This call graph is inverted by reversing the edge directions to obtain the underlying causal graph for this dataset. While the Oracle causal graph is available in PetShop, the Oracle SCM remains unavailable since the true structural equations underlying the Oracle SCM are unknown. Moreover, obtaining the exact Oracle SCM (i.c., the precise functions fi) is infeasible for any real-world dataset, and rather only observed samples can be recorded. Previous studies leveraging PetShop. such as (Okati et al.) (024) have adopted a learning-basee approach, approximating the Oracle SCM by training the structural equations on the collected finite samples. Our work also follows the same methodology.
Node KPIs. The target node, PetSite, has several ancestor nodes, including PetSearch ECS Fargate, petInfo DynamoDB Table, payforadoption ECS Fargate, lambdastatusuppater Lambda Function, and petlistadoptions ECS Fargate. Key Performance Indicators (KPIs) for these nodes were col- lected using Amazon CloudWatch, with system traces logged at 5-minute intervals. Metrics include the number of requests, and latency (both average and quantiles) for each microservice. Overall, the dataset includes 68 injected issues across five nodes, with each issue associated with a unique ground-truth root cause. Consequently, each test case in this dataset features a unique root cause.
Root Cause Test cases. The issues injected into the system span various types, including request overloads, memory leaks, CPU hogs, misconfigurations, and artificial delays. These issues affect different services, such as petInfo DynamoDB Table, payforadoption ECS Fargate, and lambdasta- tusupdater Lambda Function. Each root cause test case created by selecting a root cause node and then issuing an abnormally high number of requests through that node. This surge in traffic from the selected root cause node ultimately triggers an SL.O violation at the Petsite node. The dataset includes three types of root cause test cases: Low, High, and Temporal latency. On average, the request rates were 485 requests per second for Low latency, 690 for High latency, and 571 for Temporal latency cases. In the High and Temporal latency cases, the root cause nodes deviated significantly from the request patterns observed during training, allowing baseline methods such as traversal to perform well. However, for the more challenging Low latency cases, where the statis- tical discrepancy of the root cause nodes between the training regime and the abnormal root cause regime is less pronounced, IDI demonstrated the best performance.
23

---

Published as a conference paper at ICLR 2025
The delays parameterized with varying severities and affect different services, such as:
PetSearch ECS Fargate: 500-2000ms delays for bunny search requests
payforadoption ECS Fargate: 250- 1000ms delays for all requests
Misconfiguration errors: Affecting 1-10% requests depending on the service
E.2 SYNTHETIC DATASETS
We experiment with several synthetic datasets to benchmark the perfomance of the baselines against IDI. We desribe the generation of the causal graph first.
Causal Graph. We follow the graph generation procedure outlined prior work (Budhathoki et al, 2022b). Given the total number of nodes and the number of root nodes The the causal graph is constructed as follows: The nodes X2... Xn, are assigned as root nodes. For each internal node Xx (where k > n2,), we randomly select its parent nodes from the set of preceding nodes X1, X 22....,Xe-- X
Each internal node has at most one parent, with a slight bias towards selecting a single parent. This sampling strategy helps to maintain hierarchical structure and ensures the graph remains sufficiently deep. Once the parent nodes are chosen, directed edges are added from each parent to Xx. This process is repeated until all n nodes are connected, forming directed acyclic graph. Finally, we randomly choose a node that is at least 10 levels deep from root node to serve as the target anomalous node. We show some example causal graphs in Fig.
Node KPIs. Since cloud KPIs take on positive values such as latency, number of requests, through- put, and availability (Hardt et al. 2024 Meng et al., )020)-10 -which are the commonly logged met- rics, we ensured that all the nodes remain positive. To achieve this root nodes, we sample e, from a uniform distribution UO, 1]. For the internal nodes, we generate their observed values using one of the following three approaches:
Linear: For node X; with parents Pax,, define the functional equation linearly with random coefficients as: I; w; Paz, + E, where wy ~ U0..5.2) 2]. These coefficients ensure that the KPI remains positive. We did not use U[0, 1] for the weights because doing so caused deeper nodes in the graph to diminish, eventually reaching zero
Non-Linear Invertible: For non-linearity, we define each local mechanism f; as an additive noise three-layer ELU-activated MLP. We initialized the MLP weights using uniform distribution to ensure that even the internal nodes remain positive. Since many X; nodes have only one parent, we found that using four hidden nodes in the MLP was sufficient to generate non-trivial test cases.
this case, the equation becomes: X; = mip(Pax.) + & where & ~ U[0, 1].
Non-Linear Non-Invertible In this setting. we use the same MLP architecture in the previous case, but without additive noise. Here, each MI.P ft receives both the parents Paz, and the noise as inputs. Thus, the equation is: X, mip(Pax,, 6.) with 6, 00, 1). This configuration generates the most complex test cases in our experiments.
Root Cause Test cases. We consider two cases for the root cause test cases:
Unique: In this case, we randomly sample node from the ancestors of the target node. Multiple: Here, we randomly sample most three nodes from the ancestors, ensuring that our assumption
[ holds. Specifically, no two root causes lie the same path to the target. We experiment with test cases that satisfy this assumption and perform ablations to investigate its impact.
Suppose a* denotes the root cause node set. For a* to cause anomaly at Tn. apply grid search over e.. such that (Ea. they lead the SCM to induce an anomaly at o,. We start our search from 0 and increase them in steps of size 0.25 until the Z-score of the target node on (v.,) hits the anomaly threshold 3.
E.3 EXPERIMENTS UNDER ASSUMPTION VOOLATIOS
In this experiment, injected root causes arbitrary nodes, resulting in Assumption I violations. The results are shown in Table Both IDI and other Causal Anomaly methods face challenges in this scenario they need parents of root cause node to be usual. While CF attribut per-
24

---

Published as a conference paper at ICLR 2025
Linear
Non-Lin Inv
Non-L.in Non-Iny
Top-1
Top-3
Top-1
Top-3
Top-1
Top-3
Random Walk Yu al.
2021
0.10
0.10
0.13
0.13
0.13
0.13
Ranked Correlation Hardi
2024
0.00
0.00
0.00
0.00
0.00
0.00
Traversal (Chnn al.
2014
0.00
0.40
0.03
0.27
0.00
0.27
Smooth Traversal
Okati al.
2024
0.23
0.50
0.17
0.47
0.30
0.43
CIRCA (Li et al.
2022
0.13
0.13
0.27
0.27
0.13
0.13
TOCA (Okat et al.
2024
0.07
0.07
0.03
0.03
0.00
0.00
CF Attribution Budhathoki et al.
2022b
0.83
0.97
0.33
0.57
0.07
0.23
IDI
0.57
0.57
0.53
0.60
0.40
0.53
Table 5: Experiment under Assumption cause.
violations. simple path to X, can features more than one rool
forms best in the linear setting, it struggles in other settings due abduction errors being amplified by the presence of multiple root causes in the same path. For non-lin inv SCMs, IDI achieves the highest Recall, while in non-lin non-inv cases, it surpasses the CF method by 2x Recall, Overall, IDI achieved the best method Recall even under assumption violations.
F TIMING ANALYSIS
Method
PetShop
Syn Linear
Syn Non -inear
Correlation
Random Walk Yu ct al.
2021
2.36
1.74
4.8T
Ranked Correlation Hardi
2024
0.60
0.21
2.99
e-Diagnosis (Shan et 2019)
2.11
Causul Anomaly
Circa (Li et al. 2022
0.52
0.36
2.73
Traversal (Chen al.
2014)
0.27
0.24
1.05
Smooth Traversal OKati al. 2024
0.30
0.26
0.99
HRCD [Ikram ] 20222
11.69
Causal Fix
TOCA (Okali et al. 2024
1.96
0.95
9.16
CF Attribution (Budhathoki et al.
2022b
9.71
22.99
178.47
Ours
(CF)
0.42
0.38
8.31
IDI
0.37
1.29
9.62
Table 6:
Running time (in seconds) for datasets with unique causes. "_" indicates that the baseline not consider for the corresponding dataset.
We present the rumning time required for predicting the unique root cause across all methods for one test case in Table We show the results for the semi-synthetic PerShop dataset, as well as the Linear and Non-Linear versions of our synthetic datasets. Note that we omit the Non-Linear Nnn-Iiveet cases because their running times were comparable to the Non-Linear Invertible cases. We make the following observations:
The Correlation and Causal Anomaly methods demonstrate the best performance in terms of running time.
2. Causal Fix approaches, on the contrary, are bottlenecked by the need to learn the Structural Causal Model (SCM). Learning the SCM involves fitting a lightweight regression model f. Pax, + X; for each node i. Recall that these models are lightweight because they only need to regress the parent covariates the nodes. The Linear methods incur less time compared to the Non-Linear as they be learned using closed-fome expressions, whereas Non- methods require gradient descent-based training.
3. For predicting the unique root cause, both IDI (CF) and IDI do not require Shapley value com- putations, allowing them to run in significantly less time.
4. The baseline CF Attribution method, however, performs Shapley analysis across all nodes, even for the unique root cause, making it the perft-per method in terms of running time.
Table presents the results for the running time required to predict multiple root causes. Unlike the unique root cause, method IDI and its CF ablation IDI (CF) require Shapley analysis in this setting. However, Shapley values are computed only for the subset nodes in R.and- identified after the first step of our algorithm (the Anomaly condition). We make the following observations:
25

---

Published as a conference paper at ICLR 2025
Method
Syn Linear
Syn Non-Lin Lan Inv
Syn
Non-Lin Lin Non- Inv
Random Walk al. 2021
8.63
6.73
Cor:.
Ranked Correlation (Hardt 2024
0.26
1.54
Causal Anomaly
Circa eL 2022
0.38
2.34
2.23
Traversul (Chen cl. 2014
0.22
2.26
1.99
Smooth Traversal (Okai el al.
2024
0.26
2.37
1.95
Causal Fix
TOCA (Okaii al. 2024)
1.3
11.66
13.58
CF Attribution (Budhathoki et al.
2022h
23.48
120.61
190.08
Ours
IDI (CF)
7.08
40.12
73.41
IDI
42.6
76.24
Table 7: Running time (in seconds) for datascts with multiple root causes.
1. The running time for the random walk-based approach increases due to the presence of multiple anomalous paths leading to the target node.
2. All other baseline approaches exhibit running times comparable to those observed in the unique
root cause test cases.
3. The running times for IDI and IDI (CF) increase because of the additional Shapley computa- tions. However, this increase is significantly smaller compared to CF Attribution, as the former computes Shapley values over subset of nodes, while the latter evaluates them across all nodes.
Dataset
CF Attrib.
IDI (Ours)
Linear
36.4 = 3.6
6.0 + 1.6
Non-Linear Invertible
35.8 = 4.2
6.0 + 1.9
Non-Linear Non-Invertible
37.1 3.8
6.2 = 1.9
Table 8: Number of nodes considered for Shapley Analysis: We report the mean + standard deviation puted across all the test cases.
We report the number of nodes involved in computing the Shapley values in Table g Since Shap- ley value computations are NP-Hard, in practice, Monte Carlo simulations are commonly used to approximate these values by sampling permutations of the nodes. In our work, we sampled 500 permutations for all methods to ensure tractability. Table presents the mean number of nodes volved, along with the standard deviation across all test cases. Overall, we observe 6x reduction in number of nodes for IDI compared to the Attribution baseline. Notably, if exact computation of Shapley values were performed, this reduction factor would be even significant.
G EXPERIMENTS WITH HIGH VARIANCE OF E
In this section, experiment with different sampling distributions for e; evaluate their impact on interventional and counterfactual estimates, We begin with simple four-variable toy example.
G.1 SYNTHETIC SETTING
In this subsection, we evaluate the impact of 6 variance on other datasets used our study.
PetShop. For the real-world PetShop dataset, the exogenous variables e are latent, preventing us from characterizing or controlling their variance. So cannot experiment with this dataset.
All the synthetic experiments outlined below use 100 i.i.d. training samples to learn the SCM S.
Linear SCM. In the main paper, conducted experiments with each 6; sampled from U 1]. Here, we explore broader distributions by sampling from U (|, b]. with {0.5, 2, 5). To reduce clutter, we focus the best-performing baselines from the main results. Figure presents results for the Linear SCM. The left panel corresponds to unique root cause test cases, while the right panel shows multiple root cause test cases, with Recall @ I on the Y-axis. As expected, in the linear setting, both interventional and counterfactual variants of IDI achieve Recall across all variance settings, with the CF Attribution baseline standing out strong competitor.
26

---

Published as a conference paper at ICLR 2025
RQ1: Linear (Unique)
RQ1: Linear (Multiple)
6.5]
3.0.
U/0.
2/0.
0/0.5.01
U/0. 2.01
5/0.5.0
Recaliig
Recal
65
0.
Traversal 5m. Traversal
CRICA
CF Attrb.
(0I
101
Traversal Sm. Traversal
CRCA
CF Attto.
[DF
101
(a) Unique Root Cause
(b) Multiple Root Causes
Figure 9; Linear Oracle SCM
Non-Linear Invertible. Figure 10 presents results for this setting. For unique root cause test cases, IDI slightly outperforms the IDI (CF) variant of our approach. Among the baselines, CIRCA and Smooth Traversal emerge strong competitors, while CF Attribution performs poorly at high vari- ance. For multiple root causes, IDI falls slightly short of IDI (CF) in high variance scenarios, likely due to the overfitting or unstable training of the SCM S, We infact observed high validation errors during training for high variances 6,S.
RQ2: NonLin Invertible (Unique)
RQ2: NonLin Invertible (Multiple)
Rio)
U.0.
0.0.5.0)
UJ0, 2.01
U]0.5.0]
Recalig1
0.5
Toversal
(CF(
Traversal
Attrb.
(CF)
Unique Root Cause
(b) Multiple Root Causes
Figure 10: Non Linear Invertible Oracle SCM
Non-Linear Non-Invertible. This setting is the most challenging among all datasets because c; is not identifiable, causing CF approaches to struggle. As shown in Fig. [1] both CF Attribution and IDI (CF) variant perform poorly, often achieving near-zero recall at high variance. While IDI also experiences performance drops compared to previous datasets, remains comparatively robust when compared against the baselines. IDI stands out the best approach across all variance settings in this dataset.
RQ3: NonLin Nen-Invertible (Unique)
RQ3: NonLin Non-Invertible (Multiple)
6.5]
Ujo. UJ0, 2.01
U/0.2.0]
5.0.5.0)
0/0.5.0)
Reca @1
0.0
0.5
Traverssl 5m. Traversal
CRCA
CF Atrioo
(DI
Traversal Sm. Traverss!
CRCA
Attioo
IDI (CF)
101
(a) Unique Root Cause
(b) Multiple Root Causes
Figure 11: Non Linear Non Invertible Oracle SCM
H SENSITIVITY ANALYSIS OF THE ANOMALY THRESHOLD
We conducted this experiment the PetShop dataset. In our main paper, used default anomaly threshold 5. this section, we assess the impact of baselines that implement the anomaly condi- tion and IDI under varying anomaly thresholds, T. For Z-Score, the threshold determines how many
27

---

Published as a conference paper at ICLR 2025
standard deviations sample must deviate from the mean to be considered anomalous. We experi- mented with thresholds ranging from 2 to 7 and report the results in Table 9 Overall, we observed that IDI and the other baselines remained robust across different threshold choices, with IDI only showing performance degradation at a threshold of We acknowledge that tuning the threshold important in practice. However tune it, abnormal root cause test samples during training, since most samples in Dun are non- non-anomalous. In the absence of such abnormal test samples, specifying this hyperparameter involves domain expertise.
Recall@1
Recall@3
2
6
7
3
4
Traversal
0.80
0.87
0.90
0.90
0.87
Smooth Traversal
0.30
0.48
0.30
0.30
0.30
0.30
0.73
0.73
0.70
0.67
0.73
0.67
IDI (CF)
0.80
0.80
0.80
0.80
0.80
0.90
0.90
0.90
0.90
0.90
0.90
IDI
0.73
0.93
0.90
0.93
0.93
0.93
0.93
0.93
0.93
0.93
0.93
0.93
Table 9: Results under variation of the anomaly detection threshold in the PetShop dataset. We show both @ecall I and @3.
I SYNTHETIC CAUSAL GRAPHS
Figure 12: An example Synthetic test case
Figure 13: An example Synthetic test case
Figure 14: Random graphs sampled for our Synthetic Experiments
We show two instances of random graphs in Fig. 14 The purple node is the anomaly for which we need to find the root cause. The ground truth root cause nodes are shown in yellow. We typically observed that all nodes that are descendants of the yellow nodes also tend to exhibit anomalous behavior.
28


Reviews:
[{"review_id": "W3Eh57dxV4", "reviewer": "Reviewer_hq7T", "summary": "This paper focuses on addressing the issue of performance degradation caused by OOD problems in previous methods when localizing root cause curves through causal interventions. It proposes the IDI method and demonstrates its advantages over existing SCM-based root cause localization methods through multi-level experiments.", "questions": "1. The paper uses intervention methods to identify the root cause curve from multiple anomalous KPI curves. MicroCause uses Granger causality. Can there be a comparative result between the two? [1].\n2. On line 345, it is mentioned that to eliminate the impact of OOD, modeling needs to be performed on all subsets of candidates. What is the computational overhead when a fault is associated with many anomalies?\n\n[1] Meng, Yuan, et al. \"Localizing failure root causes in a microservice through causality inference.\" 2020 IEEE/ACM 28th International Symposium on Quality of Service (IWQoS). IEEE, 2020.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The presence of hidden confounding factors makes the OOD problem both prevalent and significant.\n2. The writing of this paper is clear and easy to read.\n3. The paper provides proof that the IDI method's in-distribution sampling has a bounded error, scaling with the distance between anomalies and normality.", "weaknesses": "See the following questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KXoi4sDVjr", "reviewer": "Reviewer_7kui", "summary": "The paper proposes an approach: IDI that identifies root causes based on two criteria: (1) the node exhibits anomalous behavior (anomalous condition), and (2) if the node had normal values, the target node would not be anomalous (fix condition). It contrasts this approach with the pre-existing methods that use counterfactuals from Structural Causal Models (trained on historical data)—often unreliable for rare anomalies (as they are OOD). IDI relies on interventional estimates within the training distribution (in reality the validation distribution).", "questions": "1. Why did the authors used the phrase \"auto-regressively\" at multiple places in the paper ? Is the data assumed to be time series ? Is the author referring to topological order as \"auto-regressive\" ? \n\n2. In page 9, for non lin non-invertible SCMs, the authors wrote \"but without additive noise\". Is non-additive noise the only way to induce non-invertibility in SCMs ? \n\n3. In page 7, under multiple root causes it is mentioned \"However, one problem is that applying fixes to any subset $\\alpha$ of ....\". This statement is not clear. This statement is the main reason behind using Shapley values and also the primary contribution of your multiple root cause subsection. Please rephrase this statement and explain it in simpler words. \n\n4. I understand that literature uses the phrase \"disrupted local causal mechanism\" and/or \"abnormal $\\epsilon_j$ for \"Anomaly Condition\". How are they all the same thing? Is it because under anomalous $x_j$, $\\epsilon_j$ is OOD ? The authors must provide explanation to this in the paper. \n\n5. **Important Question:** If I take the sample mean of $\\tilde{\\epsilon_{j+1}}$ for a large enough sample size, will it or will it not converge to the $\\hat{\\epsilon}_{j+1}$ ? -- equation 4, 5, 6\n\n6. In page 9, in the paragraph mentioning \"Generating the Oracle SCM $\\mathcal{S}$\", it is mentioned that the \"Cloud KPIs are usually positive\". Why ? If thats a fact, give references. If thats an assumption, mention it explicitly. \n\nI will wait for the rebuttal. Please address both weaknesses and questions.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "The three strengths of the paper are: \n1. Whatever is presented in the paper is technically sound. Although I believe some clarifications are needed and the contribution is not enough (i.e., more content can be added). \n2. **Main Strength:** The experiment section is very impressive. It is thorough and the ablation studies are performed well. \n3. The main research question addressed in this paper i.e., counterfactual estimates can push SCMs to OOD regions, and sampling from in-distribution to intervene to circumvent this issue, is an important contribution to the community. However, originality of this idea is unclear. \n4. Monitoring cloud KPIs for anomalies and root cause diagnosis in them is an important application. Showing the experiments on \"Petshop\" is impressive. In a way the paper is written to cater the root cause diagnosis needs of Petshop. However, the authors need to provide some insights on cloud KPIs either in Introduction or Preliminary to make the paper even stronger for engineers.", "weaknesses": "The methodological contribution of the paper is a major weakness. There are some weaknesses in the experiment section as well. \n1. Sampling the latent exogenous variable from the (validation set) distribution as opposed to inverting the function $f_i$ (abduction) is important but a minor methodological tweak that might not warrant a full research paper. It is also unclear if \"in-distribution intervention\" approach is novel or has been proposed in the literature. The authors must show an analogy of the abduction step as a probability distribution. And then they should show how sampling from that distribution is different from the proposed approach. \n\n2. Most parts of the papers are pre-existing knowledge except the two theorems. Even the difference between the two theorems is the abduction step, which is intuitive from the definitions of counterfactual vs intervention (equation 4, 5, 6). \n\n3. Section 5 is over-explained (especially for the unique root cause case). The definitions of \"anomaly condition\" and \"fix condition\" is well established before that section. The only contribution I see in section 5 (i.e., the paper's approach) is \"Assumption 1\" and \"Multiple root causes explained using Shapley values\". \n\n4. \"Low variance of exogenous variables $\\epsilon_i$ is the most important assumption of this paper. It should have been reflected in the title, and the abstract. Furthermore, the variance of $\\epsilon_i$s are not mentioned for the experiments, thus not providing transparency if the experiments satisfied this assumption. \n\n5. No experiment is done for \"high variance\" of exogenous variable. Therefore a reader cannot get insights if counterfactual method is better for such cases (as opposed to IDI). Please conduct a study to compare IDI and counterfactual methods for a range of variances (of latent exogenous variable) -- clearly showing the boundary cases where counterfactual outperforms IDI. \n\n6. In table 2 and table 4, a lot of the recall values for the proposed IDI approach seems 1.00, which raises questions on the data itself. The authors must provide insight if the data is biased towards their methodology. Just mentioning IDI outperforms the baseline is not enough, since the results are too good to be true. Please do the following -- (1) provide details on the data generation process including the randomness in the graph generation, (2) do multiple replications of the study and report the standard deviation of the recall values. If the authors have a reason to believe that the dataset might be biased for their proposed approach, they must provide reasons and mention the experimental limitations explicitly. \n\n7. The authors must provide examples or case study to show that their assumption of low variance of latent exogenous variable is true. In page 2, authors mentioned that they are \"typically small in practice\" without any references, or examples. This is not acceptable for a technical publication. The authors are required to provide specific examples and/or case studies to support their claim of low variance of exogenous variables. The authors must also show how these claims are true in the experimental dataset (both synthetic and Petshop). \n\n8. The paper is too verbose. The authors repeat the two conditions (anomaly and fix) over and over again, in multiple sections without new content. I suggest the authors to reduce the length of the current version of the paper. Add a few explanations to weakness pointed out be me and other reviewers. The authors need to understand that SCMs and causality might be jargon for a lot of readers. They need to explain everything using simpler notions of either probability or graphs like Figure 1 or Figure 2. \n\n9. The paper's formatting is not acceptable. Page 5, Page 6 mentions theorem numbers are 5 and 6 and in supplementary they are written as theorems 10, 11 respectively. Similarly Theorem 6 is referring to Theorem 10, which is not a real thing in the main text of the paper. Similarly, in page 4, under \"training setup\", the symbol $D^{trn}$ is used which is not defined at all. The paper uses a lot of abbreviations and do not expand it. Some abbreviations like IDI are used even in abstract. These are incorrect formatting. I will assume the authors can make changes in rebuttal phase to these issues. \n\n10. In section 3.3., for \"1. Anomaly Condition\", the authors write \"prior methods....\". Without references, such statements must be avoided and in my honest opinion unacceptable for a technical manuscript. \n\n11. Remark on page 6 is not comprehensive. The authors must remove the \"CIRCA\" reference and explain only their method. Otherwise, it digresses from the context. \n\n12. The authors mentioned in page 7 that their work uses \"Z-score\". Please provide some preliminary knowledge on Z-score and highlight where is it used ?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MICvGj4yBu", "reviewer": "Reviewer_9ck8", "summary": "This paper proposes an algorithm for diagnosing the root causes of anomalies in cloud-based systems. It addresses limitations in traditional counterfactual-based approaches by using in-distribution interventions on structural causal models to identify root causes with enhanced robustness and accuracy. By avoiding out-of-distribution issues common in counterfactuals, it achieves better performance in terms of robustness and root cause accuracy, as validated through experiments on synthetic and benchmark datasets.", "questions": "- Could counterfactual methods still be preferred over IDI in certain cases or types of systems? If so, can you give insights on how to choose between these approaches? \n\n- Is IDI sensitive to variations in anomaly detection thresholds?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper provides a solid theoretical foundation for IDI, including error bounds and conditions where IDI surpasses counterfactual methods.\n\n- Experimental results consistently show that IDI outperforms baselines, highlighting its effectiveness and robustness in root cause diagnosis.", "weaknesses": "- The presentation is complex and may be challenging to follow, particularly in Section 3, due to the heavy use of notation. Adding examples to clarify key concepts would improve readability.\n\n- While IDI demonstrates strong performance on synthetic and benchmark datasets, further validation in diverse, real-world industrial systems would strengthen its practical applicability.\n\n- The focus is on accuracy, but runtime and latency evaluations are limited. A discussion of computational overhead and potential optimizations for large-scale deployment would increase its relevance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fYMJRnFK0V", "reviewer": "Reviewer_ERJh", "summary": "This paper introduces the In-Distribution Interventions (IDI) method to address the problem of RCD. The authors highlight two critical conditions in the RCD problem: the anomaly condition and the fix condition. The authors demonstrate that intervention-based approaches outperform counterfactual reasoning when tackling RCD problems, particularly by avoiding OOD errors commonly associated with counterfactual estimation. The proposed RCD algorithm uses an intervention-based approach to handle both conditions simultaneously. The evaluation results show the effectiveness of the IDI algorithm and its robustness in handling complex anomaly environments.", "questions": "- Can you provide a clear and formal definition of OOD in the context of the RCD problem?\n- How will the performance of IDI be affected if an Oracle SCM cannot be obtained? How can this algorithm be applied in real-world scenarios where an Oracle SCM is unavailable?\n- Will using the Shapley value reduce algorithm efficiency in large-scale RCD scenarios?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Strengths:\n- This paper proposes a novel in-distribution intervention algorithm to address the RCD problem, which simultaneously considers both the fix and anomaly conditions in RCD, demonstrating strong robustness in complex failure scenarios.\n- A well-supported theoretical proof. The authors demonstrate the advantages of the intervention method over counterfactual reasoning when dealing with OOD scenarios and apply this method to the fixed condition in the IDI method.\n- Based on the evaluation results, IDI demonstrates better performance and robustness than the existing baseline algorithms across different datasets.\n- Good reproducibility. The authors have open-sourced the code, data, and detailed instructions.", "weaknesses": "Weaknesses:\n- The OOD problem is not clearly presented. The OOD problem in RCD is an important motivation for the paper. However, I could not find a formal definition of OOD or in-distribution in RCD in the paper. The authors should provide a formal definition of OOD in the context of RCD early in the paper, e.g., in the introduction or background. Additionally, the authors should include an illustrative diagram contrasting OOD and in-distribution scenarios to help clarify this concept for readers.\n- More details about the construction of the SCM are necessary. In the paper, the authors directly apply an Oracle SCM rather than constructing it in an automated way, which is uncommon in existing RCD algorithms. The authors should carefully explain how the Oracle SCM was obtained and discuss whether this method can be generalized.\n- The description of CIRCA may have some minor issues. In CIRCA, the authors use the Descendant Adjustment technique to handle the anomaly scores of downstream nodes in the CBN from the root cause. Therefore, Descendant Adjustment can mitigate the issue with CIRCA, as mentioned in line 276 of the paper. The authors should describe this technique in the paper.\n- The authors need to provide a more precise overview of the IDI algorithm. Currently, the proof in Section 4 takes up too much space, while the description of the IDI algorithm in Section 5 needs to be more brief and mainly presented in text. The authors should include an overview figure in Section 5 to help readers understand the overall workflow of the IDI algorithm.\n- Lack of explanation of the Petshop dataset. In Table 1, the authors evaluate the results across three categories: Low, High, and Temporal. However, the paper needs to explain why these categories are separated for comparison or what observation in each category is meant to reflect. The authors should briefly explain these categories in the Petshop dataset, including why they are relevant to RCD and what insights they provide about the performance of different methods.\n- In Section 6.3, cases of Assumption 1 violations are evaluated. However, the paper does not explain the motivation for this evaluation or whether such violations occur in real-world scenarios. The authors should explain the real-world relevance of these violations and how common they might be in practice.\n- Lack of discussion on some existing counterfactual reasoning methods. Sage [1] proposed using CVAE to generate counterfactuals for RCD. The authors should discuss and compare this method in the paper.\n- Lack of case studies on IDI with OOD data. Although the experiments show promising results, the authors should at least present a case study where IDI is applied for RCD in an OOD scenario (e.g., step-by-step results of applying IDI to the OOD cases with intermediate outputs, a comparison of how IDI performs on OOD cases v.s. other methods) to validate its effectiveness on OOD data. \n- Shapley values are used for multiple root causes, possibly introducing high time complexity. The paper does not discuss the impact of this on scalability. The authors should discuss the computational complexity of their Shapley value approach and its implications for scalability in large-scale RCD scenarios, provide empirical runtime analysis, or discuss potential optimizations for improving efficiency.\n\n[1] Gan, Yu, Mingyu Liang, Sundar Dev, David Lo, and Christina Delimitrou. \"Sage: practical and scalable ML-driven performance debugging in microservices.\" In Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 135-151. 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)