Necessary and sufficient graphical conditions for optimal adjustment sets in causal graphical models with hidden variables
Authors: Jakob Runge
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: causal inference
information theory
statistics

Abstract: The problem of selecting optimal backdoor adjustment sets to estimate causal effects in graphical models with hidden and conditioned variables is addressed. Previous work has defined optimality as achieving the smallest asymptotic estimation variance and derived an optimal set for the case without hidden variables. For the case with hidden variables there can be settings where no optimal set exists and currently only a sufficient graphical optimality criterion of limited applicability has been derived. In the present work optimality is characterized as maximizing a certain adjustment information which allows to derive a necessary and sufficient graphical criterion for the existence of an optimal adjustment set and a definition and algorithm to construct it. Further, the optimal set is valid if and only if a valid adjustment set exists and has higher (or equal) adjustment information than the Adjust-set proposed in Perkovi{\'c} et~al. [Journal of Machine Learning Research, 18: 1--62, 2018] for any graph. The results translate to minimal asymptotic estimation variance for a class of estimators whose asymptotic variance follows a certain information-theoretic relation. Numerical experiments indicate that the asymptotic results also hold for relatively small sample sizes and that the optimal adjustment set or minimized variants thereof often yield better variance also beyond that estimator class. Surprisingly, among the randomly created setups more than 90\% fulfill the optimality conditions indicating that also in many real-world scenarios graphical optimality may hold.


Necessary and sufficient graphical conditions for optimal adjustment sets in causal graphical models with hidden variables
Jakob Runge German Aerospace Center Institute of Data Science 07745 Jena, Germany and Technische Universität Berlin 10623 Berlin, Germany j jakob. rungeddlr.... de
Abstract
The problem of selecting optimal backdoor adjustment sets to estimate causal effects in graphical models with hidden and conditioned variables is addressed. Previous work has defined optimality as achieving the smallest asymptotic estima- tion variance and derived optimal set for the without hidden variables. For the case with hidden variables there can be settings where no optimal set exists and currently only sufficient graphical optimality criterion of limited applicability has been derived. In the present work optimality is characterized as maximizing a certain adjustment information which allows derive necessary and sufficient graphical criterion for the existence of an optimal adjustment set and a definition and algorithm to construct it. Further, the optimal set is valid if and only if a valid adjustment set exists and has higher (or equal) adjustment information than the Adjust-set proposed Perković et al. [Journal Machine Learning Research, 18: 1-62, 2018] for any graph. The results translate to minimal asymptotic estimation variance for class of estimators whose asymptotic variance follows a certain information iifrrmttionthhoorttiic relation. Numerical experiments indicate that the asymptotic results also hold for relatively small sample sizes and that the optimal adjustment set or minimized variants thereof often yield better variance also beyond that es- timator class. Surprisingly, among the randomly created setups more than 90% fulfill the optimality conditions indicating that also in many real-world scenarios graphical optimality may hold.
Introduction
standard problem setting in causal inference is to estimate the causal effect between two variables given a causal graphical model that specifies qualitative causal relations among observed variables [Pearl] 2009), including a possible presence of hidden confounding variables. The graphical model then allows to employ graphical criteria to identify valid adjustment sets, the most well-known being the backdoor criterion (Pearl 1993 and the generalized adjustment criterion (Shpitser et al.. 2010 Perković et al. [015]] 2018), providing complete identification of all valid adjustment sets. Estimators of causal effects based on such valid adjustment set as covariate are then unbiased, but for different adjustment sets the estimation variance may strongly vary. An optimal adjustment set may be characterized as one that has minimal asymptotic estimation variance. current work, following Kuroki and Cail [2004] and Kuroki and Miyakawa 2003) Henckel et al. 2019) (abbreviated
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

HPM19 in the following) showed that graphical optimality always holds for linear models in the causally sufficient case where all relevant variables are observed. In Witte et al. [2020] an alternative characterization of the optimal adjustment set is discussed and the approach was integrated into the IDA algorithm [Maathuis et al., 2009. 2010] that does not require the causal graph to be known. Rotnitzky and Smucler [219]] extended the results in HPM19 to asymptotically linear non- non-paraeetric graphical models. HPMI9's 's optimal adjustment set holds for the causally sufficient case (no hidden variables) and the authors gave example with hidden variables where optimality does not hold in general, i.e., the optimal adjustment set depends on the coefficients and noise terms (more generally, the distribution), rather than just the graph. Most recently, |Smucler et al. [021] (SSR20) partially extended these results to the non pon-mmrrettiic hidden variables case together with dynamic treatment regimes, i.e., conditional causal effects. SSR20 provide sufficient criterion for optimal set exist and a definition based on a certain undirected graph-construction using a result by van der Zander et al. 2019). However, their sufficient criterion is very restrictive and a current major open problem is a necessary and sufficient condition for an optimal adjustment set to exist in the hidden variable case and a corresponding definition of an optimal set.
My main theoretical contribution is a solution to this problem. Optimality for conditional causal effects in the hidden variables case is fully characterized by rnformatiion-hhoreee approach involving a certain difference of conditional mutual informations among the observed variables termed the adjustment information. Maximizing the adjustment information formalizes the common intuition to choose adjustment sets that maximally constrain the effect variable and minimally constrain the cause variable. This allows to derive a necessary and sufficient graphical criterion for the existence of an optimal adjustment set. The derived optimal adjustment set also has the property of minimum cardinality, i.e., no node can be removed without sacrificing optimality. Further, the optimal set is valid if and only if a valid adjustment set exists and has higher (or equal) adjustment information than the Adjust-set proposed in Perrooicć al. [2018] for any graph, whether graphical optimality holds or not. The results translate to minimal asymptotic estimation variance for a class of estimators whose asymptotic variance follows a certain information-theoreti relation that, at present, I could only verify theoretically for the linear case. As practical contributions the paper provides extensive numerical experiments that corroborate the theoretical results and show that the optimal adjustment set or minimized variants thereof often yield better variance also beyond the theoretically analyzed estimator class. Code is available in the python package https: gi thub , com/ akobrunge/ tigramite te More detailed preliminaries, proofs, algorithms, and further numerical experiments are given in the Supplementary Material.
1.1 Preliminaries and problem setting
We consider causal effects in causal graphical models over a set of variables V with a joint distribution
P = P(V) that is consistent with an acyclic directed mixed graph (ADMG) G = (V.E). Two nodes can have possibly more than one edge which can be directed (+-) bi-directed (++). See Fig. TA for an example. Kinships are defined as usual: parents pa(X) for "*_ spouses sp(X) for *** children ch(X) ) for "X-"".. These sets all exclude X. Correspondingly descendants (X) and ancestors (x))) are defined, which, on the other hand, both include X. The mediator nodes causal paths from X to Y are denoted M M(X, Y) and exclude X and Y. For detailed preliminaries, including the definition of open and blocked paths, see Supplementary Section A In this work we only consider a univariate intervention variable X and effect variable Y. We simplify set notation and denote unions of variables as {W) UUMM A = WMA.
A (possibly empty) set of adjustment variables Z for the total causal effect of X on Y in an ADMG is called valid relative to (X, Y) if the interventional distribution for setting do(X = x) [Paal 2009 factorizes p(Y |do(X = =x))= p( xx z)p( z (z)zz for non-empty Z and as p(Y [do
x])
p(|[x) for empty Z. Valid adjustment sets, the set of which is here denoted Z, can be read off from
given causal graph using the generalized adjustment criterion [Perković al. 2015 2018 which
generalizes Pearl's 's back-door criterion |earl 2009]. To this end define
forb Y) = X U MY
(henceforth just denoted as forb). A set Z is valid if both of the following conditions hold: (i) Z n forb = 0, and (ii) all non-causal paths from X to Y are blocked by Z. An adjustment set called minimal if no strict subset of Z is still valid. The validity conditions can principle be manually checked directly from the graph, but, more conveniently, Perković
Perkeoii et al. (2018) define an
2

---

A
B
C
H 215
z= 2,2
z= 27
zyyss
Ixass
X
Hugs
Hyp
25
Figure (A) Problem setting of optimal adjustment sets in causal graphs with hidden variables represented through bi-directed edges. The goal is to estimate the total causal effect of X on Y potentially through mediators M, and given conditioned variables S. The task is to select valid adjustment set Z such that the estimator has minimal asymptotic variance. (B) illustrates two causal effect estimates for a linear Gaussian model consistent with the graph in A, see discussion in text. (C) For certain class of estimators minimal asymptotic estimation variance can be translated into an information-thhoretinn adjustment set Z must maximize the adjustment information | Lx,z s (blue and red hatched, respectively).
optimization problem, here visualized in a Venn diagram. An optimal
adjustment called 'Adjust' that is valid if and only if valid adjustment exist. In setting including conditioning variables S we call this set the valid ancestors defined as
(X, Y,S) = an(XYS) \ forb
(2)
and refer to this set as vancs or Adjust-set.
Our quantity of interest is the average total causal effect of an intervention to set X to x vs. x' on the effect variable Y given set of selected (conditioned) variables S
A yew'ls FXY (((x),) FE(Y)do(z'))
We denote an estimator given a valid adjustment set Z as 7 ynr'|s.a. In the linear case A Ayr'ss for x' + corresponds to the regression coefficient By x- x-zs in the regression of Y on X, Z, and S. The ordinary least squares (OLS) estimator $y x zs is a consistent estimator of fy
Figure [TA illustrates the problem setting: We are interested in the total causal effect of (here univariate) X on Y (conditioned on S), which is here due to a direct link and an indi- rect causal path through
Z (Z1, Z2, Z, Zo, Z2Za, Z, Za, Z, Zz) Z4 E forb cannot be included in any set because descendant of YM. Here vancs Zi ZzS. All valid adjustment sets remove the bias due to confounding by their definition. The question is which of these valid adjustment sets is statistically optimal that minimizes the asymptotic estimation variance? More formally, the task is, given a graph G and (X, Y,S). to chose a valid optimal set Zoptimal E Z such that the causal effect estimator's asymptotic variance Var yxr'(s.2) E[(AArerls yea'ls Â A.22)).22 is minimal:
mediator M. There are six valid backdoor adjustment
Zoptimal E arrgminze Var( y..2
My proposed approach to optimal adjustment sets is based on information theory [Cover and Thomas 2006). The main quantity of interest there is the conditional mutual information (CMI) defined as a difference [x-1/2 = Hyiz - Hylzx of two (conditional) Shannon entropies Hrix = - p(z, y) In (dddy. Its main properties are non-negativity, 1x,r = 0 if and only if X TL ZiZ, and the chain rule Ixw:y = Ix,rz Iw,yizx. All random variables in CMI can be multivariate.
Throughout the present paper we will assume the following.
assume a causal graphical model over a set Assumptions (General setting and assumptions). of variables V with a joint distribution P = P(V) that is consistent with an ADMG G = (V E). We assume non-zero causal effect from on Y, potentially through set of mediators M, and given selected conditioned variables S, where S n forb set (given S) exists and, hence, the causal effect is identifiable (except when stated otherwise). Finally, assume the usual Causal Markov Condition (implicit in semi-Man eemi-Markovian models) and Faithfulness.
= 0. We assume that at least one valid adjustment
3

---

2 Optimal adjustment sets
2.1 pnormattiontheorette characterization
Figure LB illustrates two causal effect estimates for linear Gaussian model consistent with the graph in Fig. II. F.A With Z Z, Zz (blue) the error much larger than with O Z3 (orange) for two reasons: Z constrains the residual variance V Var(Y (ZS) of the effect variable Y less than O and, on the other hand, constrains the residual variance Var [X [ZS) of the cause variable more than O. Smaller estimator variance also holds for O compared to any other valid set in 2 here.
We information-theoret theoretically formalize the resulting intuition to choose adjustment set Z that maximally constrains the effect variable Y and minimally constrains the cause variable X. In terms of CMIs and given selected fixed conditions S the quantity to maximize can be stated follows.
Definition (Adjustment information). Consider causal effect of X on Y for auljustment set Z given a condition sel S. The (conditional) adjustment (set) information, abbreviated Jz. is defined as
JXYIS.Z x Ix,2/s
(5)
Hyjxs Hx|S
(Hyixzs Hx
(6)
not related to Z
adjustment entrapry
Jz is not necessarily positive if the dependence between X and Z (given S) is larger than that between Z and Y given XSS Equation (6) follows from the CMI definition. Fig. [C illustrates the two CMIs in Eq. (5) in a Venn diagram.
Before discussing the range of estimators for which maximizing the adjustment information Jz leads to minimal asymptotic estimation variance Sect. 2.2] we characterize graphical optimality in an information- framework. Our goal is to provide graphical criteria for optimal adjustment sets, i.e., criteria that depend only on the structure of the graph G and not on the distribution.
Definition information-thhorettie
graphical optimality). Given Assumptions 7 we say that graphical optimality holds if there is a Z E Z such that either there no other Z' + Z or for all other Z' + Z and all distributions P consistent with G we have Jz 2 Jzz
intormation-thooretiie
My main result builds on the following lemma which relates graphical optimality to information- theoretic inequalities in a necessary and sufficient comparison condition for an optimal set to exist.
Lemma 1 (Necessary and sufficient comparison criterion for existence of an optimal set). Given Assumptions [ if and only if there is Z such that either there is no other Z' f Z Z or for all other Z' 7 Z E Z and all distributions P consistent with G it holds that
IZ'Z 2 IZAZZYIZZS and Tx,zrzzzzs Xx'ZI'/Z''. (M)
(06)
60)
then graphical optimality holds and Z is optimal implying Jz 2 Jzz
In SSR20 and HPM19 the corresponding conditional independence statements to the terms (iii) and (iv) in the inequalities 7 are used sufficient pairwise comparison criterion. However, Lemmal] shows that for graphical optimality it is not necessary that terms (iii) and (iv) vanish, they just need to fulfill the inequalities for necessary and sufficient criterion.
In principle, Lemma can used to cross-compare pairs of sets, but firstly, difficult to explicitly evaluate 77 for all distributions P consistent with & and, secondly, iterating through all valid adjustment sets is computationally prohibitive even for small graph sizes. As an example, consider confounding path consisting of 5 nodes. Then this path can be blocked by 25 1 different subsets. In the main result of this work (Thm. 3 a necessary and sufficient criterion based purely on graphical properties is given.
2.2 Applicable estimator class
The above characterization only relates optimality adjustment sets to the adjustment information Jz defined in Eq. (5), but not to any particular estimator. Now the question is for which class of

---

causal effect estimators Ayzz'/s, the intuition of maximizing the adjustment information Jz leads minimal asymptotic estimation variance. In its most general form this class characterized as fulfilling
Zoptimal E argmaXzez + Var
(x. ys (s.ooptionl
min Var (A (srrr(ss.. Zez
(8)
where we assume that Â yzz'(s> is consistent due to a valid adjustment set and correct functional model specification. One can also further restrict the class to estimators whose (square-root of the)
asymptotic variance can be expressed as
Var A ysr') f(Hy Hx zs);
for a real-valued, strictly monotonously increasing function of the adjustment entropy. Minimizing the adjustment entropy is by Eq. (6) equivalent to maximizing the adjustment information. The following assumption and lemma then relates Jz 2 Jz to the corresponding asymptotic variances of given estimator.
Assumptions 2 (Estimator class assumption). The model class of the estimator for the causal effect correctly specified and its asymptotic variance can be expressed as in relation ②
Lemma (Asymptotic variance and adjustment information). Given Assumptions and an estimator fulfilling Assumptions 2 if and only if for two different adjustment sets Z, Z' E Z we have Jz 2 Jz,, then the adjustment set Z has a smaller or equal asymptotic variance compared Z'
Proof. By Equations 66 and 2 Jz 2 Jz (for fixed X, S)S) is directly related to smaller equal asymptotic variance for Z compared to Z', and vice versa.
The paper's theoretical results currently hold for estimators fulfilling relation (9), but at least the main result on graphical optimality in Thm. 3 relation (8) In this work, we leave the question of which general classes of estimators fulfill either relation (8) or the more restricted relation (9) to further research and only show that it holds for the
can can also be relaxed to estimators fulfilling the less restrictive
OLS estimator 3y x-2s for Gaussian distributions.
For Gaussians the entropies in (9) are given by H HY||XZS) = + in(2no) yixzs) and H (X = + x where (-1.)) denotes the square-root of the conditional variance. Then
Var( A A
1 Myyixxs-Hiya #x|zs
Tyxxzs
(10)
Vn
Vn xxiss
This relation is also the basis of the results for the causally sufficient case in Henckel et al. [209 where it is shown that holds more generally for causal linear models that do not require the noise terms to be Gaussian.
2.3 Definition of O-set
The optimal adjustment set for the causally sufficient case is simply P (YMM)) \ forb and was derived in HPM19 and Rotnitzky and Smucler [019]. In Section B.2 the derivation is discussed from an intommation-thhoreee bidirected edges "+>" which considerably complicate the situation. Then the parents of YM not sufficient to block all non-causal paths. Further, just like conditioning on parents of Y' leads to optimality in the sufficient case since parents constrain information in YM, in the hidden variables case also conditioning on spouses of M constrains information about YM.
perspective. In the case with hidden variables we need to account for
Example A. A simple graph (ADMG) illustrate this is X ++++++> (shown with an additional S in Fig. A below, or Fig. in SSR20). Here = 0 = vancs is a valid set, but it is not optimal. Consider O Z1. then term (iii) = 0 in the inequalities 7 since Z' O 0. Even though not needed to block non-causal paths (there is none), Z, still constrains information Y while being independent of X (hence, term (iv) 0) which leads to Jo > Jø according to the inequalities 7
Not only direct spouses can constrain information in Y as Fig. 2 below illustrates. Since for W YM the motif "W < C k=C2"" ("*" denotes either edge mark) is open, it holds that I(C, Cz; W) I(Cn;))+((C: W C1C > I(C1; Y) and can even further increase the first term in the adjustment
5

---

information by conditioning also on subsequent spouses. This chain of colliders only ends if we reach a tail or there is no further adjacency. However, we have to make sure that conditioning on colliders does not open non-causal paths. This leads to the notion of a valid collider path (related to the notion of district in Evans and Richardson (2014).
Definition 3 (Valid collider paths). Given a graph G. a collider path of W for k: 2 is defined by a sequence of edges W W+C ++C,, We denote the set of path nodes (excluding W) along path indexed by as T... Using the set of valid ancestors vancs = an an(XYS) \ forb for the causal effect of X on Y given S we call collider path node set Thy for W YM valid wrt. to (X.Y,S) if
for each path node C' E The both of the following conditions are fulfilled:
(1) C & forb, and (2a) C E vanes or (26) C 1L X vancs.
(1I)
Condition (1) is required for any valid adjustment set. If jointly (2a) and (2b) are not fulfilled, i.e. C' & vancs and CKX vancs, then the collider path stops before C. Our candidate optimal adjustment set is now constructed based on the parents of Y M, valid collider path nodes of YM, and their parents to 'close' these collider paths.
Definition 4 (O-set), Given Assumptions Land the definition of valid colliders in Def 3 define the
ser O O(X, Y,S) = PUPPUUC CUCURE Pc where
P (Y \ forb, C= WwEYM Wi {Tw Thy is valid wrr. to (X, Pc = pa(C).
In the following we will abbreviate O = O(X,Y,S) Algorithm C.I states efficient pseudo-code to construct the O-set and detect whether a valid adjustment set exists. Since none of the conditions of Def. 3 for adding collider nodes depends on previously added nodes, the algorithm is order- independent. The statement occurring in lines 11 and 21 ("No valid adjustment set exists.") is proven in Thm. [ If the graph is a DAG, then lines 4-22 can be omitted. The algorithm is of low complexity and the most time-consuming part is checking for a path line 12, Def. 3 2b) C IL X vancs, which can be implemented with (bi-directional) breadth-first search as proposed in van der Zander
et al. [2019]
Numerical experiments in Section 3 will show that further interesting adjustment sets are the min- imized O-set O min. where O is minimized such that no subset can be removed without making Omi invalid, and the collider-minimized O-set Oauin where only CPc \ P C O is minimized such that no collider-subser can be removed without making O OCmin invalid. Both adjustment sets can be constructed with Alg. C22 similar to the efficient algorithms in van der Zander et al.] (2019). Also the minimized sets are inder-independent since the nodes are removed only after the for-loops. Based on the idea in Ocmin in the numerical experiments we also consider Adjustxmin. where only Adjust \ pa (M(YM) M) is minimized and pa(Y (YMM always included. Finally, we also evaluate Adjustmin where Adjust is fully minimized.
Before discussing the optimality of the O-set, we need to assure that it is a valid adjustment set. Similar to the proof given in Perković et al. (2018) for the validity of the vancs-set (for the case without S), we can state that the O-set is valid il and only if a valid adjustment set exists.
Theorem 1 (Validity of O-set). Given Assumptions 7 but without priori assuming that a valid adjustment set exists (apart from the requirement S i forb = (). If and only if a valid backdoor adjustment set exists, then O is valid adjustment set.
2.4 Graphical optimality
We now move to the question of optimality. is known that there are graphs where no graphical criterion exists to determine optimality. Examples, discussed later, are the graphs in Figs. E..F.
Before stating necessary and sufficient conditions for graphical optimality, mention that next to the O-set defined above and the Adjust set vancs [Perković et al./ 2018]. I am not aware of any other systematically constructed set that will yield a valid adjustment set for the case with hidden variables. yan der Zander et al. (2019) provide algorithms to list all valid adjustment sets, but the question is which of these a user should choose. As mentioned above, Lemma | can be used to cross-compare all pairs of sets, but this is not really feasible. Hence, for automated causal effect estimation, rather than the question of whether graphical optimality holds, it is crucial to have set with better properties than other systematically constructable sets. The following theorem states that

---

A
C
Z
2
Z
Z
2
Z
3
2
2
2
2
M
2
D
Z
S
E
Z
2
2
Z
F
G
Z
Z
S
Figure 2: Examples illustrating (optimal) adjustment sets. In all examples the causal effect along causal paths (thick black edges) between X (red circle) and Y (blue circle) potentially through mediators M (light blue circle), and conditioned on some variables S (purple box), is considered. The adjustment set consists of P (blue boxes) and CPc \ P (green boxes). See main text for details.
the adjustment informations follow Jo 2 Jvanes for any graph (whether graphical optimality holds or not).
Theorem 2 (O-set vs. Adjust-set). ). Given Assumptions 7 with defined in Def. f and the Adjust-set defined in Eq. ( it holds that Jo Jvanca for any graph G. We have Jo Jvancs only if (1) = vancs, or (2) O S vancs and X T1 vancs O |OS.
In the following the O-set is illustrated and conditions for graphical optimality are explored. SSR20 provide a sufficient condition for optimality. which states that either all nodes are observed (no bidirected edges exist) or for all observed nodes V C vancs. This is very strict assumption and not fulfilled for any of the examples (except for Example G) discussed in the following.
Example B. Figure B depicts larger example to illustrate the O-set O = PCPc with P Z Zz (blue boxes) and CPc \ P ZgZ ZyZz (green boxes). We also have a conditioned variable S. Among P, only Z, Zg are needed to block non-causal paths to X, Zz are only there to constrain information in Y. Here the same holds for the whole set CPc P which was constructed from the paths Z++ a descendant of YM. Including an independent variable like Z12 in O would not decrease the adjustment information Jo. but then O would not be of minimum cardinality anymore (proven in Cor. B.I. Here, again, the condition of SSR20 does not hold (c.g.. Z5 is not an ancestor of XYY)) S). O is optimal here which can be seen as follows: For term (iii) in the inequalities 7 to even be non-zero, we would need a valid Z such that Z \ O has a path to Y given OSX. But these are all blocked. Note that while Z1u Zg E Z would open a path to Y, both of these are descendants of M Y and, hence, cannot be in a valid Z. For term (iv) to even be non non-zero O \ Z would need to have a path to X given ZS. But since any valid Z has to contain Z, and Z₂ (or Z the only nodes in O with a path to X are parents of YM and paths from these parents to X all need to be blocked for a valid Z. Hence, O is optimal here.
and Z++ Zs which does not include Za since it
Example C. In Fig. 2C a case is shown where = 2, Ze is not part of O because none of the conditions in Def. 2 is fulfilled: Z2 & vancs Z,Z2Zs and XxXX vancs. Hence, we call Zg an N-node. But Ze cannot be part of any valid Z because it has a collider path to X through Z which is always open because it is part of vancs. Hence, term (iii) is always zero. Term (iv) is zero because O% \ Z is empty for any valid Z here. Here even Jo > Jz since O is minimal and term (ii)
Ix Ox O for any Z + O (generally proven in Corollary B.I.
Z2 where Z, is an N-node. Next to Z = 0 another valid set is Z Zi Then term (iii) is non-zero and in the same way term (iv) is non-zero. The sufficient pairwise comparison criterion in SSR20 and HPM19 is, hence, not applicable. However, it holds that always term (iii (i) because the dependence between Z₁ and Y given X is
Example D. The example in Fig. 2D depiets case with
7

---

always smaller than the dependence between Zg and Y given X and correspondingly term (iv) S (ii). Hence, O is optimal here. If link Y exists, then the only other valid set Z @ and both terms are strictly zero.
Example E. The example in Fig. F (Fig. 3 in SSR20 and also discussed in HPM19) is not graphically optimal. Here O Z, Zz. Other valid adjustment sets are Zı or the empty set. From using 21 IL Y|X and X TL Z in the inequalities (7) one can derive in information-theoreti terms that both Z) Z2 and 0 are better than vancs = Zr. but since Jz, 22 = Jo + I2, - -x a superior adjustment set depends on how strong the link Z1-+X vs. Z2t+Y is. The graph stays non-optimal also with a link Z1 Zz
Example F. The example in Fig, 2F is also not graphically optimal. Here O- 0 and Ze is an N-node with a non-collider path to X. Other valid adjustment sets are Z1 and Z1 72. Higher adjustment information here depends the distribution. Also the same graph with the link x is non-optimal. If, however, there is another link Z1 -,,, then O = 0 is optimal (then Z1 is mediator).
Example G. The example in Fig. 2G is only slight modification of Example E with an added selected condition S. Then Z1, Z2 E vancs. We still get O = Z, Eg and this is now optimal since Zg is always open and any valid set has to contain Z1.
The main result of this work is a set of necessary and sufficient conditions for the existence of graphical optimality and the proof of optimality of the O-set which is based on the intuition gained in the preceding examples.
Theorem 3 (Necessary and sufficient graphical conditions for optimality and optimality of O-set). Given Assumptions [ and with O PCPe defined in Def. 4 Denote the set of N-nodes by N -sp(YMM) (forbOS). Finally, given an N E N and collider path No +++...+++++ ........ ++> W (including N+ for C C and W E YM (indexed by i) with the collider path nodes denoted by T; N (excluding N and W), denote by ON O(X, Y, = SNT,)) ) the O-set for the causal effect If and only if exactly one valid adjustment set exists, or both
of X on Y given S' =SU (NN SUUU" π. of the following conditions are fulfilled. then graphical optimality holds and O is optimal:
(I) For all N N and all its collider paths to W YM that are inside C it holds that Ozp does not block all non-causal paths from X to Y, ie., OaN is non-valid, and
(II) all E O O.P with an open path to given SO \ (E) there is link E++W or an extended collider path E+ +W inside for W E YM where all colliders C E vancs.
Condition (I) and (II) essentially rule out the two canonical cases in Examples F and E, respectively, on which non-optimality in any graph is based. Applied to the examples, we obtain that in Example A Cond. (I) holds since no N-node exists and Cond. (II) holds since X 11 Zi S. In Example also no N-node exists and Cond. (II) holds as X example C 7/2 is an N-node, but there is a collider path to X through Z₁ which is in vancs such that Cond. I is fulfilled. Further, while X K Z5 SO \ {Za}. there is a link Z5++Y such that Cond. II holds. In example D Zi is an N-node, but it has bidirected link with X and Cond. (II) holds since X 11 Zz SO (Z2). In Example E optimality does not hold, but Cond. (1) actually holds since there is no N-node. Cond. (II) is not fulfilled for E = Z₁. which has path to X given O and on the extended collider path Zz-Zz Ze E vancs. For Z' = @ and a distribution P where the link Z almost vanishes
SO \ [E] for every E O A.P.P. P. In
then have Jo < Jz. Example F has an N-node Zz and O2 = 0(X,Y, Y,S' = Z2) = 7, Zo is valid implying that Cond. (I) does not hold, while Cond. (II) is actually fulfilled with O = G. For Z' = O2, = Z Ze and a distribution P' where the link X-+Zi almost vanishes we then have Jo < Jz. Example G is optimal since there are no N-nodes and Z/ E vancs.
Similar to SSR20, HPM19, and Witte et al. 2020 I also provide results regarding minimality and minimum cardinality for the hidden variables case in the Supplement.
3 Numerical experiments
We now investigate graphical optimality empirically to answer three questions: Firstly, whether for a linear estimator under Assumptions 2 the asymptotically optimal variance also translates into better
8

---

Opt
Adjust
Optiws
Optivn
Adjusbovs
Adjuste
Cardinaltyy
10-
10-
04
Opt/Aduust
OpteDpiove
Opt10pt...
DptiAdjusts...
Opt/Adjust....
202%g
2009g
200%
2003
RASE rato
150%
150%
150%
150%
130%
100%
100%
100%
100's
100%
50%
optinal
non-sppmmall
mptimal
NOT nor-sptimal
oponal
non-optinal
optinal
roh-pp0mml
opdmaal
ron- o00-pp0mmal
Figure 3: Results of linear experiments with LinReg and sample size n = 100. Shown are letter-value plots (Hofmann et al. 2017] of adjustment set cardinalities (top row), well RMSE ratios (bottom row) for the O-set vs. other approaches for optimal configurations (left in blue) and non-optimal configurations (right in green). RMSE was estimated from 100 realizations. The dashed horizontal line denotes the median of the RMSE ratios, and the white 'plus' their average. The letter-value plots interpreted follows: The widest box shows the 25%-75% range. The next smaller box above (helow) shows the -8..5% (12.5%-259) range and so forth.
fmite-sample variance. Secondly, how the O-set performs in non-optimal settings (according to Thm. 3. Thirdly. how the O-set and variants thereof perform for estimators not captured by the class which the theoretical results were derived (Assumptions 27. To this end, we compare the performance of O, Adjust, OCmin, Omin: Adjustxmin, and AdjuStmin (see definitions in Section 2.3) together with linear least squares estimation (LinReg) on linear models. In the Supplement we also investigate nonlinear models using nearest neighbor regression (kNN), a multilayer perceptron (MLP), random forest regression, and double machine learning for partially linear regression models (DML) [Chernozhukov et al. 2018). The experiments are based on generalized additive model and described in detail Section D Among these 12,000 randomly created configurations 93% fulfill the optimality conditions in Thm.
The results in Fig. confirm our first hypothesis that for linear experiments with an estimator fulfilling Assumptions 2 Jand in settings where graphical optimality is fulfilled (Thm. 3) the O-set either has similar RMSE significantly outperforms all other tested variants. In particular, O Omi and Adjustmin are bad choices for this setting. Adjust is intermediate and OCmin and AdjustX come closest to O, but may still yield significantly higher variance.
Secondly, in non-optimal settings (only 7% of configurations) the O-set still outperforms Adjust (as expected by Thm. Compared Ocmin and Adjustxmin the O-set leads worse results for about half of the studied configurations, while O Omin and Adjustmin are still bad choices. Cardinality is slightly higher for O compared to all other sets. Fig. S7 we further differentiate the results by the cardinality of the O-set and find that for small cardinalities (up to 4) the O-set has the lowest variance in a majority of cases, but for higher cardinalities either OCmin or again O have the lowest variance (slightly beating Adjustxmin). Hence, either O or O performs best in non-optimal configurations. For very small sample sizes n adjustment set cardinality, there tends to be trade-off and smaller cardinality helps. Then Ocmin tends be better than O for high cardinalities also in optimal settings, but here this effect is only present for n 30 and for 50 already negligible compared to the gain in Jo. In Appendix D.2 are RMSE ratios for all combinations of adjustment approaches considered here and shown that, in general, results are very similar for other sample sizes.
(see Fig. S2 that become comparable to the
Thirdly, we investigate non- paramaracttric estimators on linear as well as nonlinear models (implementa- tions described in Section[D D results in the figures of Section D.3) The different classes of estimators exhibit quite different behavior. For KNN (Figs. S9) the O-set has the lowest variance around 50% of the configurations followed by O Ommin and O min. More specifically (Figs. S$15 S16). for small O-set cardinalities up to 2 the Oeset and for higher either Omin or OCmin (the latter only in non-optimal configurations) perform best. For nonlinear experiments the results are less clear for O-set cardinalities greater than 2, but Omin is still good choice. Regarding RMSE ratios, we see that, for the cases where O is not the best, the O-set can have considerably higher variance, while O min seems to be most robust and may a better choice if O too large. MIP (SO behaves much differently. Here in optimal cases neither method outperforms any other for small

---

O-set cardinalities, but for higher cardinalities (Figs. ST6) the O-set is best in more than 50% of configurations (slightly less for nonlinear experiments) and the others share the rest (except Adjustmin). For non-optimal cases O, OCmin and Adjustxmin share the ranks. Regarding RMSF, for linear experiments the O-results are almost as optimal as for the LinReg estimator in the optimal setting. However, for non-optimal cases O Cmin can have considerably smaller variance and seems to be a robust option then, similarly to AdjustXminn Also for nonlinear experiments OCmin is more robust. The RF estimator (Figs. SS12 SS1 is again different. Here no method clearly is top-ranked, O min and Adjustmin are slightly better for linear experiments and O for nonlinear experiments. Oumin and O min more robust regarding RMSE ratios (similar to Adjustxmin). Finally, the DML estimator (Fig. SS14) was here applied only to linear experiments since its model assumption does not allow for fully nonlinear settings. For optimal settings here O is top-ranked in a majority of cases, but closely followed by Ocmin and Adjustxmin- In non-optimal cases for higher O-set cardinalities these two seem like better choice. Quantitatively, Ommin and AdjustXmin
the most robust choices.
Overall, the O-set and its variants seem to outperform or match the Adjust- variants and whether higher cardinality of the O-set reduces performance depends strongly the estimator and data.
4 Discussion and Conclusions
The proposed adjustment information formalizes the common intuition to choose adjustment sets that maximally constrain the effect variable and minimally constrain the cause variable. The main theoretical contributions are a necessary and sufficient graphical criterion for the existence of an optimal adjustment set in the hidden variables case and definition and algorithm to construct it. To emphasize. graphical optimality implies that the O-set is optimal for any distribution consistent with the graph. Note that in cases where graphical optimality does not hold, there will still be distributions for which the O-set has maximal adjustment information.
Further, the optimal set is valid if and only if valid adjustment set exists and has smaller (or equal) asymptotic variance compared to the Adjust-set proposed in Perković et al. (2018) for any graph, whether graphical optimality holds not. This makes the O-set natural choice in automated causal inference analyses. Practical contributions comprise Python code to construct adjustment sets and check optimality, well as extensive numerical experiments that demonstrate that the theoretical results also hold for relatively small sample sizes.
The theoretical optimality results are limited to estimators for which the asymptotic variance becomes minimal for adjustment sets with maximal adjustment information (relation (8) This fulfilled for least-squares estimators, where even the direct relation (9) holds, but it is unclear whether this also holds for more general classes. The numerical results show that the O-set or minimized variants thereof often yield smaller variance also in non-optimal settings and beyond that estimator class. speculate that further theoretical properties of maximizing adjustment information can be shown because relation (9) for f(-) /. e Hx zs seems related to the lower bound of the
estimation variance counterpart to Fano's 's incquality (Theorem 8.6.6 in Cover and Thomas (2006)). For estimators sensitive to hiih-dimensionality one may consider data-driven criteria or penalties to step-wisely minimize the O-set. However, estimating, for example, the adjustment information from potentially small sample size carries considerable errors itself. Another current limitation is that relation
@ only holds for univariate singleton cause variables X. The however, also hold for multivariate X and preliminary results indicate that, while relation (9) does not hold for multivariate X, the less restrictive relation (8) still seems to hold.
results,
The proposed information-theoreee approach can guide further research, for example, to theoreti- cally study relations (9) for other estimators and address other types of graphs as emerge from the output of causal discovery algorithms and the setting where the graph is unknown [Witte et al. 2020 Maathuis et al. 2009 [010)] At present, the approach only applies to ADMGs and Maximal Ancestral Graphs (MAG) |Richardson and Spirtes] 2002] without selection variables. Last, it remains an open problem to identify optimal adjustment estimands for the hidden variables case based on other criteria such as the front-door formula and Pearl's general do-calculus [Pearl] 2009].
The results may carry considerable practical impact since, surprisingly, among the randomly created configurations more than 90% fulfill the optimality conditions indicating that also in many real- world scenarios graphical optimality may hold. Code is available in the python package https //gi thub. com/ wtw.waaooo/iaaeeee berborunngtiiggamtin
10

---

Acknowledgments and Disclosure of Funding
I thank Andreas Gerhardus for very helpful comments. This work was funded by the ERC Starting Grant CausalEarth (grant no. 948112).
References
Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duñoo Christian Hansen, Whit- ney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21:C1-C68, 2018.
Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. John Wiley & Sons, Hoboken, 2006.
Robin Evans and Thomas S Richardson, Markovian acyclic directed mixed graphs for discrete data. The Annals of Statistics, pages 1452-1482, 2014.
Leonard Henckel, Emilija Perković, and Marloes Maathuis. Graphical criteria for efficient total effect estimation via adjustment in causal linear models. arXiv preprint 197.0:190..02435, 2019.
Heike Hofmann, Hadley Wickham, and Karen Kafadar. Letter-value plots: Boxplots for large data. Journal of Computational and Graphical Statistics, 26(3):469-477, 2017.
Manabu Kuroki and Zhihong Cai. Selection of identifiability criteria for total effects by using path diagrams. In Proceedings of the 20th conference on Uncertainty in artificial intelligence, pages 333-340, 2004.
Manabu Kuroki and Masami Miyakawa. Covariate selection for estimating the causal effect of control plans by using causal diagrams. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 65(1):209-222, 2003.
Marloes H Maathuis, Markus Kalisch, Peter Bühlmann, al. Estimating high-dimensional interven- tion effects from observational data. The Annals of Statistics, (336)))313333111 2009.
Marloes H Maathuis, Diego Colombo, Markus Kalisch, and Peter Bühlmann. Predicting causal effects in large-scale systems from observational data. Nature Methods, 7:247-8, 2010.
Judea Pearl. [bayesian analysis in expert systems]: Comment: Graphical models, causality and intervention. Statistical Science, 8(3):666-653 08 1993. doi: 10.1214/ssh 177010894.
Judea Pearl. Causality: Models, reasoning, and inference. Cambridge University Press, 2009.
Emilija Perković, Johannes Textor, Markus Kalisch, and Marloes H Maathuis. A complete generalized adjustment criterion. Uncertainty in Artificial Inteligence-Proceeddng of the Thirty-Fire Conference (2015), pages 682-69) 692-691.20 AUAI Press, 2015.
Emilija Perković, Johannes Textor, and Markus Kalisch. Complete graphical characterization and construction of adjustment sets in markov equivalence classes ancestral graphs. Journal of Machine Learning Research, 18:1-62, 2018.
Thomas Richardson and Peter Spirtes. Ancestral graph markov models. The Annals of Statistics, 30 (4):962-1030, 08 2002.
Andrea Rotnitzky and Ezequiel Smucler. Efficient adjustment sets for population average treatment effect estimation non-parametric causal graphical models. arXiv preprint arXiv:1912.000 1912.00306, 2019.
Iyy Shpitser, Tyler VanderWeele, and James M Robins. On the validity of covariate adjustment for estimating causal effects. In Proceedings of the Twenty-Sixir -ixth Conference on Uncertainty in Artificial Intelligence, pages 527-536, 2010.
Smucler, Sapienza, and A Rotnitzky. Efficient adjustment sets causal graphical models with hidden variables. Biometrika, 3 2021. doi: 10. 00.1099/biome//sab01.....

---

Benito van der Zander, Maciej Liskiewicz, and Johannes Textor. Separators and adjustment sets in causal graphs: Complete criteria and an algorithmic framework. Artificial Intelligence, 270:1-40. 2019.
Janine Witte, Leonard Henckel, Marloes H Maathuis, and Vanessa Didelez. On efficient adjustment in causal graphs. Journal of Machine Learning Research, 21(246): (4:4666::44422222 2020.
12


Reviews:
[{"review_id": "zglqRAiywRj", "reviewer": "Reviewer_9L1v", "summary": "The paper defines the optimality of a (valid) adjustment set in terms of a notion called 'adjustment information.'  Then, it focuses on estimators for which maximizing adjustment information equates to minimizing asymptotic estimation variance. The authors provide a sufficient and necessary graphical criterion for the existence of an optimal adjustment set under this notion. They also give an explicit and efficient construction of an adjustment set called the O set, which is proven to be optimal whenever graph optimality holds (the constraints in the graph are sufficient to determine an optimal adjustment set).", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The results presented by the authors are novel and advance previous work by considering the hidden variable setting and characterizing the existence of optimal adjustment sets for estimators satisfying certain information-theoretic properties. It appears relevant work has been appropriately cited and discussed. \n\nAs far as I can tell, the results and claims presented are supported both theoretically and experimentally. The authors are very clear about the assumptions needed for their results to hold and also consider its limitations. Experiments consider both scenarios where the assumptions hold and where they don't; making sense of observed results.\n\nI find the paper well written. Relevant previous work, definitions, and results are introduced clearly and making good use of several examples. The experimental section and further discussion help understand the benefits of using the proposed result and its limitations. I was wondering why the experimental section only considers the case where $\\mathbf{S}=\\emptyset$.\n\nOverall, the paper makes important theoretical and practical contributions to the problem of identifying causal effects, using covariate adjustment, in settings where the graphical structure is not fully known. In particular, the results advance the state of the art for choosing and constructing optimal adjustment sets for certain estimators.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The limitations of the proposed criteria and algorithm have been well explained, discussed and empirically verified.\nI can foresee any societal impact that needs to be addressed at this time.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}, {"review_id": "lY-tYgSf9Ub", "reviewer": "Reviewer_uv7C", "summary": "This paper provides necessary and sufficient graphical conditions for ‘optimal’ adjustment sets in causal graphical models with hidden variables, where the optimality is defined by a newly proposed metric, instead of the commonly used smallest asymptotic estimation variance.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I think this paper needs to justify and further explain the proposed adjustment information (Definitions 1) and information-theoretical graphical optimality (Definition 2). Intuitively, an estimator is statistically preferred if it is less biased and less variable than others. In Section 1, the authors mentioned that maximizing the adjustment information formalizes the common intuition to choose adjustment sets that maximally constrain the effect variable and minimally constrain the cause variable. I am wondering whether and how this “common intuition” is related to statistically preferred estimators. (Note that, the variance is  a measure of variability, but not the only one.) Besides, please provide a more detailed explanation of the linkage between this “common intuition” and the definition of adjustment information.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have described the limitations in Section 4.", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}, {"review_id": "_BL14hI5qg", "reviewer": "Reviewer_rosK", "summary": "In this paper, the authors propose a criterion to select the optimal adjustment set. The authors propose a criterion (Eq. 4 and Def. 2) based on information theory to select which adjustment set is best. And they build the connections between this criterion to the common used minimized asymptotic variance (Eq. 7) for linear Gaussian causal model. Based on these, they propose a necessary and sufficient criterion for the existence of an optimal set. However, it is inefficient due to the large number of comparisons. Hence, the authors present an algorithm to find the o-set, which could be as the optimal set under some conditions as Thm. 3.", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "After rebuttal: The authors have addressed my questions. I think this paper tackles an interesting problem. I am happy to see it accepted.\n\n-------------------------\nAlthough I try my best to read this paper, it is hard for me to understand the details. Hence if some parts I summarized in the ``Summary'' part are inconsistent to the paper, I will be grateful if the authors could point them out.\n\nThe proposed method is technically solid. And the experimental results are convincing. My main concern is about the motivation to provide the criterion that does not depend on the distribution. As shown by ``Our goal is to provide graphical criteria for optimal adjustment sets, i.e., criteria that depend only on the structure of the graph G and not on the distribution'' on Line 143. I do not think this target is well motivated. In reality, we often have observational data. In that case, the distribution is known (although it is estimated) and which set is optimal for example in Example F seems not to be agnostic. Hence I look forward to an illustration for the motivation to provide the criterion that does not depend on the distribution. \n\nI am curious about the computational complexity of Lemma 2. It seems that this step will not cost so much because there are usually not so many valid adjustment sets. If it does not cost so much in reality, it is feasible to directly find the set by Lemma 2. What's more, if we know the (estimated) distribution of all the observed variables, we could just find the optimal st by Lemma. It seems to be more sensible than the o-set.\n\nIt is a bit puzzling by the sentence ``Z that maximally constrains the effect variable Y and minimally constrains the cause variable X'' on Line 110 implies. Could the authors please provide a detailed illustration? What does the constrain here mean?\n\n\nA suggestion:\nI understand the equation on line 124 as f(H_{Y|X,Z,S}-H_{X|Z,S})=\\frac{1}{\\sqrt{n}}e^{H_{Y|X,Z,S}-H_{X|Z,S}}, is it right? The format right now is not quite strict. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "T27Mmk8AHul", "reviewer": "Reviewer_wcA7", "summary": "The problem of selecting optimal backdoor adjustment sets to estimate causal effects in graphical models with hidden and conditioned variables is addressed.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper solves the problem of selecting the optimal backdoor adjustment sets by leveraging the theory of information theories. This paper is motivated by the interesting observation that the optimal admissible set is the set providing the most information (adjustment information, Def. 1). \n\nHowever, I have to confess that I couldn't capture the main claim of the paper. Does this paper answer that O-set in Def. 4 is the optimal backdoor admissible set whenever there exists a valid admissible set? It seems that the main claim in Thm.3 only guarantees the optimality of O-set for some special cases. \n\nDespite its interesting and novel approaches, I also had to agree that it is hard to understand this paper. I will discuss those in detail in **Readability** paragraphs. Here are some main questions regarding the contribution/novelty of this work:  \n\n- I am not sure about the distinction between SSR20 and the result of this paper. Up to my understanding, Thm 2 of SSR20 provides an (statistically-) optimal adjustment set for the nonparametric setting, and the only condition is the existence of the back-door admissible set. That is, I don't understand which particular points have never been addressed by SSR20.\n- In Eq. (3), why do you consider the conditioned variables S? Why did you prevent the possibility of S being arbitrary? (what's the role of the assumption S \\cap des(X) = \\emptyset in Line 87?)\n- Line 109-110, why is this a valid intuition? That is, what would be the specific benefits of invoking the information theory?\n\n\n## Readability \nHere are comments discussing which particular parts I felt difficult to understand. I also added my comments and recommendations to help to improve comprehensibility of the paper. \n\n- In general, it is not desirable to put the essential contents in the Appendix, because readers must go back and forth to parse the main and appendix at the same time. For example, it's hard for me to understand Thm. 3 and Line 78.\n- Please write the assumption more explicitly. I think one of the crucial assumption is the linear SCM setting (where all the variables are in the linear relation), as you wrote in line 125-126.\n- I'd like to add some overview or guideline for every starts of the section, Definitions, Theorems, and Lemma. For example, I couldn't guess what authors will discuss in Section 2.2 even after reading the first paragraph.\n- In Line 62-63, I'd recommend to give a detailed definition of ADMG, because readers might not want to look for the definition of ADMG from other sources. Without the definition, it's hard to parse the graph in Fig. 1; For example, what does the bidirected edge mean?\n- In Line 62-70, what is the definition of An(X), De(X)? Do they include the variable X? Please write it more explicitly.\n- In Line 62-70, I am not sure why you wanted to introduce the notion of MAG, given that this notion has never been used. Since MAG, PAG, Markov equivalence, \"visible edge\" are difficult concepts, I recommend to drop those in your work unless it's essential.\n- In Section 1.1, {X,Y} are singleton? Otherwise, I recommend to use the bold letters.\n- In Figure 1, what is the meaning of \"-\" and \"+\"?\n- I think the question in Line 106 is misleading and unclear what the research question is. Why did you give vancs (in line 105) before giving the research question? It distracts readers who want to focus on the research question. Also, the goal is supposed to minimize **asymptotic** estimation variance, but the word \"asymptotic\" is absent. Finally, since the line 106 is the main statement giving the research question, I think this deserves more than a line. Please highlight it more explicitly, because, as a reader, this is the very sentence that I want to know from Section 1.\n- In Line 122 and Eq. (7), I think this term is a mean-square error, not asymptotic variance.\n- I'd recommend to move line 125-126 after Assumption 2, given that this gives a concrete picture what you want to study within the assumption 2. Without concrete examples for Assumption 2, it's hard to parse this paper.\n- I don't parse Def. 2 (what is the meaning of \"no other Z' or for all other Z'\"?). Does it mean that Z is optimal if Jz is maximized among all Z' that is a valid adjustment set?\n- It's hard to parse Def. 4, because there is no definition for the notation that you used for defining C.\n- It seems that Thm 3 provides special cases when O-set is the optimal. It seems that there must be the optimal set even when the conditions in Thm. 3 fails (for sure, O-set is not the optimal one in this case). Can we have a result that we always can have a O-set whenever there are valid back-door admissible set?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "* As written above, I'd like to recommend to write the linear SCM assumption (line 125-126) more explicitly. \n* Does this result hold for the singleton X,Y? ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper aims to get around a known negative result in the theory of covariate adjustment due to Rotnitzky and Smucler: in hidden variable causal models, an adjustment set that minimizes the variance of the target parameter does not depend only on the model, but also on the element within the model (in other words, not only on the graph, but also on particular parameter settings).

To address this, the authors propose a criterion based on information theory that aims to maximally constrain the outcome, and minimally constrain the treatment. They then investigate cases when this criterion yields a variance-minimizing adjustment set.

While initially reviewer opinion on this paper was split, discussions with the authors led to a consensus that the paper is an interesting, novel, and worthwhile addition to NeurIPS proceedings.
Decision: Accept (Spotlight)