Identification and Estimation of Joint Probabilities of Potential Outcomes in Observational Studies with Covariate Information
Authors: Ryusei Shingaki
manabu kuroki
Venue: NeurIPS 2021 Poster (2021)
Keywords: Causal effect
Probability of causation

Abstract: The joint probabilities of potential outcomes are fundamental components of causal inference in the sense that (i) if they are identifiable, then the causal risk is also identifiable, but not vise versa (Pearl, 2009; Tian and Pearl, 2000) and (ii) they enable us to evaluate the probabilistic aspects of ``necessity'', ``sufficiency'', and ``necessity and sufficiency'', which are important concepts of successful explanation (Watson, et al., 2020). However, because they are not identifiable without any assumptions, various assumptions have been utilized to evaluate the joint probabilities of potential outcomes, e.g., the assumption of monotonicity (Pearl, 2009; Tian and Pearl, 2000), the independence between potential outcomes (Robins and Richardson, 2011),  the condition of gain equality (Li and Pearl, 2019), and the specific functional relationships between cause and effect (Pearl, 2009). Unlike existing identification conditions, in order to evaluate the joint probabilities of potential outcomes without such assumptions, this paper proposes two types of novel identification conditions using covariate information.  In addition, when the joint probabilities of potential outcomes are identifiable through the proposed conditions, the estimation problem of the joint probabilities of potential outcomes reduces to that of singular models and thus they can not be evaluated by standard statistical estimation methods. To solve the problem, this paper proposes a new statistical estimation method based on the augmented Lagrangian method and shows the asymptotic normality of the proposed estimators. Given space constraints, the proofs, the details on the statistical estimation method, some numerical experiments, and the case study are provided in the supplementary material.

Identification and Estimation of Joint Probabilities of Potential Outcomes in Observational Studies with Covariate Information
Ryusei Shingaki
Manabu Kuroki
Graduate School Engincering Science, Yokohama National University shingaki-ryusei -ryusei -koesnu. maroki-manabu--aa wwt.auu...mmmm.....mm.. ac.
Abstract
The joint probabilities of potential outcomes are fundamental components of causal inference in the sense that (i) if they are identifiable, then the causal risk is also identifiable, but not vise versa (Pearl, 2009; Tian and Pearl, 2000) and (ii) they enable us evaluate the probabilistic aspects of "necossity"; "sufficiency", and "necessity and sufficiency", which are important concepts of successful explanation (Watson, et al., 2020). However, because they are not identifiable without any assumptions, various assumptions have been utilized to evaluate the joint probabilities of potential outcomes,, e.g., the assumption of monotonicity (Pearl, 2009; Tian and Pearl, 2000), the inde- pendence between potential outcomes (Robins and Richardson, 2011), the condition of gain equality (Li and Pearl, 2019), and the specific functional relationships between cause and effect (Pearl, 2009). Unlike existing identi- fication conditions, in order to evaluate the joint probabilities of potential outcomes without such assumptions, this paper proposes two types novel identification conditions using covariate information. In addition, when the joint probabilitics of potential outcomes are identifiable through the proposed conditions, the estimation problem of the joint probabilities of potential outcomes reduces lo that of singular models and thus they can be evaluated by standard statistical estimation methods. To solve the problem, this paper proposes a new statistical estimation method based on the augmented Lagrangian method and shows the asymptotic normality of the proposed estimators. Given space constraints, the proofs,, the details on the statistical estimation method, some numerical experiments, and the case study are provided in the supplementary material.
1 Introduction
1.1 Practical Background
In practical science, it is crucial to evaluate the likelihood of one event causing another event. For example, epidemiologists pay attention determining the likelihood of particular exposure being the cause of particular disease. In order to assess such likelihood from statistical data, the probabilities of causation have been developed, which can be divided into "necessary causation", "sufficient causation", and "necessary-and-suffici"" In addition, recently, the field of explainable artificial intelligence (XAT), it has been pointed out that "necessity", "sufficiency", and "necessity and suffciency" are the important concepts of successful explanation and the probabilities of causation play an important role
causation".
35th Conference on Neural Information Processing Systems (NeurIPS 2021).

---

in evaluating these concepts from the probabilistic aspects [0
For the last decades, the probabilities of causation received increasing attention in the fields of epidemiology, risk analysis, legal reasoning, artificial intelligence, and policy analysis (e.g., 4 10
22 35
22 26
20
35
1.2
"Necessity", "Sufficiency", and "Necessity and Sufficiency"
Necessary cause is often used in the tort-law liability. When a 60-year-old man who worked in an asbestos textile plant develops lung cancer, we are interested in the probability that asbestos had caused it. This quantity to be estimated has been commonly referred to as It measures how necessary the exposure is for the development of the disease. Pearl 24 defined it as "Probability of Necessity (PN)", which means "the probability that disease would not have occurred in the absence of exposure, given that discase and exposure did in fact occur". The estimation of PN is important to provide a scientific basis for compensation in tort suits.
"attributable fraction" or "assigned share" 4 19
Sufficient cause is often used in health risk assessment. In practice, it is important for public policy makers to predict how health risk factor would endanger a healthy population 15 16 [21. The measure for it has been referred to as "susceptibility", : which measures how sufficient risk factor is to make an individual develop a discase. Pearl 24 defined the measure of such prediction as "Probability of Sufficiency (PS)", which means "the probability that a healthy unexposed individual would have developed the disease had he been exposed".
The assessment of PS helps public policy makers to make or revise regulations in order to minimize the occurrence of diseases.
With the definitions of PN and PS, it would be a natural extension to inquire for the "Probability of Necessity and Sufficiency (PNS)", which shows how likely individual is to be affected by both ways 24. PNS measures the probability that an exposure is a necessary and sulficient cause for a disease, that is, an exposure is the actual cause of a disease. Then, different from previous works where "probability of causation" is only referred to PN [e.g., Pearl 2 defined three types of "probability of causation", PN, PS, and PNS.
Pearl 26 and Tian and Pearl 33 developed formal semantics for the probabilities of causation based on the structural models of counterfactuals. They presented the formal definitions of the probability of necessity (PN), the probability of sufficiency (PS), and the probability of necessity and sufficiency (PNS). These probabilities of causation are formulated based on the joint probabilities of two potential outcomes. Since one cannot simultaneously observe the results of the same subjects exposed and unexposed in reality, even successful randomized experiments have failed to identify these quantities 26 pp.284 285). Tian and Pearl [3 addressed the problem by showing how to bound these quantities using data obtained from experimental and observational studies. Their bounds are sharp under minimal assumptions concerning the data generating process. Kuroki and Cai [7 derived narrower bounds of the probabilitinn bilities of causation than Tian-Pearl's bounds using covariate information. Recently, Galhotra et al. [0] generalized Tian-Pearl's bounds from binary to arbitrary variables in the context of explainable artificial intelligence (XAI). However, it has boen often pointed out that these existing bounds are too wide to evaluate the probabilities of causation. To solve the problem, Tian and Pearl [3 and Galhotra et al. J0 also stated that the probabilities of causation are identifiable if the monotonieity can be assumed and the causal risks are identifiable and Pearl 26 showed that the specific functional relationships between cause and effect. lead the identification of the probabilities of causation.
1.3 Contribution
The aim of this paper is to discuss the identification and estimation problems of the joint probabilitics of potential outcomes, which provides a general class including the probabilities of causation. The joint probabilities of potential outcomes are the -uiccing-blocks of causal inference, because the causal risk is also identifiable if they are identifiable, but not vise versa. In the context of the natural direct and indirect effects [25], under the assumption of no unmeasured confounding, Robins and Richardson B0 stated that the joint probabilities of potential outcomes are identifiable if (i) two potential outcomes are independent or (ii) one potential outcome is deterministically formulated as the function of the other potential
2

---

outeome. In addition, in the context of unit selection problems, Li and Pearl 20 showed that the linear combination of the joint probabilities of potential outcomes is identifiable under the gain equality. These existing researches show that the evaluation of the joint probabilities of potential outcomes play an important role in solving various problems of causal inference. Although considerable efforts have been devoted toward establishing the identifiabiliyy criteria and solving the confounding problem of causal risks, there has been limited discussion on how to identify the joint probabilities of potential outcomes when the present assumptions are violated.
To solve the problem, this paper proposes two types of novel identification conditions for the joint probabilities of potential outcomes without existing assumptions one uses both causal risks and one covariate (Theorem I and the other uses two covariates without causal risks (Theorem 2) The results of this paper also provide a counterexample to "sensitivity to the generative process" [66] pp. 284-285], in the sense that it is not required to specify the functional relationships that connect a cause and an effect. In addition, unlike the effect restoration proposed by Kuroki and Pearl [8] the assumption of the given number of categories of an unmeasured confounder is not required. Furthermore, from Theorem 2 it is remarkable that there are some situations where the causal risks are identifiable even if the existing identification conditions do not hold.
When the joint probabilities of potential outcomes are identifiable through the proposed conditions, the estimation problem of the joint probabilities of potential outcomes reduces to that of singular models and thus they can not be evaluated by standard statistical estimation methods. To solve the problem, this paper proposes a new statistical method based on the augmented Lagrangian method 3 and shows the asymptotic normality of the proposed estimators. Given space constraints, the proofs, the details on the statistical estimation method, some numerical experiments, and the case study are provided in the supplementary material.
2 Preliminary
This section introduces the potential outcomes used to discuss our problems. For simplicity,
we consider the case where an exposure variable X E {z1, # (xy
exposed; zy is unexposed) and an outcome variable Y E (81, (v1 is discase; yo is non-discase) are dichotomous. In addition, let p(X = z, Y = ./) = p(x, p(Y = y = x) = ply x), and p(X = z) = p(z) be the joint probability of (X = (x. y), the conditional probability of Y = y given X = x, and the marginal probability of = x for x E {01, and y E (v1, Jo respectively. Similar notation is used for other probabilities. Here, in this paper, will assumee that readers are familiar with the basic theory of causal inference 13 26 Especially, for the graph-theorelic terminology and the basic theory of structural causal models used in this paper, we refer the readers to Pearl 26
In principle, for x E {x1, hoh, the i-th of the N subjects has a potential outcome (i) that would have resulted X had been x, denoted as X(i) z. Here, note that the subject ensures a deterministic relationship between two variables X and Y in the semantics of structural causal models [6]. In addition, the present paper assumes the stable unit treatment value assumption, which can be sunmarized of any subject does not affect the outcomes of the other subjects (no interference) and (ii) the exposures of all subjects are comparable (no variation in exposure). Thus, when the N subjects in the study are considered as random samples from the population of interest, (i) and Yz (i) are referred to as the values of random variables X and Y, respectively, and thus the causal risk of X = x on Y = y is defined as p(Y² = y). Similar notation is used for other potential outcomes.
follows: (i) the exposure status
For the i-th subject,, the potential outcome Y, is obscrved only if X is x. This property is called the consistency 13 26 28 which is formulated as X = r + (i) Y()). When a randomized experiment is conducted and compliance is perfect, since X independent of (Ynor Yr, ), the causal risk is identifiable and is given by p(Y, y) ply 1)). Here, "identi- fiable" means that the causal quantities, such as p(y. =Y=pp y), can be estimated consistently from a joint probability of observed variables. In contrast, when it is difficult to conduct
3

---

randomized experiment and only observational data are available, we can evaluate the causal risk according to the graphically, the back-door criterion [26] In other words, for the exposure X, if there exists such set S of observed covariates that X is conditionally independent of (Y Y2,) given
condition [T or
S, we can say that treatment assignment is conditionally ignorable given S. Then, the causal effects are estimable using as p(Y, = y) = Ex[p(y] x, S)]. Here, Es x, S)] is the expectation of p(y x, S) regarding S. Although there are other identification conditions
that can be used to solve our problem [e.g., 26 34 this paper does not cover them due to space constraints.
Following Pearl 26 we define three probabilities of causation, that is, the probability of necessity (PN), the probability of sufficiency (PS), and the probability of necessity and sufficiency (PNS). PN stands for the probability p(Y., = yo X 31, Y y= =/=1 y1) that a disease would not have occurred in the absence of an exposure, given that the exposure and the discase occurred. PS stands for the probability p(Y,, = y1 X zo. Y = yo) that a healthy unexposed subject would have contracted the disease upon exposure. PNS stands for the probability p(Y,, = V1, Yzu = y/0 that an exposure is a necessary and sufficient cause for a disease. Since PN, PS, and PNS involve the joint probabilities of potential outcomess, even under successful randomized experiments, they cannot be estimated from the observed data without additional assumptions 26
3 Identification
3.1 Case 1: Identification based on one covariate with causal risks
According to Pearl 26 letting U be the set of all discrete and continuous covariates that could affect X and Y, both observed and unobserved, we consider the problem of evaluating the joint probabilitics of potential outcomes based on the directed graph shown in Figure Ia Here, a covariate Z, in Figure La is mcasured as proxy variable of U. Note that Z can be a set of discrete and/or continuous variables. In Figure la the directed edge from X to Y indicates that X may affect Y. In addition, the absence of directed edge from Y to X indicates that Y cannot be a cause of X, and the directed path from U to Y through X indicates that some elements of U may effect Y mediated by X. Figure La also graphically represents the data generating process
Y = gy C,,,, (U, (q), 2 (U,
(1)
where Eg. Ey. and ey are independent random disturbances and they are also independent of U. When the structural equation models, such equation (I), are used to represent the data generating process, the corresponding graph shown in Figure la is called a causal diagram.
The situation such as Figure la However, in this paper, unlike Kuroki and Pearl [+] and Pearl [27], U can include uncertain number of all discrete and continuous covariates that influence the way a subject responds to exposures. Thus, in many siluations, it is reasonable to assume the existence of a covariate Z that is independent of (X,Y) given U. Here, note that the independence assumptions between two observed variables would be affected by partitioning the states of U U U Ey Ex).
also discussed in Kuroki and Pearl 18 and Pcarl 2
In the situation shown in Figure Ia irrespective of the complexity of U (Ez, Ey. E2) the impact of U on Y remains restricted to the modification of the functional relationships between X and Y. This yiclds four functions for two dichotomous variables X and Y, and thus the value taken by Ú ((z: Cy. selects one of these four functions 26 Considering these observations, according to Rothman et al. 32] p. 59), the states of U Ey))) are divided into the following four potential outcome lypes:
"1 (Yn y1, Y.o = y1) represents a "doomed" situation where exposure is irrelevant because disease occurs with or without exposure.
U2 (Yz, = y/, Y,, = yo) represents "causative" situation where discase occurs if and only if subjects are exposed.
U3 = (V., = yo, Yop = y)) represents a "preventive" situation where disease occurs if and only if subjects unexposed.
4

---

U
z
(Y,, Yz
X
Y
(a) Unobserved covariates
(b) Potential outcomes
Figure 1: Graphical representation in Case 1.
141 -V= yo: 2o =)) represents an "immune" situation where exposure is irrelevant because disease does not oceur, with or without exposure.
According to this partition of the states of J (Ez, C2), it re-defined as a variable U laking value (u E {ui, 02, U3, u4)). Then, similar to the setting of the non-compliance problem by Pearl 26 ch.8), throughout the paper, we assume that the recursive factorization of the positive joint probability of X, Y, and Z, p(a,, y. 2), given
p(a, v,,)
Σ ply 1,,,, u:)p(x |u:)(( u:)p(u:)
according to Figure 1b It is also assumed that other joint probabilities of X, Y, and Z can be rewritten in a similar way. Then, this factorization implies that X is conditionally independent of Z, given U, denoted as X 11 Z U, which is used in the context of the impact evaluation problem of the program 23 addition, from the consistency property, we have
pf 2, uz) p(11, u:),
p( Vo. 2, (,) = 0, p V1, 2, uz) = O, == 33,4 4 p(ro: 20 u.) 0, 1,3,3 p( Ml, u:) = 0, 2,4,
1,2,
p(=1, Vo. u4) p(11, u;),
p( 2, u²)
2,
p(zo, No: uz) p(r0, ui),
Thus, for example, p( V1 =), pl=) and p(Y,, v) can be rewritten as
4 p(x (;)p(z((u)p())))) p(z) = Σ p(=l (.)(())) Σ
2
p(11, v1,
i=1
PV,, = yı, =) = /
Additionally, the other joint and marginal probabilities of X, Y, and Z can be rewritten in a similar way. Then, when Z is a variable with the number of values 2 4, say 21...., 24, letting
1 p(æ1, 31)
p(21)
pl22)
p(2z)
p(r1, y1, 21)
p(a1, 11 22)
pl 11 1/1, 23)
p=
(2)
plai, 30)
p( y0, 21)
p(ar, Vo: 22)
p , yo, 23)
p( 1/1)
p( V1. 21)
p( 22.
p 1/1, 23)
p(21)
p(22)
p((3)
Q = p(Y,, = v1) p(Yzo -1)
p(Y.. 21)
p(Y,, = 81 22)
p(Y,; = 8/, 23)
(3)
p(Y.o 21,
p(Yzo 91, 22)
p(Yzo = 81, 23
derive the following theorem:
Theorem 1 Letting Z be variable with the number of values k 2 4, say 21...., 24, P(Y,, =
y, Yoo = and p(Y., = y, Yz, = |x)) are identifiable (y.y/ E (11, No.3ol:a E (21, if the following conditions are satisfied:
Condition 1 p(Y,, = yı 2), p(Y. y1 12),), p(a1, (1.1,91222 2), p( Un, 2), p(a1, yo. z); and p( No. =) are available for E (21, ......... =1).
5

---

W
{Ya, Y
Figure 2: Graphical representation in Case 2
Condition 2 (X,Y) 11 Z U holds.
Condition 3 P is invertible and QP-1 consists of four distinct non-zero column vectors.
The proof is given in the supplementary material. Theorem 1 shows that PN, PS and PNS are identifiable. In addition, the causal risk is also identifiable from Theorem 1.
Here, we would like to state some remarks. First, the violation of the assumptions can be testable: as seen from the supplementary material, if the simultancous linear equation (A.12) in the supplementary material has no solutions, then at least one of the conditions does not hold. Second, although Condition seems to require that some distributions are available, it is not the case. Theorem 1 is applicable with a single observed distribution, say, p(a, y, 2, w) if both p(Y,, = y y1) and p(Y., = yyle [) are identifiable from the distribution. For example, regarding p(r,y. y, 2, w), when {2, W satisfies the back-door criterion relative to (X,Y), p(Y, = y 2 is identifiable and is given by
p(Y, = y Σp(e plu x, w)p(w
(4)
Second, Condition 3 implies that all potential outcome types exist and thus the monotonicity assumption (e.g. no-prevention assumption; p(uz) = 0) does not hold. In addition, it implies that X is not independent Theorem [ the causal risks are assumed to be estimable by the identification conditions in observational studies (such as the back- and front-door criteria (26] or by external studies (e.g., pilot studies scientific judgement). If we have prior knowledge that certain potential outcome type docs not exist in the situation discussed this section, that is, the monotonicity assumption, then the PN, PS, and PNS would be identifiable without the covariate Z K3 Theorem [ demonstrates that, when we observe one covariate associated with potential outcomes and the causal risks are available, the joint probabilities ies of potential outcomes can be constructed from the covariate and causal risks; this can render the joint probabilities of potential outcomes identifiable without the monotonicity assumption.
U, and thus X is not randomly assigned. Thus, in
3.2 Case 2:
Identification based on two covariates without causal risks
We will tackle the more difficult problem of estimating the joint probabilities of potential outcomes without prior knowledge of causal risks. We will show that, under certain conditions, the joint probabilitics of potential outcomes can be evaluated using only covariates. To see this, consider causal diagram shown in Figure 2 which is obtained by adding observed variable W to Figure 1b Then, when we consider the positive joint probability of X, Y, Z, and W, from the consistency property, we have
p( V1, 2, w. u;) p(11, 2, w. wz),
p(11, yo. z, w. uz) = 0, i= 1,2, p( 1101,91,20 z, w. uz) = 0,
p(r1, No. 2, w. u;) = p(11, z, w. wi),
i i=3,4. 4,
p(ro, y1, 2, w. u;) p(vo, w. uz), p(ro. yo, 2, we uz) p( 2, w. wi),
p( yo, 2, W. uz) = 0. 1=1,3. p(ao: V1, z, w. uz) = 0, i=
for z and w taken by and W, respectively. Thus, for example, p(y, 2, w (x1) and p(2, w [x)) can be rewritten as
4
ply,, z, w|x1) = Σ p(wlz, (eles w(x1) Σ pil

---

Additionally, the other joint and marginal probabilities of X, Y, Z, and W can be rewritten in similar way. Then, when Z and W variables with the number of values 4, such as 21...., , 24, for Z, and w1, wi for W, letting
1
p(21 x1)
p(22 x1
p(zy | x1)
Pei =
p(wi
x1
p( wi x
22, Wi
x
p(23, wi at
(5)
p(w2
x1
p W2 x1
pl 22, W2
x1
p(23, w2 25
p(Wa
xn
p(21, W3 (i)
p 22, W33
x1
p(23, why 25
p(21 x00)
p(22 200)
p x0))
p(w1
20))
p(21, w1 xo)
p 22, W1 xo)
p(23, w1 200)
Prg
(6)
ro)
p(21, w2 ro)
(22, W2 zo
( w2 2o
To)
p(21, Wa To)
p222, W/y
p(23, wy 20)
we derive the following theorem:
Theorem 2 Letting Z and W be variables by taking (2 4) values, say 21..... 24, for Z, and wi,..., W4 for W, p(Y,, y. Y. =y) and p(Y,, y. Yv, =y' |x) are identifiable (y.y'= E (111:00:: E (I1, if the following conditions are satisfied:
Condition 4 p(11, 2,,2,,), p(11, yo. w). p(zo, V1. 2, w), and p(zo, yo. 2, w), are available for (,,,,...,,,, 24} and w E (w),..,;
Condition 5 Both W IL Z (X,U) and X 11 Z U hold.
Condition 6 Both Ps, and Pzo are invertible.
The proof is given in the supplementary material. Although the probabilities comprising the unobserved variables are not fully identified in Kuroki and Pearl [8] Theorem | shows that the joint probabilities of potential outcomes p(ur)....,,pe .((()) are identifiable and the causal risks are also identifiable and are given by, for example,
P(Y,, y1) p(uz) + p(u2), p(Yz, v1) p(u)) + p(us).
Clearly, Theorem shows that PN, PS and PNS are also identifiable.
Estimation
When the joint probabilities of potential outcomes are identifiable through the proposed conditions, as seen from the proofs of Theorems (refer to the supplementary material A the estimation problem of the joint probabilities of potential outcomes reduces to that of singular models and thus they can not be evaluated by standard statistical estimation methods. To solve the problem, method [3. In this section, we focus on Case 2 in Section 3.2 For estimation in Case 1, see the supplementary material B.1
proposes new statistical method based on the augmented Lagrangian
Consider the matrices Pa. and Q2 that are derived by replacing p(2 [x), p(w [x), p(2, w [x), plyl x), ply, 2 x), ply, w 1.), and ply, 2, w (x) of P. and Q₂ with sample probabilities p( x), p(w] x)) pl2, 1)), Ply xx) Ply, I), P(y, x)) and P(y, (x), respectively (x {æ1, vo); (21...., } w E {wi...., Wa)). Then, for constant matrices Ma, and M
y E Uo): =
(see equation [A.33 of the supplementary material A.2 since we have
(8) from equation A. 2.2 of the supplementary material A.2 we estimate the matrix S (sce oquation (4.24 of the supplementary material A.2 as the solution θ (Bo))ssuga of the
SP M,S, (21,
optimization problem
1 - Qm, M,, + 1 0
minimize 8
(9)
2
subject to 0 (P,00-1)))) 11, = - 4 1 = a (01,20), (10)

---

Table 1: Conditional probability tables in simulation.
(a)
())
(b) p(W|()
(c) p(y = =|X,U
Z
==2
=3
Z 4
=
W = 2
W =
4
x=1
=
==1
7/10
1/10
1/10
1/10
7/10
1/10
1/10
1/10
1
U ==2
1/10
7/10
1/10
1/10
1/10
7/10
1/10
1/10
1
0
==3
1/10
1/10
7/10
1/10
1/10
1/10
7/10
1/10
0
1/10
1/10
1/10
7/10
1/10
1/10
1/10
7/10
0
0
(d) p(X
|W,0]
(e) p(U)
W W=1-
W =
W =3
W
U 1
21/46
18/43
18/43
18/43
1/4
2
9/34
21/71
9/34
9/34
1/4
U 3
9/34
9/34
21/71
9/34
1/4
U = 4
9/31
9/34
9/31
21/71
1/4
811 1, 05 1
0ak 11, i= 1= 4: 2, Σ k=2
(11)
where G1 = (1,0,0,0)T, e2 10, 63 (0,0,1,0) e4 (0,0,0,1)T, and 1 1,) In addition, || |l denotes the Frobenius norm of matrix. Equa- tion 110 is the constraint such that the first row of P comes to the valid estimators of (p(us I ..... p(u))=))
To solve the optimization problem, we use the augmented Lagrangian method B. When we obtain the estimator as the solution of the optimization problem, the estimator of = (p(u))....,,p)) P(()))) is given by the formula
à (P,, (( T-
where F(x) is the sample probability for p(æ). For details the asymptotic normality of the estimators and algorithms, see the supplementary material.
Numerical Experiment
In this section, we present a numerical experiment to examine properties of the proposed estimation method through the PNS p(u2) and the causal risk difference p(u2) - p(ua). For simplicity, letting X, Y, Z, W, and U be discrete variables, we consider the causal diagrams shown in Figure 2 where the joint probabilities of (X, Y, Z, W, are given according to Z,W, W) can be observed but U can not, the Table I Under the situation where (X, Z,
properties of the proposed estimators p(u2) and p(u2) p(us) of p(u2) and p(u2) - -((u),, respectively, are verified in a numerical experiment using the setting with sample sizes 100, 200, 1000, and 5000. In this situation, since the p(ug) and p(uz) p(us) are 1/4 and zero, respectively, the sample means of p(u2) and p(uz) p(ug) are expected to be close to 1/4 and zero, respoctively. Table 2 and Figure 3 show the basic statistics and the box plots of ((ua) and p(u2) (ua)) for 1000 replications with the given sample size n, respectively. The horizontal lines Figure show the true values of p(u2) and p(u2) pp(a).
From Table the sample means of P(u)) and p(u2) (ug)) are close to the true values and the sample standard deviations are smaller as the sample size is larger. Thus, it seems that the proposed estimation method provides the consistent estimators of p(uz) and p(u2) p((a).. In addition, from Figure 3 the interquantile ranges for p(uz) and p(uz) - p(us) are narrower and still include the true values even if the sample size is large. In contrast, it seems that p(u2) - p(ws) is symmetrically distributod and thus the asymptotic normality holds, but p(uz) may not. This is because the true value of p(uz) is relatively close to zero and Figure given as the zero truncated distribution of p(u2) for the finile sample size. Here, note that there are many outliers in each sample size. The outliers would occur when it is difficult to judge that. Condition S holds from observed data. For further discussion on the simulation experiments, see the supplementary material.

---

Table 2: Basic statistics of estimates based on the proposed method.
(a) ((12)
(b) p(x)) - -Maa)
100
200
1000
5000
100
200
1000
5000
Minimum
0.003
0.001
0.008
0.002
-0.906
-0.936
-0.957
-0.664
1st Quantile
0.176
0.210
0.234
0.241
-0.107
-0.076
-0.043
-0.033
Median
0.242
0.253
0.258
0.254
-0.015
-0.006
-0.005
-0.005
Mean
0.251
0.259
0.264
0.255
-0.026
-0.013
-0.025
3rd
0.305
0.294
0.281
0.271
0.083
0.061
0.034
0.016
Maximum
0.890
0.915
0.874
0.726
0.833
0.893
0.867
0.527
S.C. 0.126
0.103
0.082
0.072
0.228
0.189
0.139
0.129
causal risk difference
0.75
0.5
PNS
0.50
0.0
-0.5
0.00
-1.0
100
200
1000
5000
100
200
1000
5000
sample size
sample size
(a) Boxplots for PNS
(b) Boxplots for causal risk difference
Figure Boxplots of estimates based on the proposed method
Discussion
Joint probabilities potential outcomes often appears in unit selection problems 20] the impact evaluation problem of social programs [1] the non-compliance problem of treatment effects [ 2), the principal stratification 9 the identification problems of natural direct and indirect effects 25 and the prevented and preventable proportions [6, and the explainability problem of artificial intelligence 10 22
In addition, if the joint probabilitics of potential outcomes identifiable, then the causal risks are also identifiable, but not vise versa. Therefore, the identification and estimation problems of these probabilities have been an important topic in causal inference. To solve the problems, the paper proposed (i) two types of novel ifecatiiittinn conditions using covariate information and (ii) new statistical estimation method based on the augmented Lagrangian method 3 To the best of our knowledge, there has been previously much less discussion how useful covariate information identify these probabilities in nonparametric systems when existing identification conditions do not hold. In addition, the estimation problem of singular models often appears in the context of causal inference, but scems that the solution has boen given the litcrature. Thus, the results of this paper extend the range of solvable identification and estimation problems of causal inference under the nonparametric causal models.
35
Finally, we assumed that both an exposure variable and an outcome variable is dicholomous. However, straightforward to extend our results from the case of dichotomous observed variables to the case of multivalued observed variables under certain assumptions. For example, multivalued or continuous outcome can be accommodated in the model using the event Y < y as an outcome variable. Then, for given y, the problem of the paper can be the identification and estimation of p(Y,, > y, Y. S ....... p(Y, y, Y. y) and thus the result of this paper are applicable to the problems. For the related discussion, refer to Galhotra et al. [0 and Kada et al. 14. In addition, when the exposure variable is continuous, according to Balke and Pearl 2] it is reasonable to assume that there exists an exposure interval around each 2, within which subject's outcome homogeneous. Under this assumption, it is possible to apply our results. However, in such cases, it may be difficult to obtain reliable statistics of the recovered probabilities due data sparseness. is left for future work.

---

Acknowledgments and Disclosure of Funding
We would like acknowledge the helpful comments of the four anonymous reviewers. This research was partially funded by JFE Engineering Corporation and Japan Society for the Promotion of Science (JSPS), Grant Number 19K11856 and 21H03504..
References
[1] Joshua D. Angrist, Guido W. Imbens, and Donald B. Rubin. Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91 (434):444 455, 1996.
2 Alexander Balke and Judea Pearl. Bounds treatment effects from studies with imperfect compliance. Journal of the American Statistical Association, 92(439):1171 1176, 1997.
3 Dimitri Bertsekas. Constrained Oplimization and Lagrange Mulliplier Methods. Athena Scientific, 1996.
[4] J Beyea and S Greenland. The importance of specifying the underlying biologic model in estimating the probability of cansation. Health Physics, 76(3):266 10:::99-212 274, 1999.
55] Ruth M. J. Byrne. Counterfactuals in explainable artificial intelligence (xai): Evi- dence from human reasoning. In Proceedings of the Twenty-Eightt International Joint Conference on Artificial Intelligence, IJCAI-19, pages 6276-6282. International Joint
Conferences on Artificial Intelligence Organization, 2019.
[6] Zhihong Cai and Manabu Kuroki. Variance estimators for three "probabilities of causation". Risk Analysis, 25()):1611 2005.
[7] Louis Anthony Cox. Probability of causation and the attributable proportion risk. Risk Analysis, 4(3):221 230, 1984.
8 A Philip Dawid, Monica Musio, and Rossella Murtas. The probability of causation. Law, Probubility and Risk, 16(4):163-179, 2017.
[9] Constantine E. Frangakis and Donald B. Rubin. Principal stratification in causal inference. Biometrics, 58(1):21-25 29, 2002.
[10] Sainyam Galhotra, Romila Pradhan, and Babak Salimi. Explaining black-box algorithms using probabilistic contrastive counterfactuals. arXiv preprint drXie:2103.11977 2021.
[11] Sander Greenland. Variance estimators for attributable fraction estimates consistent in both large strata and sparse data. Statist. Med. 6(6):701 708, 1987.
[12] James J. Heckman, Jeffrey Smith, and Nancy Clements. Making the most out of pro- gramme evaluations and social experiments: Accounting for heterogencity in programme impacts. The Review of Economic Studies, 64(4):487 535, 1997.
[13] G. W. Imbens and D. B. Rubin. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press, 2015.
[14] Akiko Kada, Zhihong Cai, and Manabu Kuroki. Medical diagnostic test based on the potential test result approach: bounds and identification. Journal of Applied Statistics, 40(8):1659 1672, 2013.
[15] Muin J. Khoury, W. Dana Flanders, Sander Greenland, and Myron Adams. On the measurement of susceptibility in epidemiologic studies. Americon Journal of Epidemiology, 129(1):183 190, 1989.
[66 Muin J. Khoury, Quanhe Yang, Marta Gwinn, Julian Little, and W. Dana Flanders. An epidemiologic assessment of genomic profiling for measuring susceptibility to common diseases and targeting interventions. Genetics in Medicine, 6:38 47, 2004.
10

---

[17] Manabu Kuroki and Zhihong Cai. Statistical analysis of 'probabilities of causation' using co-variate information. Scand. J. Statist., 38(3):564-577 2011.
[18] Manabu Kuroki and Judea Pearl. Measurement bias and elfect restoration in causal inference. Biomelrika, (2)(2):4 437, 2014.
[19] Stephen W. Lagakos and Frederick Mosteller. Assigned shares in compensation for radiation-related cancers. Risk Analysis, 6(3):345 357, 1986.
[20] Ang Li and Judea Pearl. Unit selection based on counterfactual logic. In Proceedingss of the Twenty-Ei Eighth International Joint Conference on Artificial Intelligence, pages International Joint Conferences on Artificial Intelligence Organization, 2019.
[21] Josephine A. Mauskopf. Projections of cancer risks attributable to future exposure to asbestos. Risk Analysis, 7(4):477-486) 1987.
[22] Ramaravind K Mothilal, Divyat Mahajan, Chenhao Tan, and Amit Sharma. Towards unifying feature attribution and counterfactual explanations: Different means to the same end. Xix preprint NKw:2011.04911 2020.
[23] Cuong Nguyen. An introduction to alternative methods in program impact evaluation. Journal of Economic and Social Thought, 3(3):349 375, 2016.
[24] Judea Pearl. Probabilities of causation: Three counterfactual interpretations and their identification. Synthese, 121(1):93 -19,,,99--99,,,, 1999.
[25] Judea Pearl. Direct and indirect effects. In Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, pages 411-420, San Francisco, CA, USA, 2001. Morgan Kaufmann Publishers Inc.
[26] Judea Pearl. Causality: Models, Reasoning and Inference. Cambridge University Press, 2nd edition, 2009.
[27] Judea Pearl. On measurement bias in causal inference. In Proceedings of the Twenty- Siath Conference on Uncertainty in Artificial Intelligence, pages 425 432, Arlington, Virginia, USA, 2010. AUAI Press.
[28] James Robins. The analysis of randomized and non-randomized aids treatment trials using a new approach to causal inference in longitudinal studies. In L. Sechrest, H. Freeman, and A. Mulley, editors, Health Service Research Methodology: A Focus on AIDS, pages 113 159. U.S. Public Health Service, Washington, DC., 1989.
[29] James Robins and Sander Greenland. The probability of cansation under a stochastic model for individual risk. Biometrics, 45 45(4):1125-1133 1989.
[30] James Robins and Thomas S. Richardson. Alternative graphical causal models and the identification of direct effects. In Patrick E Shrout, Katherine M Keyes, and Katherine Orustein, editors, Causalily and psychopathology: Finding the delerminants of disorders and their cures, pages 103 158. Oxford University Press, 2011.
[31] Paul R. Rosenbaum and Donald B. Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41-58 1983.
[32] K.J. Rothman, S. Greenland, and T.L. Lash. Modern Epidemiology. Lippincott Williams & Wilkins, 3rd edition, 2008
[33] Jin Tian and Judea Pearl. Probabilities of causation: Bounds and identification. Annals of Mathematics and Artificial Intelligence, 28(1):287-313, 2000.
[34] Jin Tian and Judea Pearl. A general identification condition for causal effects. In Proceedings of the Eighteenth National Conference on Artificial Intelligence, pages 667-573, USA, 2002. American Association for Artificial Intelligence.
11

---

[35] David Watson, Limor Gultchin, Ankur Taly, and Luciano Floridi. Local explanations via necessity and sufficiency: Unifying theory and practice. orXie preprint arXic:2108.146:: 14651, 2021.
[36] Kentaro Yamada and Manabu Kuroki. Counterfactual-bess prevented and preventable proportions. Journal of Causal Inference, 5(2), 2017.
12


Reviews:
[{"review_id": "ii_lpXRzlgz", "reviewer": "Reviewer_CSH4", "summary": "This paper proposes two nonstandard identification strategies for the joint distribution of counterfactuals. One is based on a single proxy $Z$ for the unknown confounder with at least $4$ levels, such that $p(y|do(z))$ is identifiable. The other one is based two proxies $Z$ and $W$ with at least $4$ levels for each. Both identification strategies are fully nonparametric. They are given by solutions of a complex system which are nonlinear in the observed distributions. Beyond identification, the paper also proposes estimation procedures and proves the asymptotic normality of the estimators. The estimation procedures are essentially the plug-in version of the system involved in the identification strategy. ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The identification strategies are mind-blowing. Most (partial) identification strategies are based on linear programming. But the proposed ones are certainly beyond that. I really enjoy the smart ideas. Furthermore, the assumptions for those two strategies appear to be reasonable in practice. Therefore, I think this work is both intellectually and practically interesting. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "All my comments are minor. \n\n(1) For case 1, don't you need $(X, Y)\\perp Z | U$ instead of just $X\\perp Z | U$?\n\n(2) It would be more clear if $S$ and $M_x$ can be defined in Section 4. Otherwise, Section 4 is not readable enough. If the page limit is a concern, perhaps Section 3.1 can be trimmed a bit because the main text only discusses the estimation for Case 2. \n\n(3) Suppose $Z$ (and $W$) have more than four levels, is it possible to combine all levels instead of just choosing three of them? A naive idea is to ensemble (9) with all subsets of three levels. But can we expand $P_x, Q_x$ by including all levels? \n\n(4) Could you provide some heuristic explanation on how you get these strategies? More broadly, is it possible to get some generic principle from these ideas (like linear programming for other problems). ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "edz6yxBhyCW", "reviewer": "Reviewer_7Tux", "summary": "This paper studies the identification and estimation problem of the joint probabilities of potential outcomes. To solve this problem, this paper proposes two sets of identification conditions using covariate information. Furthermore, this paper proposes a novel statistical estimation method based on the augmented Lagrangian method to evaluate the joint probabilities of potential outcomes under the proposed identification conditions. ", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper studies the problem of evaluating the joint probabilities, which is important in causal inference for two reasons. First, if the joint probabilities are identifiable, then the causal risk is also identifiable. Second, the joint probabilities are critical to evaluate the \"Probability of Sufficiency\", “Probability of Necessity”, and “Probability of Necessity and Sufficiency”.\n\nThe two sets of identification conditions for the joint probabilities seem to be novel. However, I remain concerned about the practicality of these conditions. It would be nice to give a few examples to illustrate when Conditions 1-3, and Conditions 4-6 could hold in practice. Moreover, it will be helpful to clarify the conceptual challenges in proposing these conditions.\n\nThe estimation approach based on the augmented Lagrangian method is a nice solution to estimate the joint probabilities, given that the standard statistical estimation methods do not work. I appreciate that the authors provide theoretical guarantees for their estimation approach. It would be nice if the authors could provide a formal and rigorous statement of their theoretical results (e.g. a theorem or proposition) with the clearly stated assumptions (as opposed to just a sentence in line 286 and a section (i.e. B.3) in the appendix to show the proof).\n\nRegarding numerical simulations, I appreciate that the desired properties are verified through simulations, but I remain concerned about the simulation setup. It would be nice to clarify why the numbers chosen in Table 1 are of practical relevance. In addition, it would be nice if the authors could demonstrate the performance of their estimation methods on real data sets. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Identification with more than two covariates: The authors provide two sets of identification conditions. One is based on one covariate with causal risk, and the other one is based on two covariates without causal risk. What happens if there are more than two covariates, and what happens if there is more than one covariate with causal risk? It is worth explaining how the identification conditions can be generalized to these more general cases if these conditions are generalizable. Otherwise, it is also worth clarifying why these conditions are not generalizable. \n\nNumerical simulation, practical relevance, and presentation of this paper: The authors mention some healthcare examples in the introduction, which are helpful to motivate the problem. If the authors could explain and interpret Theorem 1, Conditions 1-3, Theorem 2, and Conditions 4-6 using the examples in the introduction, the identification section will be much easier to read. In addition, this paper could have a broader impact if the authors could demonstrate the practical relevance of their methods through experiments on real data sets (e.g. the real data related to the heathcare examples mentioned in the introduction). \n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "Ke22ggmAbh7", "reviewer": "Reviewer_8HrH", "summary": "This paper solves the identification of the probability of necessity and sufficiency in the case of unobserved confounder and the existence of proxy variable(s). Authors provide a novel formulation to solve the problem (presenting an unusual system of linear equations. \n", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nquality\nThe paper is in good shape (see clarity)\n\noriginality\nThe paper provides an answer (under a certain condition) for an important causal question in an original way (although solving system of linear equations have bee investigated in different contexts)\n\nclarity\nThe paper is in general pleasant to read but there are several places where improvements can be made.\n\nFor example, conditions for Theorem 1 are too shallowly explained. How should we interpret P or Q? Why do we have a marginal probability at the top row and leftmost column? Why there is no probability with x0 and y0 in P? etc. It seems that one can replace the top and the leftmost with those unused probabilities. But I am not sure. There is too much gap unexplained. The same applies to Theorem 2. Px1 and Px2 (which can be represented in a single expression with just x\\in x1, x2). Is invertibility implies that proxies (i.e. covariates) Z and W contain sufficient information to relate U (similar to (Yx, Yx’)) and X? \n\nIn the estimation section, it is difficult to read through lines 274—282, which refer to the results in supplementary materials extensively. \n\nWhat is a singular model?\n\nEmpirical results section can be reduced (with Table 2 in Appendix). What can we do with outliers when Condition 6 doesn’t hold (or the matrix is close to singular)? It would be nice to plot invertibility test results together with the estimates (PNS and causal risk difference) to ensure that those outliers are mainly due to near-singular matrices.\n\n\nsignificance\nIn the context of the theory of causal effect identifiability, this paper provides a novel identification result where non-trivially constructed matrices and their inverse is used. \n\n\nlimitation\nAlthough some of the reviewers may point out the stringent assumptions this paper relies on. I do not consider them as the limitation but the ones that ‘enable’ the identification.\n\n\nminor\nAs far as I know, the paper needs to use the basic font provided by NeurIPS.\nLine 227, ito → it", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": ".", "review_text": "", "extra_scores": {"ethical_concerns": ".", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "7CUkGr4Uw3", "reviewer": "Reviewer_Mum6", "summary": "In this paper, authors propose a mechanism to calculate joint probabilities of potential outcome under the assumption that certain covariate information is available. The first part of the paper identifies conditions under which these probabilities can be estimated and the second part aims to use augmented Lagrangian method to calculate the probability.\n\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The problem of relaxing assumptions of prior techniques would be an impactful contribution for the literature on explainable AI.\n\nWeak points:\n1. Motivation: It is unclear how the presented techniques are applicable in practice. For example, the covariate assumption is not available in most real-world settings.\n\n2. Theory: It seems like the conditional independence assumption X\\indep Z | U makes the analysis straightforward. I am not sure how the analysis would extend if this condition does not hold. \n\n3. The intuition behind the proposed method in section 4 is unclear. Please add more details and discuss the proof techniques.\n\n4. The proof is discussed with respect to two cases. How does it generalize it to other graph structures?\n\n5.  Evaluation: The empirical evaluation of the presented techniques is very superficial. \na) The experiment is conducted on a very small synthetic causal graph. Please consider a real-world dataset that motivates the motivation and consider more than 4 variables.\nb) There is no comparison with prior techniques that make monotonicity assumptions. It is unclear if the proposed method really performs better.\nc) In Figure 3, it maybe better to compare PNS scores of different attributes and validate its correctness with respect to what prior techniques report.\n\n6. Paper is not well written. The notation is not defined clearly which makes it very hard to validate the correctness of some results.\n\n7. Also, certain claims about medical results, e.g. lines 187-192 are not well justified. Please update the introduction to discuss these use cases and define formal notation.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "None identified", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers have a favorable opinion about this paper after discussion.

Paraphrasing one of the reviewer comments, identification results for joint probabilities of potential outcomes are "very rare" and definitely crucial for many important quantities like probability of sufficiency, necessity etc., individual treatment effect. This paper provides a novel conditions and an estimation procedure for solving it. 

So clearly the contributions are fundamental, original and timely.


There is a reviewer comment asking for formal statement for the asymptotic normality, some empirical comparison and further examples for the theorems. I really hope authors can revise accordingly.
Decision: Accept (Poster)