A Cross-Moment Approach for Causal Effect Estimation
Authors: Yaroslav Kivva
Saber Salehkaleybar
Negar Kiyavash
Venue: NeurIPS 2023 spotlight (2023)
Keywords: Causal inference
Difference-in-Difference
Structural causal models
Potential outcome
Proxy learning

Abstract: We consider the problem of estimating the causal effect of a treatment on an outcome in  linear structural causal models (SCM) with latent confounders when we have access to a single proxy variable.
Several methods (such as difference-in-difference (DiD) estimator or negative outcome control) have been proposed in this setting in the literature. However, these approaches require either restrictive assumptions on the data generating model or having access to at least two proxy variables.
We propose a method to estimate the causal effect using cross moments between the treatment, the outcome, and the proxy variable. In particular, we show that the causal effect can be identified with simple arithmetic operations on the cross moments if the latent confounder in linear SCM is non-Gaussian.
In this setting, DiD estimator provides an unbiased estimate only in the special case where the latent confounder has exactly the same direct causal effects on the outcomes in the pre-treatment and post-treatment phases. This translates to the common trend assumption in DiD, which we effectively relax.
Additionally, we provide an impossibility result that shows the causal effect cannot be identified if the observational distribution over the treatment, the outcome, and the proxy is jointly Gaussian.
 Our experiments on both synthetic and real-world datasets showcase the effectiveness
of the proposed approach in estimating the causal effect.

A Cross-Moment Approach for Causal Effect Estimation
Yaroslav Kivva School of Computer and Communication Sciences EPFL, Lausanne, Switzerland yaroslav. kivwaeepfl..c
Saber Salchkaleybar School of Computer and Communication Sciences EPFL, Lausanne, Switzerland saber. saleehkaleebar@eptl. ch
Negar Kiyavash
College of Management of Technology
EPFL, Lausanne, Switzerland negar. migarkkiyavash@epf..co
Abstract
We consider the problem of estimating the causal effect of treatment outcome linear structural causal models (SCM) with latent confounders when have access single proxy variable. Several methods (such difference-in- difference (DiD) estimator or negative outcome control) have been proposed in this setting the literature. However, these approaches require either restrictive assumptions on the data generating model or having access to at least two proxy variables. We propose a method to estimate the causal effect using cross moments between the treatment, the outcome, and the proxy variable. In particular, we show that the causal effect can be identified with simple arithmetic operations on the cross moments if the latent confounder in linear SCM is non-Gaussian setting, DiD estimator provides an unbiased estimate only in the special case where the latent confounder has exactly the same direct causal effects on the outcomes in the pre-treatment and post- cost-treatment phases. This translates to the common trend assumption in DiD, which we effectively relax. Additionally, we provide an impossibility result that shows the causal effect cannot be identified if the observational distribution over the treatment, the outcome, and the proxy is jointly Gaussian. Our experiments on both synthetic and real-world datasets showcase the effectiveness of the proposed approach in estimating the causal effect.
this
Introduction
Estimating the effect of treatment (or action) on an outcome is an important problem in many fields such as healthcare [SJS17], social sciences [Gan10], and economics [IR]. Randomized control trials are the gold standard estimate causal effects. However, in many applications, performing randomized experiments are too costly or even infeasible, say due to ethical or legal concerns. Thus, estimating the causal effect from merely observational studies one of the main topics of interest in causal inference. This problem has been studied extensively in two main frameworks, potential outcome (PO) framework [Rub74] and structural causal model (SCM) framework [Pea09]. The main
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

quantity of interest in PO framework is the individual-based response variable, i.e., the value of outcome for specific individual in the population considering a particular value for the treatment. In SCM framework, a set of structural causal assignments are defined to describe the data generation mechanism among a set of variables. This set of assignments is often represented by a directed acyclic graph (DAG) to show the causal relationships among the variables in the model. It can be shown that the two frameworks are logically equivalent in the sense that any theorem in one can be translated to the other [PJS17].
(DiD) [L*11] is one of the most frequently used non-experimental methods to estimate the effect of a treatment by comparing the average of outcome before and after applying the treatment in a treatment and control group. In fact, 26 of 100 most cited papers published by the American Economic Review used some variant of DiD or wo-wway fixed effect (an extension to multi-group and multi-time slots) to estimate the causal effect [DCD22]. DiD is an estimation process in PO framework for the setting where we have access to population partitioned into control and treatment groups. The goal is to estimate the effect of treatment D on outcome Y where D is equal to one if treatment is given to individual and zero otherwise. It is also assumed that the value of the outcome is observed just before giving any treatment (this pre-treatment value is denoted by Z) and it can be seen as a proxy variable for latent common causes of D and Y. DiD method computes the causal effect by subtracting the difference of average outcome in two groups before applying treatment (i.e., E Z D 1] E Z 0]) from the one after the treatment (i.e., E| = 1] D = O). It can be shown the output of DiD is an unbiased estimate of the causal effect under some assumptions such as the parallel/common trend assumption which states that the outcome of the treatment group would have followed the same trend as the control group in the absence of the treatment (see (2) for the exact definition).
Difference-in-Diffe
Although the initial setting of DID is in PO framework, its counterpart in the SCM frame- work was considered in the negative outcome control approach [SRC+16). 16]. A negative out- come variable is a type of proxy variable that is not causally affected by the treatment. The causal graph in this approach is represented in Figure where the unmeasured common cause of D and Y is represented by a latent variable U and D is not cause of proxy variable Z. The causal effect of D on Y cannot be iden- tified from the observational distribution over (D,Y, 2) because of the common confounder U. However, imposing further assumptions on the SCM, the causal effect of D on Y can become identified. Such assumptions include monotonicity [SRC* 16). knowledge of the conditional probability P(Z [KP14], or having at least two proxy variables [KP14, MGTTI8, TYC 20. CPS 220,, all of which may not hold in practice (see related work in Section 4 for a more detailed discussion). Recently, [SGKZ20] considered linear SCMs with non-Gaussian exogenous noise' and proposed a method that can identify the causal effect for the causal graph in Figure I from the observational distribution over (D, Y. Z). The proposed method based on solving an over-complete independent component analysis (OICA) [HKO01]. However given the landscape of the optimization problem, in practice, OICA can get stuck in bad local minima and return wrong results [DGZT19].
Figure 1:
The suggested causal graph in SCM framework for the approaches in DiD and negative outcome control.
In this paper, we consider the setup in causal graph in Figure 1 in linear SCM where we have access to proxy variable Z. We propose "Cross-Moment" algorithm that estimates the causal effect using cross moments between the treatment, the outcome, and the proxy variable. Our main contributions are as follows:
We show that the causal effect can be identified correctly from the observational distribution there exists n E N such that for the latent confounder U, we have: E[U"] + (n "BB(U"=1BB(U" (Theorem 1). Under additional mild assumption (Assumption 3), this condition implies that our proposed method can recover the causal effect when U is non- non-Gaussian Additionally, when the observational distribution is jointly Gaussian, we prove that it is impossible to identify the causal effect uniquely (Theorem 3).
MMore precisely, most of the exogenous noises in the system can be Gaussian.
2

---

Unlike previous work [SGKZ20, AHZ2I, YGN which requires solving an OICA problem, the proposed approach only performs simple arithmetic operations on cross moments. Therefore, does not suffer from the drawbacks of OICA such getting stuck in bad local optima.
We show that DiD estimator in general provides a biased estimate of the causal effect over the data generated by the linear SCM consistent with the causal graph in Figure unless the latent confounder has exactly the same values of direct causal effects on the outcomes in the pre-treatment and post-treatment phases. Our proposed method does not require such strong restriction.
The structure of the paper is as follows. In Section 2, we define the notation and provide some background on DiD estimator. In Section 3, we describe Cross-Moment algorithm and show that recovers the true causal effect under mild assumptions We also show that DiD estimator is in general biased if the data generative model follows a linear SCM. In Section we review the related work. In Section we evaluate the proposed algorithm experimentally and show its superior performance compared to the state of the art. Finally, we conclude the paper in Section
the distribution of the latent confounder.
2 Preliminaries and Notations
Throughout the paper, we denote random variables by capital letters and their realizations by small letters e.g., X and x, respectively. Bold capital letters are used to specify set of random variables and their realizations are denoted by small capital letters (e.g., X and x).
A SCM M over set of random variables V is defined by set of assignments {X == J (Pug (X), ex)]xev where ex the exogenous noise corresponding to X and (Xg S V. is assumed that the exogenous noises are mutually independent. Let us denote by O and U, the set of observed and unobserved variables in V, respectively. Note that V OU U and O U 0.
The set of assignments in SCM M is commonly represented by a DAG. Let g = (V, E) be a DAG with the of vertices V and set of edges E. For ease of notation, we the notation of V for the set of vertices in the graph. We also use the term "vertex" and "random variable" interchangeably. Each vertex in the graph represents some random variable and each direct edge shows direct causal relationship between pair of random variables. In particular, we say that X is a parent of Y or, equivalently, Y is child of X if (X,Y) E. We define Pag (X) set of all parents of X in graph G.
2.1 Difference-in-Differee (DiD)
Difference-ing inirencc-i-ddiffeeenee (DiD) was proposed in the PO framework in order to estimate the causal effect from observational studies under some assumptions. this framework, the population under study is divided into control and treatment groups and only individuals in the treatment group receive the treatment. In particular, the treatment variable D represents treatment assignment which equal to if the treatment was given and ( otherwise. Let (0)) and Y(1) (1) be two random variables representing the outcome under treatment value D and D 1, respectively. Denote the value of the outcome right before administering the treatment by Z and assume it is measurable. Our goal is to obtain the average causal effect the treatment group E|Y (1) - Y (0) = 1]. DiD estimate of the average causal effect equals:
D 1] - E oJ) (E[Z) 1] B oJ).
(1)
This quantity is an unbiased estimate of the average causal effect as long as the following assumptions hold.
Stable Unit Treatment Value Assumption (SUTVA):
Y = DY + (1 (0).
Common trend assumption:
( - = (080))Z((01))) Z(0 0).
SUTVA states that the potential outcome for each individual not related to the treatment value of the other individuals. The common trend assumption states that there would be the same "trend" in both groups in the absence of treatment which allows us to subtract group-specific means of the outcome in estimating the average causal effect in (1).
3

---

3 Methodology: Cross-Moment Algorithm
In this section, we propose Cross-Moment algorithm to estimate the causal effect of treatment D on outcome Y. Throughout this section, we consider linear SCMs, i.e., each random variable in SCM M is linear combination of its parents and its corresponding exogenous noise. More precisely, the linear assignments in M for the causal graph in Figure 2 are:
U == Euz
Z = a,U + 0z + C22
(3)
D = a²U + cd = addu + Ed:
Y = 3D + 7 + cy (oaa + 7)) Bed + Ey:
Without loss of generality, we assume that Eu. Ey. Eg. ea are arbitrary random variables with zero mean. This assumption always be achieved by centering the observational data.
Moreover, we assume that the only observed random variables are given by O (DDY,, Y, z}. Our goal to identify (the causal effect D on Y) from the distribution over the observed variables O. We consider SCMs that satisfy the following assumption.
ag
Odd
Assumption 1. In the linear SCM given by (3), 0g # 0 and Var(ea) O.
Figure 2: The considered causal graph with linear
Assumption 1 is necessary for identifying the causal effect. In particular, if a; 0, the di- rected edge from U to Z is removed and the causal effect cannot be identified even all the exogenous noises are non non-Gaussian zero almost surely (as we assumed that all the exogenous noises are mean zero). In this case, we can construct another SCM M' which encodes the same observational distribution as our original SCM but results different value the causal effect of on Y compared to the original SCM. More specifically, in this SCM, we delete the directed edge from U to Y and change the structural assignment Y to := (33 + 7/0aa) D + Ey. Hence, the assumption Var(ea) is necessary for the unique identification of the causal effect.
assignments in the SCM framework.
shown in [SGKZ20]. Moreover, if Var(ca) O, then ca is
Under Assumption 1, it can be shown that:
Cov (ow(D,Y) Cov(YY
(4) where Cov(A, A, B) denotes the covariance of random variables A and B and Var(A) is the variance A. 3 is identifiable as long as the ratio as/a, is known as we can obtain Cov(D,Y), CYv Z), Var(D), and Cov(D, Z) from the observational distribution. In the sequel, we will show how this ratio can learnt as long as e has bounded moments.
3 Var(D) Gui Cov (D,
Assumption 2. For all n N, assume that: E[ea] op.
When the bounded moment assumption of 2 holds, the following theorem provides an approach for recovering aa/az.
Theorem 1. For variables Z and D as defined in (3), under Assumptions 2, can be determined uniquely if 3n E N such that:
* (n2
2
(5)
where é. Vargae
The detailed proof of the Theorem is provided in the Appendix A.
It is interesting to for what families of distributions, the condition in Theorem is satisfied. Assume (5) is not satisfied. Recall that from the definition of SCM (3), E[ē.] 0 and E
E|DZ]. These in combination with E [e]]] (n 1) [2n
E [2]] for any N determine

---

X1
Figure 3: Example of causal graph extended with two observed covariates, two latent confounders, and corresponding proxy variables.
uniquely all the moments of é. More specifically, recursively solving for E [[c]] we have E [ew]] where n!! denotes double factorial. Double factorial notation n!! denotes the product of all numbers from to n with the same parity as
(n 1).!B(((()) for even > and E [< 0 for odd
n. Specifically the moments of Gaussian distribution satisfy the aforementioned moment equation.
Therefore when c, is Gaussian, we cannot identify the causal effect. Under some mild technical assumption on e (see Assumption 3 in the following), can prove that the moments of e uniguely determine its distribution. As a result, as long as is non-Gaussian, we can identify the causal effect.
Assumption 3. We assume that there exists some 8 such that the power series Σx B(ea)* converges for any 0 <r <.
Corollary 1. Under Assumptions 1, 2 and 3, the causal effect can be recovered uniquely as long is not Gaussian.
[SGKZ20), it was shown that can be recovered as long as all exogenous noises are non-Gaussian. Therefore, result relaxes the restrictions the model in [SGKZ20] by allowing ED, Ey, to be Gaussian.
Based on Theorem I, we present Cross-Moment algorithm in Algorithm that computes coefficient | from the distribution over the observed variables Z, D, Y. Algorithm is comprised of two functions GetRatio and GetBeta. In the proof of Theorem 1, show that oua/ag/ (num 1den) for the smallest n such that den jf where num and den are defined in lines and 7 of function GetRatio, respectively. Moreover, E Z] has the same sign as a ag/aa, ag and we can recover the sign of the ratio aa/a, from E[] Thus, in lines 8-10, for the smallest n such that den * O, we obtain the ratio aa/a; and then use function GetBeta to recover B.
For ease of presentation, we presented the Cross-Moment algorithm for the specific causal graph in Figure 2 with only one proxy. However, the Momsn- algorithm be utilized in more general setting with additional covariates and latent confounders such as in the graph depicted in Figure 3. This generalization is stated in the following theorem.
Theorem 2. Suppose that the linear SEM of (3) with the graph in Figure is extended with observed covariates X, non- Gans-iassian latent confounders U and proxy variables Z such that
none of the observed covariate is a descendant of any latent variable;
no latent confounder U U of variables D and Y an ancestor of any other latent confounder:
for each latent confounder U E there exists a unique proxy variable Z e Z which is not an ancestor of Y:
each latent confounder and its unique proxy variable satisfy the Assumptions 1, and 3.
Then the causal effect 3 from D Y can be computed uniquely from the observational distribution.
The proof of the theorem appears in Appendix A. The main idea of the proof is to reduce the problem set of sub-problems that can solved with Cross-Moment algorithm. mentioned earlier, an example of the causal graph satisfying the conditions of the theorem is depicted in Figure 3.

---

Algorithm 1 Cross-Moment algorithm
1: Function GetBeta D, Z, Y)
2: ratio == GetRatio D, 2)
3: [ = (F[DY) ratio TE (B((( ratio E[(2))
4: Return
1: Function GetRatio D, Z)
2: findRatio False
3: n : 2
4: while findRatio * Truc do
5: n.
+1
6: num :- DE / - (n (1-1))DD~~1E1D2
7: den :- EE[==1D]
8: if den + #0 then
9: ratio := sign sign( [E] pum 1
10: findRatio = True
11: end if
12:
end while
13:
Return: ratio
3.1
Impossibility Result
In the previous sections, we showed that the causal effect 3 can be identified if the distribution of latent confounder non-Gaussian. Herein, we show that no algorithm can learn uniquely if the observed variables are jointly Gaussian in any linear SCM defined by (3) satisfying the following assumption.
Assumption 4. In the linear SCM defined by (3), o.d + 0, F O and Var(cz) 0.
Theorem 3. Suppose that the observed variables in linear SCM defined by (3) are jointly Gaussian. Under Assumptions and 4, the total causal effect cannot be identified uniquely.
The proof of the Theorem 3 appears in the Appendix A. The key idea in the proof is show that there exist two linear SCMs that encode the same observational distribution and are consistent with the causal graph in Figure 2 but the causal effect of D Y has different values in these two models.
Note that it is known that the causal structure is not identifiable in a linear SCM with Gaussian exogenous noises [PJS17] Our impossibility result here is different from the non-identifiability result in linear Gaussian models. Specifically, in the linear Gaussian models, the goal is to recover all the coefficients in the linear SCM from the observational distribution. In our setting, we have additional knowledge of the exact DAG (restriction on the form the linear SCM in (3)), and the goal is to identify a specific coefficient (i.e., (3) from the linear SCM. Therefore, the model and need to infer less information about it. Still, we show that the target coefficient B cannot be determined in the causal graph in Figure 2 if the observed variables are jointly Gaussian.
have
constraints
3.2 Bias in DiD Estimator
Suppose that the data is generated from a linear SCM consistent with the causal graph in Figure 2. We show that DiD estimator biased except when the latent variable U/ has the exact same direct causal effect on Z that it has on Y, i.e., oz ?. Our Cross-Moment algorithm identifies the true causal effect without any such restrictive assumption the coefficients of the linear SCM.
DiD estimator given by the following linear regression [L* 11]:
= T 32 3DT,
(6)
where 31. 32. and 3 are the regression coefficients and T' is binary variable that equals zero for the pre-treatment phase and equals one otherwise. In the pre-treatment phase, Z (i.c., the outcome before the treatment) is predicted as 3gD and in the post-treatment phase, Y (the outcome after giving treatment to the treatment group) is predicted accordingly as B (6 + D. In order to obtain

---

the regression coefficients, the expectation of squared residuals over the population is minimized as follows (see Appendix for the derivations of the following minimization problem and subsequent equations in this section):
min B[(Z - B 8
- -31 -
This results in the following regression coefficients:
E|ZD]
B= EYD B[ZD]
B = 0,
322
D
DiD estimator returns B in the above equation as the estimation of causal effect which is equal to:
B = 3 + aa(y - az )E
(7)
E [
Thus, 9 is an unbiased estimate of only when y = atg. In other words, latent variable U should have the same direct causal effect on Z and Y. This is akin to the so-called common trend assumption which says that the average natural drift (here, the effect of U) is assumed to be the same across both the control and treatment groups This result is consistent with the findings in [RSBP23], which studied similar phenomenon in the DiD setting. In summary, whenever the common trend assumption is violated, the DiD estimator is biased.
Related work
In the past few years, there has been a growing interest in the literature to exploit proxy variables to de-bias the effect of latent confounders. A special type of such proxy variable is the so-called negative outcome which a variable known not to be causally affected by the treatment [LTC10]. For instance, the variable Z in Figure may be considered as negative outcome. In fact, [SRC* 16] interpreted DiD as a negative outcome control approach and proposed a method inspired by change-in-change [AT06] to identify the causal effect under the assumption that Y()) and Z are monotonic increasing functions of latent confounders and some observed covariates.
[KP14] considered three settings in causal inference with proxy variables: 1- There exists only one proxy variable such as Z as a negative outcome. In this case, for discrete finite variables Z and U, they showed that the causal effect
be identified if Pr(Z is known from some external studies such as pilot studies. 2- Two proxy variables, for instance Z and W are considered where U, Z, and W all discrete finite variables and Z does not have directed path to D Y. It has been shown that the causal effect is identifiable under some assumptions on the conditional probabilities of Pr(Y) D, U) and Pr(Z, WIX). In the setting, it is not necessary to know Pr(Z[U) but two proxy variables are required to identify the causal effect. 3- In linear SCM, [KP14] showed that 3 (the average causal effect of D on Y) can be recovered using two proxy variables. Later, [MGTT18] also considered setting with two proxy variables Z and W. Unlike the second setting in [KP14], here, Z and W can be parents of D and Y. respectively. For the discrete finite variables, they showed that the causal effect can be identified if the matrix PZ D) = d) is invertible. Moreover, they provided the counterpart of this condition for continuous variables. (SMNTT20) extended the identification result in [MGTTI8] 88, with weaker set of assumptions. Still, they required two proxy variables to identify the causal effect. Based on the results in [MGTT18], [TYC+20) 20] introduced a proximal causal inference in PO framework. Later, [CPS+ 20] provided an alternative proximal identification result to that of [MGTT18], again when two proxy variables were avaialble. More recently, [SI.Z 23] considered the PO framework under linearity assumptions for treatment and post-treatment phase. In this setting, the authors showedthe identifiability of causal effect if in the pre-treatment phase, the latent confounder jointly with observed outcome follows multivariate Gaussian distribution, and in the treatment phase, the exogenous noise of outcome variable is non-Gaussia..
In linear SCMs, to the best of our knowledge, the methods that can identify the causal effect with only one proxy variable Figure are based on solving an OICA problem. In particular, [SGKZ20] considered linear SCM with non-Gaussian exogenous noises in the presence of latent variables. They showed that under some structural conditions, the causal effects among observed variables can be identified and the causal graph in Figure 2 satisfies such structural conditions. [YGN+22] extended the results in [SGK7.20]. They showed that the causal structure (direction) can always be identified, and the causal effect can be identified up to equivalence classes depending on the graphical
7

---

conditions (the causal graph in Figure 2 is still uniquely identifiable). However, both proposed methods [SGKZ20, YGN 22] require solving an OICA, and existing algorithms for solving such a problem might get stuck in bad local optima. Recently, [AH721] provided two graphical conditions for the same setting in [SGK720] which are necessary for the identification of the causal structure. These conditions are closely related to the sparsity of the causal graphs. For the causal graph in Figure 2. the method proposed in [AHZ2I] for estimating the causal effect is the same as the one in [SGKZ20] and thus has the same drawback. Concurrent to our submission, [CHC+23) considered the problem of causal discovery for the linear non-Gaussian models. Under the specific assumptions on the true causal graph, they proposed algorithm for learning the causal structure well as the causal coefficients using high-order cumulants.
In PO framework, the setting of having just pee- tre-treatment phase and post-treatment phase can be generalized to the case with multiple time slots in the panel data model [ABD+21]. In this paper, we mainly focus on the setting with two groups and two time slots but one can also study the extensions of the current work for other settings in the panel data model described in the following. Consider two N x T matrices Y and D where N is the number of individuals in the population and ''' is the number of time slots. Assume that only the outcome for some individuals and time slots is observable. In particular: Yu (1 - Da)Y; (0) + D;r (1), where the realized outcome for individual at time slot is denoted by Y, a) DiD method has been proposed for the case T = 2, i... two time slots ppre-treatment and post- test-treatment phases). In the literature, other cases have been also studied for various assumptions on matrix Y. For instance, in unconfounded case [RR83, IR], the number of individuals is much larger than the number of time slots (N > T). and the treatment is provided only at the last time slot. Another setting is that of synthetic control [AG03, ADHI0, ADH15, DI16] where T >> N. In this setting, there is single treated individual (suppose individual N) and the goal is to estimate its missing potential outcomes for any t E [To, T] after administering the treatment at time To. The last setting considers N x T and two-way-fiedd-fle been proposed to estimate the causal effect (see for a survey on TWFE in [DCd20J. It is noteworthy that TWFE estimator equivalent DiD estimator for two groups and two time slots.
(TWFE) regression model has
5 Experiments
In this section, we first evaluate our algorithm on synthetic data and compare it to DiD estimator and as well as the related work in [KP14] which estimates the causal effect in lincar SCMs with two proxy variables. Further, we apply our algorithm to a real dataset provided by [CK93]. The implementation of the algorithm and additional experimental results are provided in https//gthhbb.om//kiiv//Cooo Homents-Method
5.1 Synthetic data
We generated samples according to the SCM in (3) and with all the exogenous noises distributed according to exponential distribution. Note that the distribution ca, i.e., the exponential distribution satisfies Assumptions 2 and 3. Therefore 3 is identifiable according to the Corollary 1.
Given the observational data, we estimated the value of /3 from the following four approaches:
1. Cross-Moment algorithm (proposed in this work).
2. DiD estimator of (6).
3. A simple linear regression model based on the following equation: y aZ + +3D..
4. Causal effect estimate for linear SCM with two proxy variables (proposed in [KP14]). In the experiments, we call this estimate "two-proxy" method.
It is noteworthy that we also evaluated the method in [SGKZ20] which uses OICA as a subroutine. Unfortunately, the performance was too poor to be included.
For each sample size, we sampled parameters Oz, Ocd, 8, randomly and then generated the samples of Z, D, Y accordingly to (3) (More details regarding the data generation mechanism can be found in Appendix B). We ran an experiment 10 times and reported the the average relative error for each value of sample size: err = 8-8
8

---

Exponcntial
Exponenbaal distribbion
10
2.-
3D
Osty Plann & NE2
Tee-Proyy Vathec
0.3
Nomberr
Skurber
(a) The average relative error of Mooos,M DiD estimator, and simple linear regression with one proxy variable.
(b) The average relative error of three variants of Cross-Moment algorithm and the two-proxy method in [KP14] when we have access two proxy variables.
Figure 4: The performance measure against the number of samples. Colored regions show the standard deviation of the err.
Figure 4a, depicts the performances of linear regression when we have access only proxy variable. The colored region around each curve shows the empirical standard deviation of (BB /). Cross-Moment algorithm outperforms the other two methods significantly. In fact, DiD estimator is biased if a * which occurs with measure as Ot2 and 7 are generated randomly. Moreover, DiD estimate is better than simple linear regression if y is not close to Oz. In the literature, has been noted that the parallel trend assumption (in linear SCM, this assumption is equivalent to the condition o= =Q) is violated if the scale of the proxy variable Z and outcome variable are different which can be the case in many practical applications [L* 11].
algorithm, DiD estimator, and the simple
We compared Cross-Moment with the two-proxy method in [KP14] when we have access to two proxy variables. In particular, assumed that there is additional proxy variable W such that W == a, + Ew- For Cross-Moment algorithm, we considered three versions: I Z". which estimates the causal effect by using only proxy variable Z (we denote this estimate by W", which estimates / from only proxy variable W (which we denote the estimate by (w), III "Cross-Moment: W-Z", which estimates B from aggregating the estimates of the methods and II. In particular, "Cross-Moment: W-Z" uses bootstrapping method (Monte Carlo algorithm for case resampling [ET94]) estimate the variances of estimates /3z and ßw. denoted by o² and 2 respectively. Subsequently, ,3 is estimated by combining two estimates 32 and Sw with an peverse-varianee weighting scheme [SHK11] where we give a higher weight to
3z), III (Cross-Moment
"22 the estimate with the lower variance: 03 + Bw + 032 + 3y
22w
/Va( (W) and .z. When Var(ew)) //ae/////(((//))
(ar(e=)/Var(2 Z) are small, the causal effect can be estimated with low estimation error from either Z or W as they contain low noise versions of the latent confounder U. In our experiments, we considered the case where Specifically, we Var //Vr((r) = 10 and Var(c,) Var(c:)/Var(c))) /Va((c.) = 0.1. Figure 4b illustrates the performances of the three aforementioned variants of Cross- method in [KP14]. "Cross-Moment: Z" has the best performance since it uses Z with less noise as the proxy variable. Moreover, "Cross-Moment: W-Z" has comparable performance by combining the estimates of 3z and Sw. The two-proxy estimate does not exhibit robustness and has a large average relative error for various values of sample size.
of the proxy variables (herein, W) is too noisy but not the other one.
algorithm and the two-proxy
5.2 Minimum Wage and Employment Dataset
We evaluate our method on the real data which contains information about fast-food stores (Burger King, Roy Rogers, and Wendy's stores) in New Jersey and Pennsylvania 1992, and some details about them such as minimum wage, product prices, open hours, etc [CK93]. The goal of study was to estimate the effect of the increment minimum wage in New Jersey from 4.25 to $ 5.05 per hour on the employment rate.

---

TWFE
Cross-Moment
With X
2.68
2.68
Without X
3.24
4.03
Table 1: Causal effect estimation of minimum wage employment level in the real dataset in [CK93].
The data was collected by interviews in two waves, before and after the rise in the minimum wage. The information was gathered from 410 restaurants with similar average food prices, store hours, and employment levels. In this experiment, stores from Pennsylvania and stores from New Jersey are considered as the treatment group. We define employment level Y as Y = Y + bY,, where Yp number of employees working part-time and Y is a number of employees working full-time.
treated
control group
First, reproduced the results in [CK93]. We considered an extended version of TWFE model [CK93]:
Y= + x a 3,D + 3DT,
where Y is the estimate of number of employees in the store, T binary variable that equals prior to raising the minimum wage and equals to 1 after the raise. D is equal to 0 if the store is in if the store
Pennsylvania and equal mation such as the opening hours, product prices, etc. á is also a vector of parameters corresponding to the vector X. We dropped all the stores from the dataset that contain NaN values after which 246 restaurants were left. The estimate of [ computed by TWFE is given in the first row of Table According to [CK93]. the estimate of 3 is equal to 76. The difference estimation is due to the slight difference in the features of the vector X, i.e., [CK93] added a few additional manually computed features to X.
New Jersey. X a vector that contains additional infor-
For the Cross-Moment algorithm, in order to incorporate the features X in estimating B, we first regressed Y on X and then used Xå instead of as the outcome. The result of applying Cross-Moment algorithm to this newly defined outcome is given in the first row of Table I and is very close the estimate by TWFE.
Finally, we assumed that the additional information X gathered during the interview is not available. Then TWFE model for the employment level takes the following form
AT 32 + 3D.
We used the previously me-processed dataset but dropped the columns corresponding to X. Sub- sequently, applied TWFE and Cross-Moment method to estimate 3. The respective estimates appear in the second row of Table 1, which stipulate the rise in the minimum wage had positive effect on the employment level.
Conclusion
We considered the problem of estimating the causal effect treatment on an outcome in the linear SCM where we have access to a proxy variable for the latent confounder of the treatment and the outcome. This problem has been studied both PO framework (such as DiD estimator) and SCM framework (such as the negative outcome control approach). We proposed method that uses cross moments between the treatment, the outcome, and the proxy variable and recovers the true causal effect if the latent confounder is non-Gaussian. We also showed that the causal effect cannot be identified if the joint distribution over the observed variable are Gaussian. Unlike previous work which requires solving OICA problem, our performs simple arithmetic operations on the cross moments. We evaluated our proposed method on synthetic and real datasets. Our experimental results show the proposed algorithm has remarkable performance for synthetic data and provides consistent results with previous studies on the real dataset we tested on.
Acknowledgments and Disclosure of Funding
This research was in part supported by the Swiss National Science Foundation under NCCR Automa- tion, grant agreement 51NF40_ 180545 and Swiss SNF project (00002122435555
10

---

References
[ABD*21] Susan Athey, Mohsen Bayati, Nikolay Doudchenko, Guido Imbens, and Khashayar Khosravi. Matrix completion methods for causal panel data models. Journal of the American Statistical Association, 1 1166536):171- 1730, 2021.
[ADHI0] Alberto Abadie, Alexis Diamond, and Jens Hainmueller. Synthetic control methods for comparative case studies: Estimating the effect of california's tobacco control program. Journal of the American statistical Association, 105(490):493-50) 2010.
[ADH15] Alberto Abadie, Alexis Diamond, and Jens Hainmueller. Comparative politics and the synthetic control method. American Journal of Political Science, 59(2):495-510 2015.
[AG03] Alberto Abadie and Javier Gardeazabal. The economic costs of conflict: A case study of the basque country. American economic review, 93(1):113-133) 2003.
[AHZ21] Jeffrey Adams, Niels Hansen, and Kun Zhang. Identification of partially observed linear causal models: Graphical conditions for the non-gaussian and heterogeneous cases. Advances Neural Information Processing Systems, 14:22822-228333 2021.
[AI06] Susan Athey and Guido W Imbens. Identification and inference in nonlinear difference- in-differences models. Econometrica, 74(2):431-49) 2006.
[Bro83] Gavin Brown. P. billingsley, probability and measure (wiley, 1979), pp. 532,f 28- 95. Proceedings of the Edinburgh Mathematical Society, (943))398-39) 1983.
[CHC+23) Ruichu Cai, Zhiyi Huang, Wei Chen, Zhifeng Hao, and Kun Zhang. Causal dis- covery with latent confounders based on higher-order cumulants. arXiv preprint 1rXi::2355.19582. 2023.
[CK93] David Card and Alan B Krueger. Minimum wages and employment: case study of the fast food industry in new jersey and pennsylvania, 1993.
[CPS CPS$+20 20] Yifan Cui, Hongming Pu, Xu Shi, Wang Miao, and Eric Tchetgen Tchetgen, Semipara- metric proximal causal inference. arXiv preprint arXiv:2011.084/ 2020.
[DCd20] Clément De Chaisemartin and Xavier d' H'Halltfouille. Two-way fixed effects estimators with heterogeneous treatment effects. American Economic Review, 110(9):2964 96, 2020.
[DCD22] Clément De Chaisemartin and Xavier D H'Hallffouulll. timators of intertemporal treatment effects. Technical report, National Bureau of Economic Research, 2022.
[DGZT19] Chenwei Ding, Mingming Gong, Kun Zhang, and Dacheng Tao. Likelihood-free overcomplete ica and applications in causal discovery. Advances in neural information processing systems, 32, 2019.
[DI16] Nikolay Doudchenko and Guido W Imbens. Balancing, regression, difference-in- differences and synthetic control methods: A synthesis. Technical report, National Bureau of Economic Research, 2016.
[ET94] Bradley Efron and Robert Tibshirani. An introduction to the bootstrap. CRC press, 1994.
[Gan10] 10] Markus Gangl. Causal inference in sociological research. Annual review of sociology, 36:21-47, 2010.
[HKO01] Aapo Hyvärinen, Juha Karhunen, and Erkki Oja. Independent Component Analysis. John Wiley & Sons, 2001.
[IR] Guido W Imbens and Donald B Rubin. Causal inference in statistics, social, and biomedical sciences (2015). Google Scholar Google Scholar Digital Lübrary Digital Library.

---

[KP14] Manabu Kuroki and Judea Pearl. Measurement bias and effect restoration in causal inference. Biometrika, 101(2):223-43) 2014.
[L+ Michael Lechner et The estimation of causal effects by differeene-in-diifferene methods. Foundations and Trends@) in Econometrics, 4(3):165-224, 2011.
[LTC10] Mare Lipsitch, Eric Tchetgen Tchetgen, and Ted Cohen. Negative controls: tool for detecting confounding and bias observational studies. Epidemiology (Cambridge, Mass. (3:))3883 2010.
[MGTTI8] Wang Miao, Zhi Geng, and Eric J Tchetgen Tchetgen. Identifying causal effects with proxy variables of an unmeasured confounder. Biometrika, 105(4):987-993, 2018.
[Pea09] Judea Pearl. Causality. Cambridge university press, 2009.
[PJS17] Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: foundations and learning algorithms. The MIT Press, 2017.
[RR83] Paul R Rosenbaum and Donald Rubin. The central role the propensity score in observational studies for causal effects. Biometrika, 70(1):41-55 1983.
[RSBP23] Jonathan Roth, Pedro HC Sant' Anna, Alyssa Bilinski, and John Poe. What's trending in biffference-in-diffeeeeee synthesis of the recent econometries literature. Journal of Econometrics, 2023.
[Rub74] Donald B Rubin. Estimating causal effects of treatments in randomized and nonran- domized studies. Journal of educational Psychology, 66(5):688, 1974.
[SGKZ20] Saber Salehkaleybar, AmirEmad Ghassami, Negar Kiyavash, and Kun Zhang, Learning linear non-gaussian causal models the presence of latent variables. Mach. Learn. Res., 21:39-1, 2020.
[SHK11] Bimal K Sinha, Joachim Hartung, and Guido Knapp. Statistical meta-analysis with applications. John Wiley & Sons, 2011.
[SJS17] Uri Shalit, Fredrik D Johansson, and David Sontag. Estimating individual treatment effect: generalization bounds and algorithms. In International Conference on Machine Learning, pages
PMLR, 2017.
[3] Kang Shuai, Shanshan Luo, Yue Zhang, Feng Xie, and Yangbo He. Identification and estimation causal effects using non-gaussianity and auxiliary covariates. arXiv preprint arXiv:2304. 1123300444499 14895, 2023.
[SMNTT20] Xu Shi, Wang Miao, Jennifer C Nelson, and Eric J Tchetgen Tehetgen. Multiply robust causal inference with double-negative control adjustment for categorical unmeasured confounding. Journal of the Royal Statistical Society Series B: Statistical Methodology, 82(2):521-540, 2020.
[SRC+16] Tamar Sofer, David B Richardson, Elena Colicino, Joel Schwartz, and Eric J Tchetgen Tchetgen. On negative outcome control of unobserved confounding as a generalization of differrence-in-dfffereem Mathematical Statistics. 31(3):348, 2016.
Statistical science: a review journal of the Institute of
[TYC+20] Eric Tchetgen Tehetgen, Andrew Ying, Yifan Cui, Xu Shi, and Wang Miao. introduction to proximal causal learning. arXiv preprint arXiv:2009. 10982, 2020.
[YGN+22] Yuqin Yang, AmirEmad Ghassami. Mohamed Nafea, Negar Kiyavash, Kun Zhang, and Ilya Shpitser. Causal discovery in linear latent variable models subject to measurement error. Advances in Neural Information Processing Systems, 35:874-888 -876, 2022.
12


Reviews:
[{"review_id": "q0vfn0uyUq", "reviewer": "Reviewer_dXHi", "summary": "The authors consider the estimation of causal effect in linear SCM with independent errors when there is a latent confounder U and one proxy variable of U (negative control outcome).\n \nThey generalize the DiD literature by relaxing the assumption of common trends and propose a general identification formula under the stated structural assumptions.\n\nThe authors show that under some restrictions on the latent confounder moments and on the U-Z, U-D relations, causal effects are uniquely identified nonparametrically using the identification formula. Furthermore, they propose a general estimation algorithm based on the cross-moments of Z and D.\n\nIn addition, the authors provide an ``impossibility result\" which shows that under fully gaussian linear SCM, causal effects can not be uniquely identified. \n\nThey illustrate their proposed method in simulation and a data example.", "questions": "In the data example, estimation using the cross-moments algorithm is performed on the residuals of the outcom~covariates regression. Are there any theoretical guarantees (e.g., similar to Theorem 1) when covariates are included? are the covariates also need to have a linear relation to the treatment/outcome/negative control for the uniqueness of $\\beta$?\n\nProximal learning (Tchegen Tchegen et al, cited by the authors) provides a flexible approach for estimating causal effects with latent confounders when there are at least two proxy variables. In practice, many studies do have more than one possible proxy variable. Do you think it is possible to extend the cross-moments algorithm for scenarios with more than one negative control (e.g., under linear SCM)?\n\n", "limitations": "As already stated, the theoretical results strongly rely on the linearity and independent error assumptions. The authors did not adequately address those limitations. ", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Correcting the bias due to residual confounding using proxy variables (negative controls) is an emerging topic in causal inference. The authors consider the more difficult task that seeks to correct the bias with only one proxy variable. \n\nUnder the assumed linear structural model with independent errors, the authors utilized a well-known identity that can be thus used as an identification formula. Theorem 1, which provides the uniqueness guarantees, is novel and motivates a computationally easy estimation algorithm, which is an improvement in comparison to other proximal learning methods.\n\nThe formal results are rigorous and nontrivial. The theoretical guarantees provide a meaningful illustration of the limitations of the proposed identification formula.\n\nThe paper is well-written and easy to follow. \n\n", "weaknesses": "The authors provide results only for linear SCM with independent errors. Both linearity and exogenous errors are fairly strong assumptions that are not likely to hold in practice. Moreover, the theoretical results heavily depend on both assumptions and are not likely to extend to other SCMs.\n\nOn line 117, D is assumed to be a binary treatment. On line 119, the authors explicitly define the causal estimand of interest as the average causal effects on the treated. However, the SCM (line 133) states that $D = \\alpha_dU +\\varepsilon_d$, which, coupled with the assumption of independent zero mean errors (line 134), yields that \n$$\\Pr(D=1)=E[D]=\\alpha_dE[U] + E[\\varepsilon_d]=0$$\nThat is, if D is binary, the SCM implies that it is a deterministic random variable that equals 0 with probability one. \nIn addition, in Algorithm 1, $num$ is identically the same for all $n$ whenever $D$ is binary. \nThe authors are most likely well aware of this inconsistency since in the simulation study $D$ is not taken to be a binary variable.\nTheir proposed method works well for non-binary D, but causal estimands should be adjusted accordingly.\n\nSection 3.2 (lines 209-228) is well known in the literature (see for example the recent review by Roth et al. 2023 ``What’s trending in difference-in-differences? A synthesis of the recent econometrics literature\").\n\nExperiments under misspecification (linearity, additional latent variables, etc.) are not presented. The robustness of the proposed methods is an open question. \n\n\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Regarding comments in weaknesses**\n1. [About linearity and independent exogenous noises assumptions] Independence of exogenous errors is the main and standard assumption in structural causal models (SCM) which is based on the principle of independent mechanism (For more discussion on why this is a reasonable assumption, please see Section 2.1 in [1]). Regarding the linearity assumption, please refer to the global response. \n\n[1] Peters, Jonas, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: foundations and learning algorithms. The MIT Press, 2017.\n\n2. [About binary treatments] Section 2.1 (lines 117, 119) is referring to the DiD approach and it does not pertain to our assumption which we state later in our linear SCM setting (lines 133-134). As mentioned in line 134, without loss of generality, we can assume that all the variables have a mean equal to 0, as we can always achieve this by centering the data. It is noteworthy that our method is applicable to both discrete or continuous variables that satisfy the linear equations in (3). In the DiD setting, for instance, if there is a binary variable $D$ that takes values 0 or 1 with probability $1/2$ each, it can be represented in the SEM by a binary variable that equals $-1/2$ or $1/2$ with probability $1/2$ each. Please note that the experiments for the Minimum Wage and Employment dataset are done for the binary variable $D$, which shows that our method works well for the binary case. We will clarify these details in the revised paper.\n3. [The reference \"Roth et al. 2023\"] Thanks for mentioning the reference. We will discuss it in the revised paper.\n4. [Experiments under miss-specification] Regarding additional latent variables, please refer to the global response. \nAbout linearity assumption, we carried out some experiments under miss-specification in the linear relations. For the edges from the latent confounder $U$ to observed variables $Z,D,Y$, we replaced the linear relations with nonlinear functionalities. In particular, $Z= 10\\tanh(\\alpha_zU/a)+\\epsilon_z$, $D= 10\\tanh(\\alpha_dU/a)+\\epsilon_d$, and $Y= \\beta D+ 10\\tanh(\\gamma U/a)+\\epsilon_y$, where $a$ is some constant in $[2,10]$. Please note that we kept the linear relation from $D$ to $Y$ as it is challenging to quantify the causal effect with a single value if the relation is non-linear. In fact, in the non-linear case, the causal effect depends on the value of the treatment. For instance,  one possible candidate to capture the causal effect is $\\partial \\mathbb{E}[Y|do(D:=d)]/\\partial d$,  which is a function of $d$.  \nThe average relative error against the parameter $a$  is depicted in Figure 1 in the attachment of the global response. The relations from the latent confounder to the observed variables become more non-linear for lower values of $a$. Cross-Moment method still has a decent performance compared with the baselines.\n\n**Questions**\n1. In the case of covariates with linear relations, we can show that the causal effect can be identified uniquely if the latent confounder is not an ancestor of any covariates in the system. We will add a remark about this in the paper and also provide proof in the appendix. The main idea in the proof is to regress dependent variables on the covariates and reduce the problem to the one considered in the paper.\n2. In our experiments (see lines 316-323), we extended our method to work with more than one proxy variable. As we mentioned above, the proposed method can be applied to linear SCMs with any number of latent confounders as long as for any latent confounder $U$, there exists a proxy variable such as $Z$, which is not an ancestor of $D$."}, {"review_id": "fEDQLCDMng", "reviewer": "Reviewer_p1qb", "summary": "This paper introduces an innovative technique for estimating the causal effect of a treatment on an outcome within linear structural causal models. This method utilizes cross moments, which are statistical moments derived from the joint distribution of the treatment and outcome variables, to quantify the causal effect. The authors demonstrate that this approach can relax the conventional assumption of a common trend in the difference-in-difference estimator, allowing for causal effect estimation in scenarios where traditional methods may fall short. To validate the effectiveness of the proposed method, the authors provide both simulation studies and a real-world application. These empirical analyses showcase the promising potential of this novel approach for estimating causal effects in linear structural causal models.", "questions": "See weakness", "limitations": "Discussed.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Novelty: The paper introduces an innovative approach to estimating causal effects in linear structural causal models with latent confounders by leveraging cross moments. This method deviates from conventional approaches and exhibits the potential to yield more precise estimates within specific contexts.\n* Rigor: The authors establish a rigorous theoretical framework for their proposed method, delineating the conditions that allow for the identification of the causal effect and the applicability of the method. Furthermore, they substantiate their claims through comprehensive simulation studies and a real-world application, bolstering the robustness and effectiveness of the approach.\n* Significance: Estimating causal effects is a crucial task across various domains, and the proposed method holds substantial importance as it can potentially deliver more accurate estimates in specific scenarios. By addressing the limitations of traditional approaches, this method offers a valuable contribution to the field of causal effect estimation.\n", "weaknesses": "The authors should compare their proposed method to more existing methods for estimating causal effects, such as negative outcome control. This will help readers understand how the proposed method compares to existing methods in terms of accuracy and efficiency.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Regarding comments in weaknesses**\n\nAs we mentioned in the global response, please note that most of these methods require at least two proxy variables and also further \"Completeness\" assumptions. Nevertheless, in the attachment of global response, we did additional experiments for comparing with more recent method in [TYC+20]. The settings of experiments in Figure 2 and Figures 3 in the attached file are exactly the same as Figure 3(b) and Figure 5(a) in the submitted paper, respectively. As can be seen, Cross-Moment algorithm outperforms both methods in [KP14] and [TYC+20] when two proxy variables exist."}, {"review_id": "Nb58yYsuTr", "reviewer": "Reviewer_5Myy", "summary": "This work focuses on estimating the causal effect in the presence of an unmeasured confounder within a linear causal model. The authors demonstrate that the desired causal effect can be identified by employing a single proxy variable, leveraging its non-Gaussian characteristics. Additionally, they propose a Cross-Moment algorithm for estimating the model. Furthermore, they demonstrate the effectiveness of the proposed method through experiments on both synthetic and real-world datasets.\n", "questions": "Identification:\n\n1.  ﻿Is the causal effect \\beta identifiable under the Non-Gaussianity assumption? This is not clear to me. ﻿In my opinion, the reason why Assumptions 2 and 3 are introduced instead of non-gaussian assumption is because one need to use the cross-moment. Am I correct?\n2. If Z directly affects D, the identification of the causal effect of D on Y may be affected. It would be helpful to investigate and discuss the potential implications of this scenario in the paper.\n3. In the current setting, the paper assumes the existence of a single unmeasured confounder U. However, it is worth exploring and addressing the situation where there are multiple unmeasured confounders U. This could enhance the comprehensiveness and applicability of the proposed method.\n \n\nRelated work: \n\nThe following paper may be related to this work and is deserved to discuss. \n\nShuai, K., Luo, S., Zhang, Y., Xie, F., & He, Y. \"Identification and Estimation of Causal Effects Using non-Gaussianity and Auxiliary Covariates.\" arXiv preprint arXiv:2304.14895 (2023).\n\n", "limitations": "Estimating the causal effect using the proposed method requires the availability of one proxy variable that is associated with the unmeasured confounder but does not directly affect the treatment variable. However, in practical scenarios, it can be challenging to identify or obtain such a proxy variable. \n", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Estimating the causal effect becomes challenging in presence of unmeasured confounders.\n\nThe proposed sufficient identification condition of this paper is novel.\n\nThe analysis in this paper is presented in a logical manner.\n\nThis paper is clearly written.\n", "weaknesses": "The model is restricted as a linear causal model.\n\nThe sufficient identification condition is applicable only to a single unmeasured confounder, and the proposed method may not be suitable for cases involving multiple latent variables.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Regarding comments in weaknesses**\n1. [Linearity assumption] Please refer to the global response.\n2. [Single latent confounder] Please refer to the global response.\n\n**Questions**\n1. As shown in Theorem 1, the causal effect is identifiable if condition in (5) is satisfied for the latent confounder. Moreover, as mentioned in lines 173-175, under some mild assumption (Assumption 3), this condition is equivalent to non-Gaussianity of latent confounder. Please note that the non-Gaussianity of latent confounder always implies the condition in (5). Moreover, in our results, we do not have any constraint on the type of distributions of the other observed variables in the system. Assumption 2 is indeed introduced so that all the cross moments are well-defined.\n2. According to [SGKZ20], with non-Gaussian exogenous noises, if there is an edge from $Z$ to $D$ with the causal coefficient $\\rho$, another model which is consistent with the same distribution over observed variables exists. The linear SCM equations for this model are as follows: $U:=\\epsilon_z$, $Z:=U+\\alpha_z\\epsilon_u$, $D:=-(\\alpha_d/\\alpha_z)U+(\\alpha_d/\\alpha_z+\\rho)Z+\\epsilon_d$, $Y:=-(\\gamma/\\alpha_z)U+(\\gamma/\\alpha_z)Z+\\beta D+\\epsilon_y$. Incidentally, the causal effect in the second model is also $\\beta$. Thus, without any prior knowledge about the causal structure, we can identify $\\beta$ uniquely using the method in [SGKZ20] (which is based on OICA) as both models have the same causal effect from $D$ to $Y$. Regarding the Cross-Moment method, it can be shown that the same Eq. (4) holds for the causal graph with the additional edge from $Z$ to $D$. As a future work, it is interesting to devise a cross-moment method to identify $\\alpha_d/\\alpha_z$ in this setting.\n3. We addressed this comment in the global response.\n4. Thanks for mentioning it. We will discuss (Shuai et al. 2023) in the related work.\n\n**Regarding comments in limitations**\n\nAs we discussed in Introduction (lines 50-60), the causal graph we consider is applicable to settings in negative outcome control in SCM framework as well as DiD in the potential outcome framework. Moreover, our experiments in real dataset showcase our approach's effectiveness in the DiD setting. "}, {"review_id": "smS8Kw6wBJ", "reviewer": "Reviewer_XfVt", "summary": "This work proposes a cross-moment approach to estimating the average causal effect with latent confounders in linear SCM. One proxy variable of the latent confounder can be observed. In contrast to prior research (e.g., difference-in-difference) that requires stringent assumptions, this work shows that the causal effect can be identified and estimated using cross moments between the treatment, the outcome, and the proxy variable. It also discusses when the effect with latent confounder cannot be identified. Experiments on both synthetic and real-world data show its effectiveness.", "questions": "1. Why SGKZ20 is very poor and what makes KP14 and the proposed approach much better than it?\n2. What are the limitations of the proposed approach?\n3. What are the potential applications of the proposed method?\n\n\n\nI acknowledge I read the authors's response and I keep my positive score.", "limitations": "No limitations are discussed. Consider shortening Related Work and add limitations in the last section.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. This work introduces simple arithmetic operations on the cross moments to estimate causal effects with latent confounders in linear SCM. It addresses a conventional challenge in the field of solving an OICA problem and biased estimation of DiD, which may result in bad local optima.\n2. It is technically sound and the idea is clearly and concisely described.\n3. It can have significance in the field and practical importance given the prevalence of the studied problem.", "weaknesses": "1. The major concern is the evaluation. The baselines included are quite weak and old, e.g., KP14 published in 2014. Why the OICA method  [SGKZ20] has very poor performance is not clear. Other baselines may include proximal causal inference [1], for example. Also, for real-world data experiments, other baselines are not included. And the differences between the two methods when x is not included are not explained. \n2. The limitations of the proposed approach are not discussed. \n3. How applicable this method is is not clear. \n\n[1] Mastouri, A., Zhu, Y., Gultchin, L., Korba, A., Silva, R., Kusner, M., ... & Muandet, K. (2021, July). Proximal causal learning with kernels: Two-stage estimation and moment restriction. In International Conference on Machine Learning (pp. 7512-7523). PMLR.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Regarding comments in weaknesses**\n\n1. [Concerns about evaluations] Regarding the method in [SGKZ20], as mentioned in lines 70-71, it is based on solving an over-complete independent component analysis (OICA) which, in practice, can get stuck in bad local minima and return wrong results. In our experiments, we observed that it is often the case and it could not find the true mixing matrix and thus, has quite poor performance. We will add this explanation to the revised paper. The work mentioned by the reviewer requires at least two proxy variables $W,Z$ which may not be available in some settings. Moreover, they often need \"Completeness\" assumptions (such as in [MGTT18,TYC+20]), which in the discrete setting is equivalent to matrices $P(U|Z,d)$ and $P(W|U)$ be invertible. Nevertheless, we compared our algorithm with recent work [TYC+20] in proximal causal inference requiring two proxy variables, and the results are given in the attached file. The settings of experiments in Figure 2 and Figure 3 in the attached file are exactly the same as Figure 3(b) and Figure 5(a) in the submitted paper, respectively. As can be seen, Cross-Moment algorithm outperforms both methods in [KP14] and [TYC+20] when two proxy variables are available.\n2. [About limitations] Please refer to the global response.\n3. [Applicability] As mentioned in Introduction (lines 50-60), the causal graph we consider is applicable to settings in negative outcome control in the SCM framework as well as DiD in the potential outcome framework.  Moreover, our experiments in real dataset showcase our approach's effectiveness in the DiD setting. \n"}]

Author rebuttal:
# Global Response
We thank the reviewers for their time and valuable feedback. In the following, we provide a global response to some of the concerns/questions raised in the review.

**About linearity assumption:** The linearity assumptions present in a large body of the research in causal discovery/inference (for instance, see the survey in [1]) as it is important to see which learning tasks are feasible in this setting. Moreover, in practical scenarios where the available data is sparse, which is often the case in fields such as social science or medical research, linear models may serve as a good starting point. The use of more sophisticated models typically requires a larger dataset to train effectively. However, in situations where such extensive data is not available, the linear model can provide valuable insights, despite its simplicity. Therefore, while linear models may not capture the full complexity of real-world systems, they continue to be valuable tools in the domain of causal discovery and inference due to their interpretability and feasibility, especially when data is limited.

[1] Shimizu, Shohei. "LiNGAM: Non-Gaussian methods for estimating causal structures." Behaviormetrika 41 (2014): 65-98.

**About multiple latent confounders:** Regarding additional latent variables, the proposed method can be applied to linear SCMs with any number of latent confounders as long as for any latent confounder $U$, there exists a proxy variable $Z$, which is not an ancestor of $D$ (under a subtle change in Eq. (4)). The main idea is to adjust the values of $Cov(D,Y)$ and $Var(D)$ in Eq. (4) using proxies for each of corresponding latent variables similar to our treatment in the submitted version. 
It is noteworthy that most previous work required at least two proxy variables $W,Z$, and even often came with some "Completeness" assumptions (such as in [MGTT18,TYC+20]), which in the discrete setting is equivalent to matrices $P(U|Z,d)$ and $P(W|U)$ be invertible.

We will add a remark about the above discussion in the revised paper. Please note that if a latent confounder does not come with any proxy variable, one can construct  SCMs in which the causal effect is not identifiable.

Meta-review: The paper gives a strong theoretical result, giving a new way for estimating causal effects in linear SCMs with proxies for latent confounders, using only a single proxy, unlike previous approaches. This is an important advance and has applications for several widely used methods such as difference-in-differences and negative controls. While the result is restricted to linear SCMs, this genuinely novel result could open the door to useful follow up work in the field.
Decision: Accept (spotlight)