Identifiable Causal Inference with Noisy Treatment and No Side Information
Authors: Antti Pöllänen
Pekka Marttinen
Venue: Accepted by TMLR ()

Abstract: In some causal inference scenarios, the treatment variable is measured inaccurately, for instance in epidemiology or econometrics. Failure to correct for the effect of this measurement error can lead to biased causal effect estimates. Previous research has not studied methods that address this issue from a causal viewpoint while allowing for complex nonlinear dependencies and without assuming access to side information. For such a scenario, this study proposes a model that assumes a continuous treatment variable that is inaccurately measured. Building on existing results for measurement error models, we prove that our model's causal effect estimates are identifiable, even without side information and knowledge of the measurement error variance. Our method relies on a deep latent variable model in which Gaussian conditionals are parameterized by neural networks, and we develop an amortized importance-weighted variational objective for training the model. Empirical results demonstrate the method's good performance with unknown measurement error. More broadly, our work extends the range of applications in which reliable causal inference can be conducted.

Published in Transactions on Machine Learning Research (09/2024)
Identifiable Causal Inference with Noisy Treatment and No Side Information
Antti Pöllänen
antti.pollanen@Baltt,
Department of Computer Science
Aolto University
Pekka Marttinen
pekka. Bakka.martiinee@@aatoooo
Department of Computer Science
Aolto University
Reviewed on OpenReview: openrevveee....ew.....me net/ net/forum? forum? id=
ESN2c
Abstract
In some causal inference scenarios, the treatment variable is measured inaccurately, for in- econometrics. Failure to correct for the effect of this measurement
stance in epidemiology error can lead to biased causal effect estimates. Previous research has not studied methods that address this issue from a causal viewpoint while allowing for complex nonlinear depen- dencies and without assuming access to side information. For such a scenario, this study proposes a model that assumess a continuous treatment variable that is inaccurately mea- sured. Building on existing results for measurement error models, we prove that our model's causal effect estimates are identifiable, even without side information and knowledge of the measurement
variance. Our method relies doep latent variable model in which Gaussian conditionals are parameterized by neural networks, and develop amortized importance-weighhe variational objective for training the model. Empirical results demon- strate the method's good performance with unknown measurement error. More broadly, our work extends the range of applications in which reliable causal inference can be conducted.
1 Introduction
Causal inference deals with how a treatmett variable X causally affects an outcome Y. This is different from just estimating statistical dependencies from observational data, because even when we know that causes Y and not the other way around, the statistical relationship could be affected by confounders Z, i.., common causes of X and Y. Knowing the causal relationships is crucial in fields that seek to make
interventions, e-g., medicine economics [Pearl 2009 Peters et. al. 2017 Imbens & Rubin 2015)
Causal inference may be complicated by variables being subject to noise, e.g., inaccurate measurement, clerical error, or self-reporting (often called misclassification for categorical variables). not accounted for, well-known that this error can bias statistical and causal estimates in fairly arbitrary manner, and consequently, measurement error models have boen widely studied address this bias (Carroll et al. 2006: Buonaccorsi 2010 Schennach 2016 2020).
In this paper, assume the treatment X and outcome Y to be continuous, while the confounders Z can be catogorical, discrete or continuous. (The case of binary or categorical X is briefly discussed in Appendix A While measurement error may occur in any of the variables, we explicitly model it only in X. We implicitly also include measurement error Y, but in our model is indistinguishable from the inherent noise in the true value of Y. The practical impact of this is limited, since the mcasurement error for does not bias regression due to its zero-mean additive nature that we assume. On the other hand, measurement error in Z could be relevant (Miles et al. 2018 but it is restricted outside the scope of this work.

---

Published in Transactions on Machine Learning Research (09/2024)
Y
X
Z
X
Figure 1: Causal graph for our proposed model. The observed variables X (noisy treatment), Y (ef- /ott/oue) and Z (confounders) are shaded to distinguish them from the hidden variable X* (true treatment).
The causal graph expressing our assumptions is depicted in Figure [ An example reflecting this scenario in the context of healthcare is the effect of blood pressure on some continuous health outcome Y, such as arterial stiffness as measured by pulse wave velocity, or kidney function as measured by glomerular filtration rate (GFR) or by the concentration of albumin in urinc. We assume X is a single measurement of blood pressure, and thus it is highly subject to instantaneous variation, which plays the role of the measurement error. Some additional error probably also results from measurement apparatus inaccuracy. Here, the blood pressure that actually matters in terms of health outcomes (and whose effects we are interested in) is longer term sliding window average, which is unobserved and which we denote by the latent variable X*
To enable statistical identifiabiliyy, we assume the measurement error to be additive, zero-mean and indepen- dent from the other variables. These are standard set of assumptions in the measurement error literature, called strongly classical measurement error. Non-classical measurement error is also widely studied and well documented to occur in practice (see [Schennach (2016) for review), but it is restricted outside the scope of this work. However, we believe modeling and inference methodology is general enough that extensions to these cases are fairly straightforward. Figure 2 presents a synthetic example of the skewing effect of measurement. error in the treatment. While it is often believed that this type of measurement. error only introduces attemuating bias to a naive estimate not accounting for the error (Yi et al., 2021), this example demonstrates that on the contrary, amplification is also possible. Here attenuation happens only around X* 0, but for small and large X the naive estimate amplifies the strength of the dependency.
The measurement error problem is often addressed by making additional measurements to enable model identification. The information obtained in this way could be known measurement error variance, so- called side information, such as repeated measurements, instrumental variables, or a gold-standard sample of accurate measurements (Yi et al. 2021). However, we study the scenario where none of this is available, and we must rely on assumptions that could reasonably be made a priori, such as the error being additive and having a zero mean. Further, we study the scenario where the dependencies between the variables can be complex and nonlinear, and an observed confounder is present. There is a gap in the literature for causal inference when all of these challenging aspects are present.
To address this gap, our paper provides solution by inferring structural causal model (SCM) from ob- servational data using deep latent variable modeling and importance weighted variational inference (Zhang let al. 2019). The inferred SCM enables the computation of any interventional and counterfactual distri- butions on the variables in the model, but to evaluate the fit, we consider the accuracy of the estimation of the following quantities: 1) the function by (2,xx) and 3) the function p(y)do(=")), which maps x² to the density of Y |doox"). The combination of 1) and 2) also evaluates the estimation accuracy of p(y) do(=)) since with Z satisfying the backdoor criterion and These
E|||,, do(x")), 2) the noise variances 72 and o²,
by assuming conditionally Gaussian Y, we have p(y12, do(=,,,(("))))) paramctons/distributuen distributions were chosen because they are either model parameters, reflect probable use cases of the model, or both. To facilitate practical compulation, we assume conditionally Gaussian variables. However, we emphasize that the purpose of this is to simplify computations, and the identification of the model docs not rely on this assumption, see Section [.3.T for the details. We leave up to future work to relax it by using flexible probability density estimators such as normalizing flows. We evaluate our algorithm on wide variety of senthetic datasets, as well semi-synthetic data.
p(ylz,z*) ((,y)uy
2

---

Published in Transactions on Machine Learning Research (09/2024)
1.2
accurate
our method
1.0
naive estimate
ground truth
0.8
0.6
0.4
0.2
0.0
-0.2
-10
5
10
X
Figure 2: Comparison of our method (CEME) against naive method which does not account for measure- ment error in the treatment X The accurate values of X are hidden from both methods. Ground truth is the true mean function of the data generating process. The same data are displayed both with and without measurement error in X* It can be seen that our method "CEME" fits the error-free data (even if they are not seen by any method) whereas the "naive" method fits the data with error and cannot estimate the true regression function accurately. A similar example presented by Zhu al. (022).
There is previous literature on all of the challenges tackled in this work, although not previously studied all together. For an overview the literature on the bias introduced by measurement error causal estimation, see Yi et al. (2021). This literature includes qualitative analysis by encoding assumptions of the error mechanism into causal graph (Hernán & Robins 2021), and quantitative analysis of noise the treatment (Zhu et al. 2022), outcome (Shu & YiJ, 2019 confounders (Pearl 2012 Miles et al. 2018) and mediators (Valeri & Vanderweele) 2014]. However, most of this literature focuses on simple parametric models, such as the linear model, and the identifiability of the causal effect is achieved through a fully known error mechanism or side information, such as repeated measurements, instrumental variables, or a gold standard sample of accurate measurements. Exceptions include Miles et al. (2018) which studies confounder measurement error with no side information and a semiparametric model, Hu et al. (2022) which similarly to us uses deep latent variable models and variational inference, but assumes known measurement error variance, and ZZuu al. (2022) which studies treatment measurement error with nonparametric model and notably even allowing for unobscrved confounding, but utilizes both repeated measurements and instrumental variable as side information. Schennach & Hu (2013) use non-parametric sieve estimation with no side information, without considering covariates 7 that could increase complexity significantly. We take them into account not only to increase the accuracy of predictions, but crucially to allow adjusting for confounders to obtain causal effects. Concurrently to us, Gao et al. (2024) study a similar model to ours and also use variational inference. However, they do not include a theoretical analysis of identifiability as we do. Soe Appendix for additional references identifiability results on models related ours.
To summarize, this work contributes to the literature by successfully inferring causal effects in setting with novel combination of challenging aspects: measurement error, no side information, no knowledge of the measurement error variance, and complex nonlincar dependencies including dependency on confounders that must be adjusted for in order to estimate the causal eflect. To show the identifiability of our model, we extend a previous result by Schennach & Hu 2013) to allow for feffoonndrs/covaritttttett and adjust to the specific assumptions our model. Finally, this work contributes by bridging different disciplines: it highlights that measurement error models constitute SCMs and uses machine learning, specifically variational inference, foundation for inferring causal effects.
3

---

Published in Transactions on Machine Learning Research (09/2024)
2 Methods
2.1 Models for causal estimation
For causal inference, this study uses the structural causal model (SCM) framework (Paall 2009 SCM consists of the following components: 1) list of endogenous random variables = (V1, V2, ... Vy). 2) A directed acyclic graph G E) called the causal graph, whose vertices are the variables V and whose edges E express causal effects between variables. 3) A list of exogenous random variables that represent external noise and are mutually independent. 4) A list of deter- ministic functions F = f2........ fN): called the causal mechanisms, such that for i = 1....., N we have (/==((AA Ui), where РA; are the random variables that are the parents of V; in G. This construct allows not only the modeling of observational data but also the effects of interventions. An intervention on variable V4 in an SCM is defined as a replacement of the mechanism fi with another mechanism fi. The simplest case is hard intervention, denoted by do(V; = vt), where ft simply assigns a constant value 2 to V.. When estimating any causal quantities from data, we assume the data to be i.i.d. according to the SCM. An intervention applied on one individual unit (each corresponding to data point) does not affect others. The properties of consistency and not having multiple versions of an intervention (Heráán & Robins 2021 follow trivially from the definition of an SCM.
U -U,,U2,.... ......,,N))
Note that the causal graph is also a Bayesian network that implies conditional independencies according to d-separation Z, the accurate treatment value
To model measurement error, we use an SCM with the causal graph G depicted in Figure I
(Bihhop 2006). The variables in the graph are the mnnfounders/covariieen X "hicc unobserved, the noisy measurement of the treatment, denoted by X, and the noisy value of the outcome, denoted by Y. Except for X*, the other variables are observed. The SCM with independent noise terms and no hidden confounders implies the common assumption of causal sufficiency. Therefore, when the SCM is identified, any interventional or counterfactual distributions can be computed. These include e.g. Yg+, which denotes the distribution of Y after intervening with do(X' = x²) (alternatively denoted by Yldo(X" x")), or the distribution of Yg; - Y=z3 which compares the effects of two treatment values æi and 22 or the distribution of YY. = y, Z 2, which denotes the outcome that would have been obtained from intervention do(X* = =x) when in reality no intervention was made and the values Y y and Z were observed.
This paper combines themes of statistical and causal cstimation, which use related but different definitions of identifiability. However, we use everywhere the same unified definition:
Definition 1 (Tdentifiability of estimand f). Let (BBL be family of joint probabilily distributions of the observed variables B and latent variables L, parameterized by 9. Denote by Pt the corresponding marginal distribulion for B. Then an eslimand 5(0) (where ʃ is a deterministic function) is identifiable if for every 0, O' E θ we have
Pg Pg' =
= 100)=1(0')
Here f represents anything one is interested in estimating from data (including but not limited to causal parameters), and as a special case when f(0)= = 0, we get the definition of model identifiability. Intuitively, these mean that knowing the distribution of the observed variables, for example by having estimated it with an infinite amount of i.i.d. samples, the estimand f() (or simply 8) can be uniquely determined. In causal inference literature based on SCMs, identifiability often means that an interventional distribution can be computed from infinite observational data and the causal graph structure (Peters et al. (017). In contrast, with our definition this computation can additionally use any assumptions made about the underlying SCM.
Because X is hidden, the SCM or any causal queries that consider the effects of an intervention do(X' x²) are not identifiable without further assumptions, as the relationship between X* and the other variables can not be est imated. Therefore, for identification, we rely on additional assumptions on the measurement error mechanism X = fx(X* Ux) such that the SCM becomes statistically identifiable. In the literature, much attention has been devoted to making the model identifiable using known error variance or side information. However, we instead rely on the assumptions of independence and zero-mean additive errors (Schennach & Hu 2013) First, we assume that the measurement error is strongly classical, i.e., the observed treatment. X
4

---

Published in Transactions on Machine Learning Research (09/2024)
is generated by causal mechanism
X=X+AX,
(1)
where the measurement error AX, which acts as the exogenous variable Ux, is independent of all other variables except X and has a zero mean. Similarly, assume for the outcome Y that
Y by (2, AY;
(2)
where the noise AY is the exogenous variable Uy independent of all other variables except Y and has zero mean. The conditional mean of Y, py(2,z*) E| = 2, x²), is deterministic, possibly nonlinear function that is continuously differentiable everywhere with respect to x². (Note that by uppercase letters we denote random variables and by lowercase letters their specific realized values. Thus ny(2,x)) refers to the value of py with argument values and x², while jy (Z) denotes the random variable that is obtained by applying to the random variables Z and X)) Although not necessary for identifiability (see Section 2.3.1 for details), we further assume Gaussian distributions to facilitate practical computations, yielding
1 ~ N(ux. 0Z), (2))
(3)
AX N(0,7)),
(4)
AY N(O,
We assume that X, Y, and X are real-valued scalars, although multivariate extensions are relatively straight- forward. The covariate Z can be combination of categorical, discrete, continuous variables, but the other variables are assumed to be continuous. Consequently, the learnable parameters the variance parameters 72 and o² as well as the (deterministic) functions μх*, xx- and μy, which are all parameterized by neural networks to allow flexible dependencies between the variables. The zero-mean assumptions in Equations (4) and 5 are made to achieve ical identifiability. Otherwise, for any non-zero mean JAY, we could obtain a new observationally equivalent model by using (2, = py (z,=*) - May. Similarly, for any non-zero mean JAN, we could obtain a new observationally equivalent model by using x'x((2) = px- (z) - Pax and ply = py (2, + Max).
Finally, define the following model variants that we evaluate against each other:
CEME:
[ Causal Effect estimation with Measurement Error. This is the model defined in Equations
The mean and standard deviation functions px- (2), (2) and ny (2, are fully connected neural networks.
CEME+: Causal Effect. estimation with Measurement Error with known noise. This is hewwewise the same as CEME, except that the variance of the measurement error 72 is assumed to be known.
Oracle: This method assumes the model in Figure 3a where the true treatment X* is observed and is used directly for fitting the model. Hence, this model provides loose upper bound on the performance that can be achieved by the models CEME and CEME which only observe the noisy treatment X. The model consists of only one neural network, parametrizing Jy (Z,
This is the same as Oracle, except that it naively uses the observed noisy treatment instead of the true treatment X when estimating the causal effect (model depicted in Figure 3b). Hence, it provides baseline that corresponds to the usual approach of neglecting the measurement error causal estimation. The model consists of only one neural network, parametrizing Hy X..
Naive:
To summarize the roles of the methods, CEME is our proposed method, CEME+ enables comparison with the case of a known measurement error variance 72, Oracle is a loose upper bound for performance, and Naive is a baseline. For inference, CEME and CEME+ use amortized variational inference to estimate the latent variable (described in the next section). In contrast, Oracle and Naive, which do not have latent variables, are trained using gradient descent with mean squared error (MSE) loss for predicting Y.
2.2 Inference
Amortized variational inference (AVI) ZZhngg et al. 2019 can be used to estimate the parameters of deep latent variable models (LVMs). The method includes one or more latent variables (true treatment X in our
5

---

Published in Transactions on Machine Learning Research (09/2024)
Y
2
(a) Causal graph assumed by Oracle, which has ac- cess to the true treatment X", unlike the other meth- ods.
(b) Causal graph (incorrectly) assumed by the method Naïve.
model), observed variables whose distributions are modeled (X and Y in our case), and optionally observed variables whose distributions are not modeled but on which the model is conditioned (the covariates Z in this paper). We use this method to infer the SCM, after which causal effect estimates of interest can be calculated from the model parameters by Monte Carlo sampling over observed covariates Z.
For the distribution p(X*, YIZ), the method requires dcfining generative model (also called decoder) pa(x*. 2, v12), where 9 denotes the parameters. In this paper, use the CEME model defined in Equations 1-6 5 with 0 consisting of the standard deviations T and as well as the functions _x (=), a (z) and My (2,x*) x"),,, which are modeled as fully connected neural networks. (TThu in terms of the practical computa- tions, the parameters are not the functions itself but their respective neural network weights.)
AVT also requires modeling the posterior (XX*),,XXY) with a so-called encoder, for which we use go(x*12,,,,) v) N (M, 15 o,),), y; where denotes the encoder parameters, which are the functions 22, y) o) and (, yi d), or alternatively, the weights of the two fully connected neural net- works that model these functions.
The parameters 0 and á are optimized with stochastic gradient descent using the negative evidence lower bound (ELBO, defined in the next section) as the loss function. This approach can be used for LVMs that generalize the variational autoencoder (VAE) to more than just one hidden and one observed node (Kingma & Welling 2019) (CEME/CEME* uses the Bayesian network Figure
2.2.1 Model training
As our objective function, we use the importance weighted ELBO Ls bound for the conditional log- likellioihood (= u/z):
(Burda et al. 2016), which is a lower
N
log pa (x, y(z) =
Σ
log pa (a,, vilzi)
(6)
1
K
pa Fze 21
log (...)
(...)
K
2 ( Izcs 2it m/i
(7)
j=1
1
po : 8/ U11
2
log
(8)
K
go (2 1zs Fis Ms)
(9)
where the expectations are with respect to qo( [z4, Ni, W)) For each data point i, there are K i.i.d. realizations of the latent variable z*, indexed by j. The inequality in (8) is obtained by applying Jensen's 's inequality. The standard ELBO corresponds to K 1. For practical optimization, the expectation in Equation is estimated by sampling the K realizations of the latent variable x once per data point i.
We use the importance weighted ELBO instead of the standard ELBO because the latter places a heavy penalty on postcrior samples not explained by the docoder, which forces bias on the dccoder com- pensate for a misspecified encoder (Kingma & Welling, (019). Increasing the number of importance sam- ples alleviates this effect, and in the limit of infinite samples, decouples the optimization of the decoder

---

Published in Transactions on Machine Learning Research (09/2024)
from that of the encoder. To calculate the gradient of the ELBO, it is standard to use the reparame- terization trick, meaning that we sample o Ti, y/ ~ ( [z, Iz, vi) as I = (2, di, Uei u) + Pg (g Mii o), where e follows the standard normal distribution. The
ayportance-weightee ELBO ob- jective is optimized using Adam gradient descent. Further training details are available in Appendix B The algorithm was implemented PyTorch, with code available for replicating the experiments at https://github.com pontttipplllaaen. ci ctrrcci_nooo wtweooooiitttttaaaeee treatment
2.3
Identifiability analysis
2.3.1
Identifiability of causal estimation with a noisy treatment
The identifiability proof of the CEME model builds closely on an earlier result by Schhnnach & Hu (2013) of which we provide an overview here. It is an identifiability result on a slightly different mcasurement error model, which differs from ours in that it does not include the covariate Z, but on the other hand it is more general in that it does not assume that X"; AX and AY are (conditionally) Gaussian. The exact form of The model includes scalar real-valued random variables Y, X", Xx,,,AA X
the result is included in Appendix C and AY related via
Y Y= -==(XX)= + AY and X
(10)
where only X and Y are observed. We assume that 1) X", AX and AY are mutually independent, 2) B[AX] = [|A]] = 0, and 3) some fairly mild regularity conditions. It is shown by Schennach & Hu (2013)
that this model is identifiable if the function g Equation (10) is not of the form
g(~") + b bln(e==*++
(11)
and even if it is, nonidentifiability requires x to have a specific distribution, e.g. a Gaussian when g is linear. From this result, we see that the CEME model assumptions of conditionally Gaussian variables (made to simplify computation) do not help with identification, as it brings the model towards the non-identifiable special cases. With these preliminaries, we are now ready to prove the following proposition that establishes the identifiability of the CEME model:
Proposition 1. The measurement error model defined in Equations 1- is model identifiable if 1) for every 2, Uy(2, is continuously differentiable everywhere as function of x", 2) for every 2, the set 8
X = fx" D (x, = 0} has at most a finile numbber of elements, and 3) there exists z for which (2(22 is not linear in x² (i.e. of the form by (x*))= =ox" b).
Proof. The full proof is provided in Appendix D but its outline is the following: First, we show that a restricted version of our model where can only take one value is identifiable as long as py x²) is not linear in x", This follows from the identifiability theorem by Schennach & Hu (2013) because its assumptions are satisfied by the restricted version of our model, together with the assumptions of Proposition D
Second, we show the identifiability of our full model by looking separately at the values of = for which x² is or is not lincar in x². For the model conditioned on the latter type of 2 (nonlincar cases), identifiability was shown in the first part of the proof. For the remaining linear cases, we use the fact that we already identified I and a with the nonlinear cases, which results in a Linear-Gaussin model with known errors, which is known to be identifiable (sce o.g. Equations (4.9-4.17) in Gustafson (2003)). Thus the entire model is identified.
We note that assumption 1) in Proposition is satisfied by the neural networks used in this work, as they are contimuuusy differentiable due to them using the extended linear unit (ELU) activation function. Assumption 2) is true in general and does not hold only in the special case where the derivative of the neural network by (2, x)) (2," is constant on an interval. Breaking assumption 3) would require the network (2,x) to be linear in x² for every z. Even when this is the case, we still hypothesize that the model is identifiable as long as there are multiple values of the lincar slope for different z. This is suggested by solving a system of equations similar to Gustafson (2003), which appears to have one or at most two distinct solutions. On another hand, the proof of Proposition shows that if there was z for which py (x, was linear in œ, we would not need
is zero with respect to x² in infinitely many points, which is possible only if the function
7

---

Published in Transactions on Machine Learning Research (09/2024)
0.75
15
32
2.7
10
24
2
0-
0.75
0.9
8. -2
1,50
-0.
-4
1.5
-6
0.9
33,00
24
-10
3.
-8
3375
4.3
4
Figure 4: Three realizations of synthetic datasets generated from Gaussian process with 3000 data points (black dots). The figures show the covariate Z (x-axis), the noiscless versions of the treatment X (y-axis), and the outcome E[Y|Z, X*] (healmap).
assume any restrictions on how т and depend on Z, and we surmise that even when there are such 2, milder restrictions than constant and would be sufficient. However, we leave detailed investigation of these cases for future work.
3 Experiments and results
We conducted two experiments: One with datasets drawn from Gaussian processes, where large number of datasets used establish the robustness of results. Second, the relationship between years of education and wages, to study the effect of complex real-world patterns for which the assumptions of our model do not hold exactly.
augmented real-world dataset
3.1 Synthetic experiment
3.1.1 Synthetic datasets from Gaussian processes
For the synthetic experiment, we generate datasets whose underlying distribution follows the CEME model with all variables being scalars. Gaussian processes (GPs) are used, see Rasmussen & Williams 2006) for an introduction. In brief, a random function f() for & R Gaussian process if f(t1)..... f(t)) is jointly Gaussian for any k E N and any (1,...,4 4g R" They are denoted by GP K) and characterized by mean function p(44) and kernel function K(4,,y)) by which one obtains the mean vector and covariance matrix for the jointly Gaussian f(t1), ..., ftx) (for any k E N and any ti...,, R²). In this study, the mean functions p(t) are constant. The data generation proceeds according to Algorithm For the GPs, we use the squared
A Algorithm Generation of synthetic datasets using GPs
For i [1,.., N), sample 2, from N(0, 1).
Sample μх* from GP K) and g-1 o ax- from GP(1, K). Here denotes function composition and g(=) log(1 exp(2)) is used to ensure the positivity of ox-
For i E [1,.. NJ, sample xu from (x),
4: Sample
from (),
5: Set r = VV J, where Var denotes unbiased sample variance and L is a constant.
Set
L. (tVVa(((xx((i (y......((((a) (zN,
7: For i E [1,.., sample from N(n,, +2)).
8: For i E NJ,, sample from N(ay (2,, 2;),
exponential kernel K K(u,, u2) a exep
[w "u--wal with = and lengthscale / and where | denotes 21
the Euclidean norm. Note that for px- (z) and o (2) the corresponding GPs are over while for jy (2, x²) the GP is over TR2 To avoid excessive computational cost, the functions px- (z), ax- (z) and Jy (y (2, x²) are

---

Published in Transactions on Machine Learning Research (09/2024)
only approximations of true samples from the GPs (steps and 4 in Algorithm 1. described in Appendix B along with other experiment details. Three datasets generated in this manner are shown in Figure 4
The constant L is varied to obtain datasets with different levels of noise, which are either L = 0.1 (small noise), L = 0.2 (medium noise), or = 0.4 (large noise). The different training dataset sizes used are 1000, 4000, and 16000 data points. The test data (used for evaluating the models) consist of 20000 data points. For each combination of noise level and training dataset size, we generate 200 different datasets (with different underlying distributions) using Algorithm For all of these, fit each model (CEME, CEME* Naive and Oracle) 6 times, and for the results pick the run with the best from the test dataset). All models use the same neural network architecture to facilitate fair comparison, Lee. three fully connected hidden layers with 20 nodes each and the ELU activation function.
separate validation dataset (distinct
Error Fly|2,do(s )] estimation
CEME
CEME
Orocie
Naive
noise level: 10%
noise level: 20%
noise level: 40%
0.25
0.25
0.25
0.20
0.20
0.20
0.15
0.15
0.15
1MSE
MMSE
0.05
0.05
0.05
0.00
0.00
0.00
1000
4000
16000
1000
4000
16000
1000
4000
16000
training
Figure Error in Efyl do(x=)) estimation in the synthetic experiment. In addition to data for Naive, missing from the figure are some outliers for CEME and CEME+ for noise level 40% and training dataset size 1000. Key observations are that 1) the proposed CEME/CEME+ methods ofler clear benefit over Naive that does not account for measurement error, 2) CEME and CEME seem to converge with increasing training set size and compare relatively well with the loose upper bound Oracle, and 3) CEME handles unknown measurement. error variance well, since CEME+ that knows it, performs only slightly better.
3.1.2 Results
The accuracy of the estimation of py (z,xx)) |[ do(x")) is reported in Figure using the root mean squared error
MSS
Σ (uv,o(z(,))) Jy (2(,,)))²
where 8 denotes the estimate and the sum is over the N 20000 data points in the test set. The accuracy of the estimation of is reported in Figure 0 which uses the metric relative error (ag - a)/a, where dg is the estimate and the true value. The relative error in the estimation of the measurement error noise is reported in Figure in the same way. The accuracy in the estimation of p(y do(x")) is presented in Figure
and assessed using Average Interventional Distance (AID) Rissanen & Marttinen 2021
AID (p(ee) / po(e)ldo(=")) p(yldo(x")) dydz*
(12)
Here, both the estimated (o(y)do(z")) and the ground truth p(y)do(=*)) are obtained using the adjustment formula p(y)do(=")) "uyl=" (2)p(2ddz Z is assumed to satisfy the backdoor criterion (Pearl 2009

---

Published in Transactions on Machine Learning Research (09/2024)
Estimation of AY standard deviation
CEME
CEME
Oracle
Naive
ground Iruth
level:
nosse
100%
100%
100%
60%
60%
60%
relative
relalive
40%
40%
40%
20%
20%
20%
0%
0%
0%
1000
4000
16000
1000
4030
16000
1000
4000
16000
training size
training size
training set size
Figure 6: Error in estimation of AY standard deviation in the synthetic experiment. In addition to data for Naive, missing from the figure are 4 outliers for CEME, 1000 training data and 10% noise as well as 2 outliers for CEME, 4000 training data and 10% noise. Key observations are that 1) the estimates seem to converge for the proposed CEME/CEME* but not for the baseline Naive, and 2) all the methods tend to overestimate, rather than underestimate, the standard deviation of AY.
All integrals estimated using Monte Carlo integration, using test data sample for Z and X and sampling Y uniformly, with the exception that for the ground truth p(yldo(z")) we integrate over using the trapezoidal rule.
In these figures, the data are represented as boxplots, where the box corresponds to the interquantile range, and the median represented as horizontal bar. The whiskers extend to the furthest point within their maximum length, which is 1.5 times the interquantile range. Outliers beyond the whiskers are represented singular points. Fach data point. corresponds separate data-generating distribution and the best of six runs based validation loss.
Overall the CEME algorithms offer significant benefit over not taking measurement error into account at all (algorithm Naive), both terms of estimating individual causal effect (Figures and G and average causal effect (Figure 7 In addition, the results suggest convergence estimates when increasing dataset size, as expected from the thcoretical identifiability analysis. On the other hand, even with knowledge of the true standard deviation (SD) of AX, CEME+ does not achieve performance comparable to that of Oracle. This is be expected as Oracle accurate values of X for individual data points, which cannot be identified even in principle from information available to CEME and CEME
Interestingly, the performance improvement from knowing the true SD of AX (with CEME*) over learning it (with CEME) seems relatively modest. This suggests that identifiability CEME, even though the data distributions could be arbitrarily close to the non-identifiable linear-Gaussian case. We also note that all algorithms almost always overestimate the SD of AY, which could be explained imperfect regression fit. Furthermore, detailed in Section 2.2.1 the training of CEME and C is initialized such that they try to predict X close to X, which could be another cause for this and also explain why CEME tends to underestimate the SD of AX (Figure
generally not an issue for
3.2 Experiment with education-wage wage data
We also test CEME with semisynthetic data based on a dataset curated by Card (1995) from data from the National Longitudinal Survey of Young Men (NLSYM), conducted bet years 1966 and 1981. The items
10

---

Published in Transactions on Machine Learning Research (09/2024)
in ply]do(x") estimation
CENE
CEME
Oracle
Neive
noise level: 10%
noise level: 20%
noise level: 40%
0.40
0.40
0,40
0.35
0.35
0.35
0.30
AID estimate
AID eslimute
0.20
0.20
0.20
0.15
0.15
0.15
0.10
0.10
0.10
0:05
0.05
0.05
0.00
0.30
1000
16000
training set size
training set size
training set SiZe
Figure 7: Error in estimation p(y]do(x")) in the syuthetic experiment. Besides data for Naive, missing from the figure are outliers for CEME and for CEME* both for 40% noise and 1000 training data. This figure tells a similar story Figure except that the differences between the methods are more pronounced.
in the dataset correspond to persons whose number of education years is used treatment X As outcome Y, we use the logarithm of the wage. The relationship between these is also studied in the original paper (Card 1995 The dataset also contains multiple covariates, of which this experiment uses a total of 23.
To evaluate our methods, we need to know the ground truth values of the parameters that we es- timate. Having access to real observed data alone does not permit this, which is why augment the real data with synthetic variables that mimie the real ones. The known data generating processes of these synthetic variables enable us to compute the ground truth. To this end, we first train a neural network to predict the outcome, and then modify the dataset by replacing the outcome values with the neural network predictions to which Gaussian noise is added. The noisy treatment. X is obtained by adding Gaussian noise to X Six separate datasets are created, each corresponding to a different level of SD of AX. The levels are proportional to the SD of X", and are 0%, 20%, 40,, 60%, 80% and 100%. The choice of education years as treatment was in part motivated by the need to know the true treat- ment accurately to obtain the ground truth causal effect. The CEME/CEME* models are misspeci- fied for this dataset because the true treatment, the number education years, is ordinal, instead con- ditionally Gaussian, as the models assume. This presents an opportunity evaluate the sensitivity of the CEME/CEME algorithms to model misspocifi- cation.
Estimation of AX standard deviation
training set size
ground truth
1000
4000
20%
16000
0%
relative
200%
-40%
10%
20%
40%
noise level
Figure 8: Error in estimation of AX standard devia- lion by CEME in the synthetic experiment. The es- timates seem to converge with increasing training set size.
11

---

Published in Transactions on Machine Learning Research (09/2024)
Elyiz, dooo" estimation
Enror in plyjdotx" DD esstime zion
CEME CEME
0.14
0.15
Oracie
No/ve
Nalve
0.14
AD eztiinnaa
0.32
0.08
0.10
0.06
0.08
0%
20%
40%
EORS
80%
100%
0%
20%
40%
50% rel2tive SD af AX
80%
200%
relctive SD of AX
(a) Error in B 2,, =')] x')]
estimation.
(b) Error in p(y)do(x")) estimation.
Figure 9: Error in causal effect estimation in the education-wage data experiment. The proposed CEME/CEMPE methods offer improvement compared to the baseline Naive, but not as much as in the synthetic experiment. A likely reason for this is that CEME/CEME /CEME* incorrectly assume a continuous X", unlike Oracle and Naive. Morcover, the difference between CEME and CEME+ is larger than in the synthetic case.
We run this experiment for the same four algorithms as in the synthetic experiment, defined in Sec- tion 2.1, with three hidden layers of width 26 and the ELU activation function. The training proce- dure and model structures are the same as in Section 3.1 except that now multivariate and hy- perparameters are different. Details on the experiment and how the semisynthetic datasets were cre- ated are available in Appendix B The full data of 2990 points is split into 72% of training data, 8% of validation data (used for learning rate annealing and early stopping) and 20% of test data (used for evaluating the models), all amounts rounded to the ncarest integer. The code used for data pre- processing and running the experiment is available at attpp://gitaen Miggtthu..be. wttp:/ctio///iii/eeeeeee www.untippollaeenneeeee caceanccinnniiiio treatment
3.2.1 Results
The main results of the experiment with education-wage data are presented Figures 9a They use the same metrics as the corresponding results for the synthetic data experiment. The points represent median values and the bands represent interquantile ranges. Each data point corresponds to one training run of the model, and the x-axis in each figure indicates which dataset was used (they differ only in the SD of the augmented noise AX). There is no entry for CEME+ for the 0% AX SD dataset, because using variational inference for model with no hidden variables is not useful and is problematic in practice because terms in the ELBO become infinite.
9b
10a and
10b]
We notice that while CEME and CEME+ still offer a clear improvement over not accounting for measurement all (Naive), they seem to be further below in performance from Oracle (which acts as benchmark for optimal, or even beyond-optimal performance for the CEME/CEME" algorithms). A potential reason for this is that the CEME models, which model the true number of education years variable, are misspecified unlike Oracle and Naive, which only condition on the number of education years but do not model it (though Naive assumees that there is no measurement error). Moreover, CEME and CEME have more parameters than Oracle and Naive; therefore they could be hurt more by the limited dataset size and high-dimensional covariate. The larger difference between CEME and CEME+ than in the synthetic experiment might be because CEME avoids some of the detriment from model misspecification by having access to the true standard deviation of AX.
continuous latent
12

---

Published in Transactions on Machine Learning Research (09/2024)
Edtimation wandard deviation
Estimation standard deviation
10
CRM
3.20
ground truth
GEME
Shorle
ground trut'
50 cstisto
SPSU estmatee
2.14
0.12
0.2
2.30
0.0
0%
20%
60%
00% 80%
100%
0%
20%
40%
50%
80%
100%
relative SD of AX
retatvv SD
(a) Estination of AX standard deviation.
(b) Estimation of AY standard deviation.
Figure 10: Error in the estimation of treatment and outcome noises in the education-wer wage data experiment. On the left, only CEME is depicted because it is the only method that estimates AX standard deviation. These figures help demonstrate that the proposed CEME/CEME offer clear benefit over the baseline Naive even when they are misspecified.
4 Conclusion
In this study, we provided a model for causal effect estiimation in the presence of treatment noise, with complex nonlinear dependencies, and with no side information or knowledge of the mcasurement error variance. We confirmed the model's identifiability theoretically and evaluated it experimentally, comparing it to a baseline that does not account for measurement error all (Naive) and to an upper bound in performance in the form of Oracle. A notable advantage of the model is its flexibility: It offered good performance diverse set of synthetic datascts and useful for correcting for measurement error even real-world dataset for which it was clearly misspecified. Our approach is also flexible in second way: after using amortized variational inference to infer the SCM and the posterior of hidden variables, any interventional or counterfactual distributions may be computed.
A limitation of the CЕME/ CCEE++ methods their assumption of independent additive noise for both treatment and outcome, which might not always hold [Schennach] (2016). On the other hand, these assump- tions were critical for attaining identifiability without side information; therefore relaxing them would likely mean having to introduce side information to the model/Schennade (2020). Another limitation of our study is the lack of comparison with other state-of-the-art methods. However, the authors are not aware of any that exist which estimate the same quantities as our method and are designed for the setting where the treatment is noisy, the regression function does not have any strict parametric form, the model conditioned on a covariate, and no side information is available. In addition, our study does not consider the robust of estimation when there no access to a perfect set of covariates satisfying the backdoor criterion.
Finally, the model is limited by assuming Gaussian distributions in all conditionals. An interesting direction for future research could be to relax this assumption by using flexible distributions, such normalizing flows. The identifiability result in Schennach & Hu (2013) suggests model gencralized in this way would retain its identifiabillty. In addition, since it is straightforward to generalize the amorlized variational approach for other related latent variable models, an interesting future direction would be to apply the algorithm models that relax the classical measurement error assumptions Schennach (2016) 2020), for which efficient inference methods are not yet available.
13

---

Published in Transactions on Machine Learning Research (09/2024)
Broader Impact Statement
Our method could used for estimating treatment effects in applications where incorrect inferences might in the worst case lead to severe adverse outcomes, egg. heare For this reason it important consider the validity of the assumptions of our method in any particular use case as well as test any implementations carcfully before practical deployment.
Author Contributions
All authors have accepted responsibility for the entire content of this manuscript and approved to its sub- mission. AP had the main responsibility on all aspects of the work. PM supervised the work and provided the initial research idea. The authors wrote the paper together.
Acknowledgments
This work was supported by the Academy of Finland (Flagship programme: Finnish Center for Artificial Intelligence FCAI, and grants 336033, 352986, 358246) and EU (H2020 grant 101016775 and NextGenera- tionEU).
References
Dan Ben-Moshe, Xavier D D'Haultfouille, and Arthur Lewbel. Identification of additive and polynomial models of mismeasured regressors without instruments. Journal of Econometrics, 200(2):207-22)) 2017.
Christopher M Bishop. Pattern recognition and machine learning. Springer, 2006.
John P Buonaccorsi. Measurement error: models, methods, and applications. Chapman and Hall /RR, 2010.
Yuri Burda, Roger B. Grosse, and Ruslan Salakhutdinov. Importance weighted autoencoders. In Yoshua Ben- gio and Yann LeCun (eds.), Proceedings of the 4th. International Conference on Learning Representations (conference track), 2016.
David Card. Using geographic variation in college proximity to estimate the return to schooling. In Louis N Christofides, E. Kenneth Grant, and Robert Swidinsky (eds.), Aspects of Labour Markel Behaviour: Essays in Honour of John Vanderkamp, pp. 201-222. University Toronto Press, 1995.
Raymond J Carroll, David Ruppert, Leonard A Stefanski, and Ciprian M Crainiceanu. Measurement error in nonlinear models: a modern perspective. Chapman and Hal//CRC, 2 edition, 2006.
Erdun Gao, Howard Bondell, Wei Huang, and Mingming Gong. A variational framework for estimating continuous treatment effects with measurement error. In The Twelfth International Conference on Learning Representations, 2024.
Paul Gustafson. Measurement error and misclassification in statistics and epidemiology: impacts and Bayesian adjustments. CRC Press, 2003.
Miquel A Hernán and James Robins. Causal inference: What if. Boca Raton: Chapman & Hall/CRC 2021.
Zhirui Hu, Zheng Tracy Ke, and Jun S Liu. Measurement error models: from nonparametric methods to deep neural networks. Statistical Science, 37(4):473 493, 2022.
Guido W Imbens and Donald B Rubin. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press, 2015.
Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autoencoders and nonlinear ica: unifying framework. In Silvia Chiappa and Roberto Calandra (eds.), Proceedings of the Twenty Thind International Conference on Artificial Intelligence and Statistics, pp. 2207-2217. PMLR, 2020.
14

---

Published in Transactions on Machine Learning Research (09/2024)
Diederik P Kingma and Max Welling. An introduction to variational autoencoders. Foundations and Trends@ in Machine Learning, 12(4):307-393 -92, 2019.
Christos Louizos, Uri Shalit, Joris M Mooij, David Sontag, Richard Zemel, and Max Welling. Causal effect inference with deep latent-variable models. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
Caleb H Miles, Joel Schwartz, and Eric Tehetgen Tchelgen. A class of semiparamelric tests of mreatment effect robust to confounder measurement error. Statistics in Medicine, 37(24):3403 3416, 2018.
Judea Pearl. Causality. Cambridge university press, odition, 2009.
Judea Pearl. On measurement bias in causal inference. arXiv preprint arXiv:1203.300. 2012.
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: foundations and learning algorithms. The MIT Press, 2017.
Carl Edward Rasmussen and Christopher KI Williams. Gaussian processes for machine learning. The MIT Press, 2006.
Severi Rissanen and Pekka Marttinen. A critical look at the consistency causal estimation with deep latent variable models. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.). Advances in Neural Information Processing Systems, volume 34, pp. 4207 4217. Curran Associates, Inc., 2021.
Susanne M Schennach. Recent advances in the measurement error literature. Annual Review of Economics, 8:341-377, 2016.
Susanne M Schennach. Mismeasured and unobserved variables. In Handbook of Econometrics, volune 7, pp. 487-565. Elsevier, 2020.
Susanne M Schennach and Yingyao Hu. Nonparametric identification and semiparametric estimation of clas- sical measurement error models without side information. Journal of the American Statistical Association, 108(501):177-18,, 2013.
Di Shu and Grace Y Yi. Causal inference with measurement error in outcomes: Bias analysis and estimation methods. Statistical methods in medical research, 287)):004,-2068, 2019.
Linda Valeri and Tyler Vanderweele. The estimation of direct and indirect causal effects in the presence of misclassified binary mediator. Biostatistics, 15(3):498-512, 2014.
Michelle Xia, Paul Richard Hahn, and Paul Gustafson. A bayesian mixture experts approach to covariate misclassification. Conadion Journal of Statistics, 48(4):731 750, 2020.
Grace Y. Yi, Aurore Delaigle, and Paul Gustafson. Handbook of Measurement Error Models. CRC Press, 2021.
Cheng Zhang, Judith Bütepage, Hedvig Kjellström, and Stephan Mandt. Advances in variational inference. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(8):2008 2026, 2019.
Yuchen Zhu, Limor Gultchin, Arthur Gretton, Matt J. Kusner, and Ricardo Silva. Causal inference with treatment measurement error: nonparametric instrumental variable approach. In James Cussens and Kun Zhang (eds.), Proceedings of the Thirty-Eight Conference on Uncertainty in Artificial Intelligence, pp. 2414-2 2-2444. PMLR, 2022.
15

---

Published in Transactions on Machine Learning Research (09/2024)
Appendix
A Review of literature on the identifiability of related models
CEME with categorical variables. Identifiability is proven by Xia et al. (2020) for a model with the same Bayesian networks as ours (Figure ! where the treatment X and covariate are categorical. For this result, misclassification probabilities need to have a certain upper bound to avoid the label switching problem. It is also assumed that the conditional distribution YIZ, X either Poisson, normal, gamma, or binomial distribution. This scheme also allows dealing with missing treatment values by defining missing value additional category.
Measurement error models from a statistical perspective. There is a wide variety of research on measurement error solely from a statistical (non-causal) perspective, studying assumptions under which measurement error model parameters can be identified, and exploring different practical methods for inferring them. These include the general method of moments (GMM) as well as kernel deconvolution and sieve estimators (Yi et al. 2021 Schennach 2020). Many of our assumptions are relaxed, such as having Gaussian noise terms, the independence of the measurement error AX outcome noise AY of the other variables,
and the outcome Y being non-differential, i.e. independent of AX. See Schhenachh 2016 2020 for a review on this kind of results.
A wide varicty of literature also studics measurement error models with additional assumptions compared to ours. These consist mainly of assuming the availability of side information such as repeated measurements or instrumental variables (IVs), or assuming a strict parametric form (e.g. linear or polynomial) for the regression function E| | x]] (and E[X* See Yi et al. (2021) for review. The IVs differ from our covariate Z in that they are conditionally independent of Y given X*. On the other hand, Ben-Moshe et al. (2017) consider a covariate Z that directly alfects Y. However, they assume that the effects of Z and X* on Y are either decoupled or E| has a polynomial parametric form.
Related latent variable models. There are also latent variable models studied that resemble the mea- surement error model. These include the causal effect variational autoencoder (CEVAE), where only a proxy of the confounder observed (Louizos et al. 2017 Rissanen Marttinen, 2021) and nonlincar independent component analysis (Khemakhem et al., 2020). Also, Schennach (2016 2020 consider the identification of latent variable models more generally.
B Experiment details
Amortized variational inference is prone to getting stuck at local optima, for example, in the case of the so-called posterior collapse (Kingma & Welling, 2019 To counter this, we start training both experiments with an increased but gradually annealed weight for the ELBO terms log 9s (2t, (s;) and log (z(z)(=? which causes the model to initially predict posterior x² close to x, to ensure that the posterior does not get stuck
at the prior. A related approach was taken by Hu et al. 2022 where the model was pre-trained assuming no measurement error.
B.1 Synthetic experiment
In the synthetic experiment data generation, the functions px=, dx- are only approximately sampled from GP save in computation cost, as with exact values the size of the GP kernel scales proportionally to the square of the number of data points, and the matrix operations performed with it scale even worse. Thus, sample only K 1000 points from the actual Gaussian processes corresponding to each of the functions px', ax. and py. These functions are then defined as the posterior mean of the corresponding GP, given that the K points have been observed. The points are evenly spaced such that the minimum is the smallest value in the actual generated data minus one quarter of the distance between the smallest and largest values in the actual data. Similarly, the maximum is the largest value in the actual data plus the distance between the smallest and largest values in the actual data. This is achieved by first generating the actual values Rii=..,, then sampling the K points for pxx, and ox,, then generating ",=.., and then finally sampling
16

---

Published in Transactions on Machine Learning Research (09/2024)
the K points for py.) For ,y, as it has two arguments, we used K 31 961 points arranged in two-dimensional grid.
When training the model, learning rate annealing is used, which means that if there have been a set number of epochs without improvement in the validation score, the learning rate will be multiplied by a set learning rate reduction factor. The training is stopped when a set number of epochs has passed without the validation score improving.
The hyperparameter values used are listed in Table 1 They were optimized using random parameter search. We use 8000 data points as the validation set used for annealing learning rate and for early stopping. Unless otherwise stated, the same hyperparameter value was used for all algorithms and datasets. From all the 43200 runs in the synthelic experiment, only 33 runs crashed, yielding no result.
Table 1: Hyperparameter values used in the synthetic experiment.
Hyperparameter
Value
Number of hidden layers in each network (fully connected)
3
Width of each hidden layer
20
Activation function
ELU
Weight decay
0
Number of importance samples
32
Initial weight of lo (o("), and log (1, (oT,,) terms (see Section 2.2.1
Number of epochs to anneal above weight to 1
10
Learning rate reducer patience
30
Learning rate reduction factor Early stopping patience in epochs
0.1
40
Adam 31
0.9
Adam 32
0.97
Batch size used for CEME/CEME* when training dataset size is 16000
256
Batch size otherwise
64
Learning rate for CEME/CEME" with training dataset sizes 1000 and 4000
0.003
Learning rate for CEME/CEME" with training dataset size 16000
0.01
Learning rate for Oracle and Naive
0.001
B.2 Experiment with education-wage data
For the education-wage dataset by Card (1955), the covariates used were personal identifier, whether the person lived near a year college 1966, whether the person lived near 4 year college in 1966, age, whether the person lived with both parents or only mother with step parents at the age of 14, several variables on which region of USA the person lived in, whether the person lived in metropolitan area (SMSA) in 1966 and/or /or 1967, heether the person was enrolled in a school in 1976, whether the person was married in 1976. and whether the person had access to a library card at the age of 14.
The semi-syuthetic dataset used in our experiments obtained from the original dataset by Card 1995 as follows: We exclude multiple covariates present the original dataset that correlate heavily with the number of oducation years and would thus make mcasurement error correction much less uscful. These include the number of schooling years of the mother and father, their intelligence quotients, Knowledge of the World of Work (KWW) score, work experience in years, its square, work experience years divided into bins, and wage. We also drop NLS sampling weight. Missing values are handled by dropping all data items that contain them. This reduces the size of the dataset from 3010 to 2990.
Then, both X* (number of education years) and all covariates in Z are scaled to have a zero mean and unit variance. The noisy treatment X obtained by adding normally distributed additive noise AX X*. Six separate datasets are created, each corresponding to different level of SD of AX. The levels are proportional to the SD of X and are 0%, 20%, 40%, 60%, 80% and 100%.
17

---

Published in Transactions on Machine Learning Research (09/2024)
A synthetic outcome Y is generated follows based the true value of the logarithm of wage: First, we train a neural network (five hidden layers of size 30, weight decay 0.01 and ELU activation function) to predict the logarithm of wage (scaled to have zero mean and unit variance). We then use the trained neural network as the function Hy [y| = E| X*). To obtain Y, we add to this expectation a Gaussian noise AY whose SD is set to 10% of the true SD estimated with training data for the jy [Z, X*] neural network. This neural network was trained on all the data, but was regularized so as not to overfit to meaningful extent. The SD only 10% of the estimated true SD, because otherwise the predictions are so inaccurate to begin with that better handling of measurement error has little potential to improve the results. The synthetic Y is also used as the ground truth for Z ]
The hyperparameter values used are listed in Table The hyperparameters are shared by all algorithms and were optimized using a random search. A separate search was conducted for each type of algorithm, but the optima were close enough that for simplicity, the same values could be chosen for each algorithm.
Table 2: Hyperparameter values used in the experiment with education-wage data.
Hyperparameter
Value
Number of hidden layers in each network (fully connected)
3
Width each hidden layer
26
Activation function
ELU
Batch size
32
Learning rate
0.001
Weight decay
0.001
Number of importance samples
32
Initial weight of log go (*%, and log (2; (1g) terms (see Section 2.2.1
8
Number of epochs to anneal above weight to Learning rate reducer patience Learning rate reduction factor Early stopping patience in epochs
5
25
45
Adam 31
0.9
Adam ,3
0.97
C Identification theorem used prove Proposition 1
For completeness, we include below Theorem by Schennach & Hu (2013) (repeated mostly word-for- fordwor))::
Definition 2. We say that a random variable has an F factor if r can be written as the sum of two independent random variables (which may be degenerated), one of which has the distribution F.
Model 1. Let y, x, x,, Au. Ay be scalar real-valued random variables related through
y g(x") + Ay x² + Az.
(13)
(14)
and y are observed while all remaining variables are not and satisfy the following assumptions:
Assumption 1. The variables x", Ax, Ay, are mutually independent, E[Ar] 0, and E[Ay] 0 (with E[|Ar|] < do and E||Ay[] < oo).
Assumption 2. Elc and E Eleiny] do not vanish for any E. R, where i = V-1. Assumplion 3. (1) F[[es==] * 0 for all g in a dense subset of IR and (ii) Eletrola")] # 0 for all in a dense subset of R (which may be different than in (i)).
Assumption 4. The distribution of x* admits uniformly bounded density far (x*) with respect the Lebesgue measure that is supported interval (which may be infinite).
18

---

Published in Transactions on Machine Learning Research (09/2024)
Assumption 5. The regression function g(x") is continuously differentiable over the interior of the support of x²
Assumption 6. The set = f-² g'(x*) 0} has at most a finite number of elements ii, Fmn If is
nonempty, fz (x") ) is continuous and nonvanishing in a neighborhood of each zyz L,... m.
Theorem 1. Let Assumplions 1-6 hold. Then there are three mulually exclusive cases:
g(x*) is not of the form
g(a") = a bln(e + d)
(15)
for some constants a, a,b. R. Then, (e-( and g(a") (over the support of fee (x*)) and the distributions of Ar and Dy in Model are identified.
g(x") is of the form (1) with d > (A case where < can be converted into a case with d > 0 by permuting the roles of x and y). Then, neither f2-(x") nor g(x") in Model are identified iff x* has a density of the form
(x=) = ee + +CDr")(e +E)
(16)
with C R, &, I0, oot and Dy has a Type extreme value factor (whose density has the form Sa(u) = exp(K exp(K3r) + Kąu) for some K1, K2, K3, KĄ RR).
3. g(x") is linear (i.e., of the form 115 with d).).
0). Then, neither Ja (x=) nor g(x") in Model 1 are identified iff x² is normally distributed and either Ax or Ay has a normal factor.
D Proof of Proposition 1
Proof. First, consider the measurement crror model defined in Equations 11 15) conditionod on a specific value = z, effectively removing Z as a variable. We show that this model, called the restricted model, as opposed the original full model, satisfies the assumptions in Theorem ! which thus determines when the rostricted model is identifiable. First, the restricted model satisfies Equations (13) and (14). Assumption 1 follows directly from the definition of the model, as |X and |AY| follow the half-normal distribution, which has the known finite expectation 0/ VT. Assumption 2 is satisfied because AX and AY are Gaussian their characteristic functions have the known form exp(iut - o²t2/2) /2) and thus do not vanish for any t E R (t is denoted by & or in Assumption 2).
Assumption of Theorem is not needed because it is proof by Schhnnach & Hu (2013) only used find the distributions of the errors AX and AY given that the density fa- (x*) and regression function g(x")
are known. However, in our case we already know by assumption that the error distributions are Gaussian with zero mean, and moreover, we can find their standard deviation from
Var (2, Var[AY]
and
Var[X] = Var[X*] Var
which hold because of the independence of (2, and AY well as X and AX, respectively. Assump- tion 4 is satisfied because X is normally distributed and thus admits a uniformly bounded density w.r.t. the Lebesgue mcasure that is supported everywhere. Assumption 5 is the same as assumption 1 of our Proposition] [ Assumption 6 follows from assumption 2 of Proposition [ since the density of X = is continuous and nonvanishing everywhere.
With the assumptions of Theorem [ satisfied, we check what the cases 1-3 therein imply for our restricted model. From case 1 we obtain that it is identified except when py (2,x*) as a function of x" is of the form ln + +d)
19

---

Published in Transactions on Machine Learning Research (09/2024)
Case considers the case when Ju (2, function of x² is of the form b bl(ee + d) with d> 0. (Having d < O is impossible as we assume the function is defined on R). In this case, our model is identifiable, as X is Gaussian and thus its density is not of the form
for (x") Acxp(-Bel Ca" + CCDx")(e Ca" +E
(17)
with Ce Ih and A, A, E. P E (0) We see this by laking a logarithm of both the Gaussian density and the density in Equation 17 and noting that the first is a second-degree polynomial, but the latter is sum without a second-degree term, so they are not equal. Thus, our model is identifiable in this case.
The remaining case for when (z(2z=²) linear in x². In this case, the restricted model is not identified as x² is normally distributed and both Ax and Ay have normal factors as they are normal themselves.
Next, prove the identifiability of the full model, i.c. where 2 may take any value in its range. We start by assuming in accordance with Definition that two conditional observed distributions from the model match for every
po( u|=) - pa (rr (r, u/z).
(18)
Now based on assumption 3 of Proposition 18 we obtain that
there exists for which by (2,xx) not linear in x². From
Vx, y
po 212) - po (a, 213),
which in turn implies
= 0 = o'
nx. (3) = p'x (z) xx- = d'x (3)
(19)
Va E R: (y(2xx)) = u'y (2,x*)
by using Definition on the restricted model, which identified according to the first part of the proof.
Similarly, we obtain the equalities in 19 for every for which py x²) is not linear in x². Thus, for the full equality of the models (i.e. for 0'), it remains to be shown that we obtain (x-(22) = (z)- (x) = (2) and py (2,xx") (2, also for those 2 for which py x²) is linear in x². Noting that we already know that =' and = o', we obtain for such a Shear-Gaussiae measurement error model with known measurement error, which is known to be identifiable (see e.g. Gustafson 2003 Thus 0 = 0' and the proof is complete.
20


Reviews:
[{"review_id": "7ghCGhqTtb", "reviewer": "Reviewer_La5Z", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The setting in this paper, causal inference under the inaccurate observation variables and without side information, is interesting. \n2. An identifiability analysis is provided. Sufficient experiments are conducted to verify the methods.\n\nWeaknesses:\n\n1. The organization and writing of this paper is poor.\n2. In Figure 1, is only the noisy data available for the model? The accurate data also contains noise.\n3. In equations 5-6, why do the authors assume that the mean of the Gaussian distribution is zero, not a learnable variable?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies causal inference under inaccurate observation variables and without side information. To measure the error, a new causal model is proposed. The authors propose a method to train the model and analyze its identifiability. Experiments on synthetic and real-world datasets are conducted to verify the method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. In subfigure (a), Figure 10, the experiment results for other methods are missing.\n2. It is better to denote which is the proposed method (CEME and CEME+), which is the baseline (CE-simple), and which is the goal (CE-oracle).", "broader_impact_concerns": "This work does not present direct ethical concerns or adverse broader societal impacts.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Hrw7dYkFX7", "reviewer": "Reviewer_qb2H", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. This work investigate a challenging setting with measurement treatment, complex nonlinear dependencies, and no side information.\n\n2. This work provides extensive empirical results to demonstrate the effectiveness of their proposed method.\n\nWeaknesses:\n\n1. The contribution of this paper is marginal. The setting investigated in this paper is similar to Gao et al. (2024), both its inference method and identifiability analysis builds closely upon previous works (Zhang et al., 2019) and (Schennach & Hu (2013)). Besides, all experiments employ Guassian noises, I'm not sure whether the proposed method can handle non-Gaussian cases properly.\n\n2. The organization of this paper is confusing. The authors present too many details in the introduction, which should be deferred to \"Related Work\" section or \"Preliminaries\" section. In the introduction, they only need to formulate the setting they investigated clearly and emphasize their main contributions.\n\n3. The writing of this paper should be improved, please see more details in \"requested changes\".", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper focus on the scenario with noisy treatment, nonlinear dependencies, and no side information. It not only provides identifiability analysis but also develops a method for inferring causal effects. Extensive experimental results demonstrate the effectiveness of their proposed method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. In line 2 of the second paragraph of Section 1, \"Unaccounted\" should be replaced with an adverb.\n\n2. In the fourth paragraph of Section 1, they authors claim that \"The assumption of independent exogenous variables implies that there is no unobserved confounding\", which is incorrect. The author should implicitly claim that they employ the causal sufficiency assumption, violation of this assumption is not incompatible with independent exogenous variables.\n\n3. In page 3, the authors claim that \"... using so called side-information, such as a known error mechanism, ...\". However, they assume independent and zero-mean additive errors, (i.e. known error mechanism) and claim that they need no side information. This is quite confusing.\n\n4. In Section 2.2.1, \"The standard ELBO corresponds to $k=1$\" should be replaced with \"The standard ELBO corresponds to $K=1$\".", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "NT0TINQAvF", "reviewer": "Reviewer_y5Kg", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\nI think that the paper is well written with generally clear exposition. However, there are some notation in equations that could be explained better.\n\n- It is good that the key hyper parameters of the training procedures is mentioned.\n\n### Weaknesses\nSee Requested Changes.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper aims to carry out causal inference in settings where the observed variables are subject to noise. They aim to account for this noise _without_ using extra _side_ information. The work solves for a causal graph using amortised variational inference, and test their method on synthetic data, as well as on education wage data.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Could an example be given in the introduction for where the scenario in which the work in the paper is relevant? The paper mentions _complicated by variables being subject to noise, e.g., inaccurate measurement, clerical error, or self-reporting (often called misclassification for categorical variables)._, etc, but I think a grounded example could help. I think this would go well when describing Figure 2, where you can given an example of a dataset / scenario where the causal graph of Figure 2 applies.\n\n- Page 1: _'...how a **cause** variable X, called treatment...'_: is this a typo? Should it be **causal** variable?\n\n- Page 1 and 2: '_Figure 1 presents an example of this skewing effect_':  Could you explain more about how Figure 1 shows a skewing effect in the text where you say that Figure 1 presents a skewing effect?\n\n- Figure 1:  I am confused about how Figure 1 is generated. Could this be explained in more detail? What do you mean by a Ground Truth _regression_ function? Is this the function from which the data is generated? If so, what is the difference between the accurate data and the noisy data?\n\n- Page 3: What is the difference between $\\mu_y(Z, X^*)$ and $\\mu_y(z, x^*)$? If there is a difference, could this be explained?\n\n- Page 3: I am confused by the notation of $\\Delta X$, etc. Why not just use $U_X$, etc?\n\n- Algorithm 1: In step 2, what does it mean to sample approximately from a GP(0, K)?\n\n- Page 10: _To evaluate our methods, we need to know the ground truth causal effect, _, could you explain this further?\n\n- Figures: I think that the figure captions should have sentences about what the key takeaway of each figure is.\n\n- Appendixes: I think it would be clearer if the neural network architecture details are added to a table, or are a part of Tables 1 and 2, or are highlighted in some, so that they are easier to parse quickly.", "broader_impact_concerns": "Since the applications of this paper are medical and economical, I wonder if a brief Broader Impact statement would be good.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper contributes to the field of causal inference. It specifically targets complex scenarios with noisy treatments, nonlinear dependencies, and absence of side information.

During the review process, concerns were initially raised regarding the organization and clarity of the manuscript. Following the authors' revisions and responses, these concerns have been adequately addressed, and the claims are now well-supported. Reviewers have appreciated these revisions, showing a consensus towards acceptance of the paper. The remaining point of discussion pertains to the technical contributions, which is not a major concern according to TMLR's evaluation criteria. Given the above points, and aligning with the reviewers' final leanings, I recommend acceptance of this paper.
Decision: Accept (poster)