Paper: A Variational Framework for Estimating Continuous Treatment Effects with Measurement Error (ICLR-2024)

Reviewer comment (Theory):
It's not really apparently where the measurement errors comes into play (more concretely I'm not sure the DAG makes sense to me). To take your meme example: Doctor prescribes S, which was prescribed due to X, and the resulting treatment as a result of the patient not adhering to the prescription is T. The resulting outcome is Y, with measurement error too. So wouldn't the DAG be: X->S | S+U->T | X + T + eps -> Y

Author response:
**[Q1. It's not really apparently where the measurement errors comes into play (more concretely I'm not sure the DAG makes sense to me). To take your meme example: Doctor prescribes S, which was prescribed due to X, and the resulting treatment as a result of the patient not adhering to the prescription is T. The resulting outcome is Y, with measurement error too. So wouldn't the DAG be: X->S | S+U->T | X + T + eps -> Y]**

Thanks for this valuable suggestion for the improper meme we introduced in Fig.1. According to the current meme, the corresponding DAG would be as you suggested instead of our focused one. 

In our paper, we focus on continuous treatment data measured with classical error; instead of observing the received true treatment, researchers only observe the sum of the treatment and a random error, which mainly comes from the error reporting issue. The real case of this problem can be found in the Epidemiologic Study Cohort data from the first National Health and Nutrition Examination (NHANES-I) [1,2], where over $75\%$ of the variance in the fat intake data is made up of measurement error since the daily saturated fat intake was measured using a single 24-hour recall. In our paper, we focus on continuous treatment data measured with classical error; instead of observing the received treatment, researchers only observe the sum of the treatment and a random error.

To correct this mistake, we have updated the example and provided a new meme in the revision to avoid misunderstanding our objective.

Reviewer comment (Evaluation):
The experimental results section is disappointing. On synthetic datasets, unless you have a high measurement (which I'm not sure if completely reasonable but regardless), the non-VI approaches perform basically on par. But semi-synthetic really shows the this method is ineffective: VI underperforms significantly. And only reason to use NN is when there is when there is high-dimensions, so not sure what the point of this method is then.

Author response:
**[Q3. The experimental results section is disappointing. On synthetic datasets, unless you have a high measurement (which I'm not sure if completely reasonable but regardless), the non-VI approaches perform basically on par. But semi-synthetic really shows the this method is ineffective: VI underperforms significantly. And only reason to use NN is when there is when there is high-dimensions, so not sure what the point of this method is then.]**

We politely disagree that our experimental results are disappointing, as we believe the perceived disappointment may stem from an insufficient clarification of the limitations of our methods. After reading your comment, we think it should be the unclearificaiton of the limitations of our methods lead to all these confusion. We intend to address your concerns point by point.

> Unless you have a high measurement (which I'm not sure if completely reasonable but regardless), the non-VI approaches perform basically on par.

We partially agree with this argument that our method is only useful in the presense of a high measurement error. We contend that its advantages extend even when the measurement error is relatively low. We clarify this argument from three points as follows.

(1) *Why do non-VI methods perform well when we have a low measurement error?*

The presence of measurement error (ME) can introduce bias in estimating the true underlying parameters, and this bias tends to decrease as the measurement error diminishes [1]. Therefore, non-VI methods would be expected to perform well when we have a low ME. Notably, some previous works [2,3] also directly take the methods which ignore the measurement error as baseline methods. 

(2) *In our experiemental results, why does the non-VI approaches perform basically on par.*

In our experiment involving "varying ME standard deviation", we systematically increased the standard deviation of the measurement error to illustrate the consistent advantages of our method. As you correctly noted, our approach exhibits performance gains, particularly when the ME is relatively high (std$\geq0.2$). For lower errors ($0.05, 0.1, 0.15$), the non-VI methods perform on par with that of our methods. There are two primary reasons for this case.

- The impact of measurement error on non-VI methods is expected to be small when the ME is low.
- While our method is expected to outperform non-VI methods since it provides an unbiased estimator for treatment effect, it is essential to note that our method introduces additional parameters (see Figure 6d) to infer the treatment from measured variables. This complexity in the learning process necessitates more data to fit the whole model.

Therefore, when the number of observations is limited ($2000$ for this setting in our paper), the performance of non-VI methods and our methods may be comparable.

(3) *With more data, our method can perform better with a low measurement error.*

As we discussed, our method provides an unbiased estimator for the treatment effect, albeit requiring more data. To verify this, we added an experiment when the ME is realtively low (std=0.1), where we systematically varied the number of observations to compare our method with a non-VI method. For the sake of illustration, we present the results (Average MSE) of VCNet and VF-VCNet (ours) for example. The results are detailed below:

| Number of observatioins| VCNet (AMSE) |VF-VCNet (AMSE) |
|------|--------------------|-------------------|
| 2000 |   0.029 ± 0.009    | 0.024 ± 0.007     |
| 3000 |   0.016 ± 0.007    | 0.021 ± 0.005     |
| 4000 |   0.015 ± 0.006    | 0.015 ± 0.006     |
| 5000 |   0.018 ± 0.007    | 0.015 ± 0.009     | 
| 6000 |   0.018 ± 0.006    | 0.013 ± 0.007     | 
| 7000 |   0.017 ± 0.005    | 0.009 ± 0.002     |
| 8000 |   0.018 ± 0.006    | 0.008 ± 0.004     | 

Based on the above results, it is evident that VCNet tends to converge to a biased model as the number of observations increases. In contrast, the AMSE of our method consistently decreases with a larger number of observations.

Reviewer comment (Theory):
VAE suffers from non-identifiability. Although beta VAE might help disentangling the factors of variations, it doesn’t provide a guarantee. This is a potential limitation that is not discussed.

Author response:
Thank you for your insightful suggestion, prompting us to refine the difference between our variational framework and the existing treatment effect estimation (TEE) methods based on Variational Autoencoders (VAEs).

In the realm of TEE, a fundamental challenge is addressing the selection bias, rooted in confounding variables that influence both treatment choice and outcomes. Machine learning approaches typically branch into two methods: discriminative methods [2] rely on Generalized Propensity Score (GPS) to extract the confounders, while generative methods [1] design a latent generative model for covariates and aim to identify the latent variables as you noted.

In our approach, we primarily employ GPS to extract the confounder, aligning with the discriminative methods. However, our differentiator lies in utilizing the variational framework, focusing on the unobservability of the treatment variable rather than covariate-related issues. We leverage variational methods because our learning objective—specifically, the marginal log-likelihood of observed variables, entails an integral over the unobserved treatment, presenting a computational challenge for analytical solutions. 

For generative models, they assume that the covariates are generated by latent confounding variables and adjustment variables via a mixing function. The adjustment variables only affect the outcome values. Therefore, the key of this method is to (1) recover/identify the latent factors, (2) and separate the confounding variables, which inevitably raise the model identifiability issue. As you rightly pointed out, Nonlinear Independent Component Analysis (ICA) [3] techniques achieve model identifiability by introducing distribution assumptions on latent variables relying on some auxiliary information. Moreover, recent research [4] also explores theoretical guarantees for separating latent factors, offering promising contributions to this challenge. 

We have incorporated these refinements into the Appendix of the revision to help the audience who may have similar concerns on this point. Thank you for guiding us towards a more comprehensive explanation.

Reviewer comment (Presentation):
Plots have very small fonts and proposed method is not marked (e.g. “ours”) making it difficult to read.

Author response:
We have updated our figure to make it clear and emphasized our methods with "ours" according to your suggestion.

Reviewer comment (Theory):
Can you please comment / discuss on the non-identifiability of VAEs? Could identifiability be improved by the use of some auxiliary information, similarly to https://proceedings.mlr.press/v108/khemakhem20a/khemakhem20a.pdf?

Author response:
Thank you for your insightful suggestion, prompting us to refine the difference between our variational framework and the existing treatment effect estimation (TEE) methods based on Variational Autoencoders (VAEs).

In the realm of TEE, a fundamental challenge is addressing the selection bias, rooted in confounding variables that influence both treatment choice and outcomes. Machine learning approaches typically branch into two methods: discriminative methods [2] rely on Generalized Propensity Score (GPS) to extract the confounders, while generative methods [1] design a latent generative model for covariates and aim to identify the latent variables as you noted.

In our approach, we primarily employ GPS to extract the confounder, aligning with the discriminative methods. However, our differentiator lies in utilizing the variational framework, focusing on the unobservability of the treatment variable rather than covariate-related issues. We leverage variational methods because our learning objective—specifically, the marginal log-likelihood of observed variables, entails an integral over the unobserved treatment, presenting a computational challenge for analytical solutions. 

For generative models, they assume that the covariates are generated by latent confounding variables and adjustment variables via a mixing function. The adjustment variables only affect the outcome values. Therefore, the key of this method is to (1) recover/identify the latent factors, (2) and separate the confounding variables, which inevitably raise the model identifiability issue. As you rightly pointed out, Nonlinear Independent Component Analysis (ICA) [3] techniques achieve model identifiability by introducing distribution assumptions on latent variables relying on some auxiliary information. Moreover, recent research [4] also explores theoretical guarantees for separating latent factors, offering promising contributions to this challenge. 

We have incorporated these refinements into the Appendix of the revision to help the audience who may have similar concerns on this point. Thank you for guiding us towards a more comprehensive explanation.

Reviewer comment (Presentation):
Plots are very small, please update the fonts to make it more readable. Also mark your method as "ours" to make sure the reader sees what they compare.

Author response:
We have updated our figure to make it clear and emphasized our methods with "ours" according to your suggestion.

Reviewer comment (Presentation):
Figure 2: it's common to also add the parameters of the models in the plot (\theta_y, \theta_t, \phi). e.g., check also the original VAE paper.

Author response:
Thanks for this suggestion. We have added the parameters of the models in Figure 2 to make it clear.

Reviewer comment (Writing):
Figure 3: What does MiM stand for?

Author response:
MiM stands for the multiple index model. We have updated the full name of this abbreviation in the revision.

Reviewer comment (Evaluation):
Metrics: "we employ the widely recognized metric" - can you please provide citation?

Author response:
Thanks for your suggestion. We have added the related citations in the revision.

Reviewer comment (Evaluation):
"this metric quantifies the proximity of our predicted outcomes to the true potential outcomes when considering covariates and the corresponding treatment" -> from the definition of the metric this doesn't quantify the error for the corresponding treatment but for all possible treatments in T. Please update the text.

Author response:
Thanks for your advice. We have updated the text to *this metric quantifies the average approximating error of our predicted outcomes to the true potential outcomes for all possible treatments in $T$* in the revision.

Reviewer comment (Presentation):
Figure 4: True response seems same between all plots, is this correct? So in that sense it's one simulation dataset but for different noise distributions?

Author response:
> Is this correct?

Yes, this is correct. 

> it's one simulation dataset but for different noise distributions?

In these four different simulations, we only change the noise distributions. We showcase these results to demonstrate the robustness of our methods in the face of diverse measurement error scenarios.

The consistency across all true responses in the four sub-figures arises from the fact that they correspond to the same data generation process, but with varying types of measurement errors. It is essential to emphasize that the underlying true response remains unaffected by the measurement error, consistently maintaining its integrity across all sub-figures.

Reviewer comment (Evaluation):
Table 1: Why APE is 0.0 +- 0.0? The model seems to have high AMSE error but no error (!) on APE. Would be curious to understand what happened there.

Author response:
AMSE quantifies the error in outcome prediction, while APE measures the error in optimal policy estimation. While these two metrics are often correlated. We found that, in the News dataset, the optimal policy is relatively straightforward to learn as the optimal policy is the largest treatment and the true ATE increases with the treatment. However, the intricate nature of $p(T|\mathbf{X})$ contributes to a suboptimal performance in outcome prediction.

Reviewer comment (Presentation):
Appendix B.4 last sentence: Fix the citation (?)

Author response:
We have fixed this citation issue in the revision.

Reviewer comment (Experiments):
I think one of the weakest section in the paper is the experiments. The proposed algorithm is only compared with state-of-art algorithms in synthetic and semi-synthetic datasets where both treatments and outcomes are synthetically-generated. Additionally, IHDP data has only ~700 samples. It would have been much stronger if the proposed algorithms are tested in real-world dataset with more samples.

Author response:
**[Q2. I think one of the weakest section in the paper is the experiments. The proposed algorithm is only compared with state-of-art algorithms in synthetic and semi-synthetic datasets where both treatments and outcomes are synthetically generated. Additionally, IHDP data has only 700 samples. It would have been much stronger if the proposed algorithms are tested in real-world dataset with more samples.]**

> Why do we test all methods on simulations and semi-synthetic data?

The primary challenge in causal inference lies in the unobservability of counterfactual outcomes. In other words, the true treatment effect is never observed (i.e., missing in any training data), rendering it impractical for training predictive models. This makes estimating treatment effects a more difficult problem than the standard supervised learning problem. Furthermore, the challenge extends to not allowing for meaningful evaluation of real-world datasets due to the inability to observe the counterfactuals. Consequently, akin to prior research practices [3,4,5,7], we resort to employing well-established benchmark synthetic models to assess the performance of all treatment effect estimation methods.

> Large dataset with more samples.

Thanks for your suggestion. We have expanded our analysis to include experimental results from a larger dataset, namely The Cancer Genomic Atlas (TCGA), which encompasses gene expression measurements for cancer patients [6]. There are in total $9659$ samples for which we used the measurements from the $4000$ most variable genes. In alignment with established methods [5,7], we have adhered to the same procedures for generating the values treatment and outcome variables. Details of this comparison are presented in the following table.

| Method          | TCGA (AMSE)      | TCGA (APE)      |
|-----------------|------------------|-----------------|
| TARNet          | 0.042 ± 0.007    | 0.136 ± 0.069   |
| DRNet           | 0.047 ± 0.007    | 0.182 ± 0.040   |
| ATTNet          | 0.056 ± 0.019    | 0.076 ± 0.050   |
| VCNet           | 0.046 ± 0.010    | 0.067 ± 0.045   |
| VF-ATTNet(Ours) | 0.042 ± 0.010    | 0.057 ± 0.037   |
| VF-VCNet (Ours) |**0.019 ± 0.014** |**0.036 ± 0.017**|

Compared to the baseline methods, our approaches consistently demonstrate enhanced performance across both AMSE and APE metrics. Notice that our two methods should be compared with their respective base methods, namely VF-VCNet versus VCNet and VF-ATTNet versus ATTNet.

We have updated this result in the revision to further verify the effectiveness of our proposed method.

Reviewer comment (Experiments):
What happens to performance of the algorithm if the treatments are measured without an error?

Author response:
**[Q3. What happens to performance of the algorithm if the treatments are measured without an error?]**

Notice that in our paper, we develop our method under the assumption that **the distribution formulation of the measurement error, denoted as $U$, is known.** When there is no measurement error present, the distribution of the measurement error becomes a discrete distribution with its probability mass concentrated solely on zero. Consequently, it allows for the direct determination of the true variable $T$ without the need for a variational step to infer $T$ from observed variables $S$ and $Y$. **In this case, our variational framework would degenerate to the basic model that directly takes $T$ to estimate the treatment effect.**

In real-world scenarios [1], researchers usually devise specific methods to deal with measurement errors in situations where there is a confirmed presence of measurement error in the data. This suspicion typically arises when there are inconsistent multiple measurements of a variable for a single individual. Furthermore, the existence of multiple measurements enables the identification of characteristic functions associated with the measurement error [2,8]. Building upon this fact, our proposed method capitalizes on prior knowledge of the measurement error and strategically maximizes the joint likelihood to get an unbiased estimator for treatment effects.

We hope this clarification addresses your question adequately. Additionally, we have incorporated this discussion into the Appendix to help any audience who may have concerns on this point.

[1] [Carroll R. J., Rupper D., Stefanski L. A., Crainiceanu C. M. (2006). Measurement error in nonlinear models: A modern perspective. Chapman Hall/CRC.](https://www.taylorfrancis.com/books/mono/10.1201/9781420010138/measurement-error-nonlinear-models-ciprian-crainiceanu-raymond-carroll-leonard-stefanski-david-ruppert)

[2] [Huang, W. and Zhang, Z. (2023). Nonparametric estimation of continuous treatment effect with measurement error. JRSSB, 85, 474 - 496.](https://academic.oup.com/jrsssb/article/85/2/474/7098034)

[3] [Claudia Shi, David Blei, and Victor Veitch. Adapting neural networks for the estimation of treatment effects. In Advances in Neural Information Processing Systems, pp. 2503–2513, 2019.](https://proceedings.neurips.cc/paper/2019/file/8fb5f8be2aa9d6c64a04e3ab9f63feee-Paper.pdf)

[4] [Lizhen Nie, Mao Ye, Dan Nicolae, et al. Vcnet and functional targeted regularization for learning causal effects of continuous treatments. In International Conference on Learning Representations, 2020.](https://openreview.net/pdf?id=RmB-88r9dL)

[5] [Xin Wang, Shengfei Lyu, Xingyu Wu, Tianhao Wu, and Huanhuan Chen. Generalization bounds for estimating causal effects of continuous treatments. Advances in Neural Information Processing Systems, 35:8605–8617, 2022.](https://proceedings.neurips.cc/paper_files/paper/2022/file/390bb66a088d37f62ee9fb779c5953c2-Paper-Conference.pdf)

[6] [John N Weinstein, Eric A Collisson, Gordon B Mills, Kenna R Mills Shaw, Brad A Ozenberger, Kyle Ellrott, Ilya Shmulevich, Chris Sander, Joshua M Stuart, Cancer Genome Atlas Research Network, et al. The cancer genome atlas pan-cancer analysis project. Nature Genetics, 45(10):1113, 2013.](https://www.cancer.gov/ccg/research/genome-sequencing/tcga)

[7] [Bica, Ioana, James Jordon, and Mihaela van der Schaar. Estimating the effects of continuous-valued interventions using generative adversarial networks. Advances in Neural Information Processing Systems 33 (2020): 16434-16445.](https://proceedings.neurips.cc/paper/2020/file/bea5955b308361a1b07bc55042e25e54-Paper.pdf)

[8] [Delaigle, Aurore, Jianqing Fan, and Raymond J. Carroll. A design-adaptive local polynomial estimator for the errors-in-variables problem. Journal of the American Statistical Association 104.485 (2009): 348-359.](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2846380/)