Paper: Nuisance-Robust Weighting Network for End-to-End Causal Effect Estimation (ICLR-2024)

Reviewer comment (Evaluation):
Considerable diﬀerence between the results of NuNet and SNet (best performing) pertaining to PEHE on the additive noise dataset.

Author response:
Dear reviewer KLj8, many thanks for your positive feedback and for concerning the empirical comparison.

## Q1. Considerable diﬀerence between the results of NuNet and SNet (best performing) pertaining to PEHE on the additive noise dataset.

Compared to SNet, our base model DRNet is also outpaced in the AN setting (NuNet generally outperforms DRNet).
SNet has a special network architecture designed for CATE estimation (shared representation for $f$s and $\mu$), which may serve as an effective inductive bias.
The exploration of architecture is an important aspect that is orthogonal to our considerations.
Combining such an architecture is an important topic for future work as we wrote in the conclusion.

> On the other hand, it cannot be applied to methods that are originally
formulated as joint optimization of models and weights, such as representation decomposition. Such
models that have representation shared by the model and weights often exhibit effective inductive
bias, which is another aspect of the combination of the weighting approach and DNNs. Deriving
a pessimism-based theoretical framework for such methods and investigating principled learning
methods would be a promising future direction for versatile causal inference methods.

Again, thank you for your time and enthusiasm in reading our paper and providing feedback.

Reviewer comment (Theory):
I am not convinced of the advantages of the proposed method. The robustness of CATE estimation with respect to estimated nuisance functions is central to recent well-established works on CATE estimation that make use of semiparametric estimation theory, e.g., by Chernozhukov et al. (2018), Foster and Syrgkanis (2019), Nie and Wager (2021), Kennedy (2023). One of the key results is that CATE estimators with Neyman orthogonal loss functions (e.g., DR-learner, R-learner) are robust with respect to estimation errors of nuisance parameters (response surfaces, propensity score) in the sense of a fast guaranteed convergence rate. It is not clear to me that the proposed adversarial end-to-end approach improves on that. The proposed method already uses the Neyman orthogonal loss of the DR learner which makes the adversarial approach of optimizing for a pessimistic propensity score seem redundant.

Author response:
Thank you for raising this quite important point of discussion.
Since the current manuscript mainly discussed methods with deep models, there was only a limited description of parametric statistical methods in the introduction as follows.

> Various countermeasures have been tried to alleviate this problem, such as doubly robust (DR) methods (Kang et al., 2007; Kennedy, 2020; Dudík et al., 2014), which are clever combinations of outcome prediction models and only weighting its residuals using estimated propensity. Nevertheless, the IPW-based approach’s limitation is the instability of the two-step procedure in non-asymptotic situations (Athey et al., 2018).

As implied above, we believe that these (semi-)parametric analyses based on asymptotics assuming $p < N$ (at least w.r.t. the target $\tau$) have much room for improvement when applied to a complex (deep) model.
The larger the hypothesis space, the more likely a hypothesis with an optimistic loss value will exist within it (and such a hypothesis is to be selected).
In addition to the large hypothesis space, the extreme weights (even if accurate) exacerbate the effective sample size (Tanimoto et al. 2022, reference in the manuscript), which is usually ignored in the convergence rate analyses as a constant coefficient.
Therefore we need a theory and a method that take care not only of the inaccuracy of the nuisance propensity model but also the estimation variance due to extreme weights.

A series of DNN-based CATE estimation methods including (Shalit et al., 2017) avoid weighting, but this too has been pointed out to be theoretically limited (Johansson et al., 2019; Zhao et al., 2019; Wu et al., 2020).
Some DNN-based methods such as (Shi et al. 2019; Hassanpour and Greiner, 2019) adopt simultaneous training of the model and weights; however, they adopt naive simultaneous training of them and lack a principle of their joint objectives.

We would like to expand the description of the manuscript in this comparison.

Reviewer comment (Experiments):
Despite being central to the topic of the paper, three of the four works mentioned above are not cited in the paper. Neither is the R-learner considered as a baseline.

Author response:
I would like to refer to the references you mentioned and clarify clarify along with our answer to Q1.

We added the R-Learner as a baseline. See the overall response.
R-Learner (RNet) exhibited similar trends with the DR-Learner.
Our NuNet outperformed RNet in most settings.

Reviewer comment (Novelty):
A property of the DR loss is that only requires **either** the propensity score **or** the response surfaces to be estimated correctly to achieve a fast convergence rate (Kennedy, 2023). This would make the proposed approach redundant if the response surface estimators converge sufficiently fast.

Author response:
Actually, deep models are highly expressive and converge even with modeling surface learning alone.
In order to speed up the convergence, models with flexible induction bias for the response surface are being explored.
SNet is one such consideration.
In practice, however, DRNet and NuNet with weighting often outperform SNet even without such a sophisticated model architecture (Figs. 2 and 3 in the manuscript).
Therefore, it is important to have a framework that can utilize the expressive power of DNNs with weighting.

Reviewer comment (Experiments):
While the method performs well in the experiments, the datasets seem to favor methods that focus on response function estimation rather than propensity score. PW-Net has a huge variance and the estimation error seems to grow with sample size for some reason. I suspect that there might be possible overlap violations in the simulated data. I could imagine that the proposed method offers some advantages in dealing with overlap violations, which might lead to an alternative way to frame the paper. However, this would require additional intuition and experiments.

Author response:
Thank you for presenting a suggestive hypothesis for overlap violation.
This seems to have substantially in common with our hypothesis.
In a literal sense, our data generation process satisfies the overlap assumption (Appendix C1).
On the other hand, it may be very close to that (as is the case with real data), and the true density ratio (and thus the true weight) is to be very large.
In this situation, as noted in the Q1 section, accurate weighting would rather result in a large estimation variance.
This explanation is also consistent with the fact that methods using cross-fitting (presented in the overall response) have struggled.

Again, we so much appreciate the very thorough discussion.

Reviewer comment (Novelty):
The problem of CATE estimation (or more generally, statistical estimation with nuisance parameters) is already quite well understood regarding the robustness to nuisance errors. I am not convinced that the proposed approach adds much benefit to the existing state-of-the-art.

Author response:
Thank you for raising this quite important point of discussion.
Since the current manuscript mainly discussed methods with deep models, there was only a limited description of parametric statistical methods in the introduction as follows.

> Various countermeasures have been tried to alleviate this problem, such as doubly robust (DR) methods (Kang et al., 2007; Kennedy, 2020; Dudík et al., 2014), which are clever combinations of outcome prediction models and only weighting its residuals using estimated propensity. Nevertheless, the IPW-based approach’s limitation is the instability of the two-step procedure in non-asymptotic situations (Athey et al., 2018).

As implied above, we believe that these (semi-)parametric analyses based on asymptotics assuming $p < N$ (at least w.r.t. the target $\tau$) have much room for improvement when applied to a complex (deep) model.
The larger the hypothesis space, the more likely a hypothesis with an optimistic loss value will exist within it (and such a hypothesis is to be selected).
In addition to the large hypothesis space, the extreme weights (even if accurate) exacerbate the effective sample size (Tanimoto et al. 2022, reference in the manuscript), which is usually ignored in the convergence rate analyses as a constant coefficient.
Therefore we need a theory and a method that take care not only of the inaccuracy of the nuisance propensity model but also the estimation variance due to extreme weights.

A series of DNN-based CATE estimation methods including (Shalit et al., 2017) avoid weighting, but this too has been pointed out to be theoretically limited (Johansson et al., 2019; Zhao et al., 2019; Wu et al., 2020).
Some DNN-based methods such as (Shi et al. 2019; Hassanpour and Greiner, 2019) adopt simultaneous training of the model and weights; however, they adopt naive simultaneous training of them and lack a principle of their joint objectives.

We would like to expand the description of the manuscript in this comparison.

Reviewer comment (Novelty):
There seems to be substantial discussion of related work missing.

Author response:
## W1. related work missing

> * (Shi et al., 2019) Adapting Neural Networks for the Estimation of Treatment Effects, https://arxiv.org/abs/1906.02120
> * (Chernozhukov et al., 2022) RieszNet and ForestRiesz: Automatic Debiased Machine Learning with Neural Nets and Random Forests https://arxiv.org/abs/2110.03031

We have added (Shi et al., 2019) in our revision as you mentioned.
The other one (Chernozhukov et al., 2022) is a similar approach based on Neyman orthogonality and (cooperative) joint optimization.
These approaches share the same difficulties when considering their application to our setting.

* Neyman orthogonality itself only guarantees the first-order robustness of nuisance estimation error. Higher-order also matters when the target model and nuisance are complex such as DNNs for CATE estimation rather than the ATE estimation (both of the above methods are for the ATE estimation).
* A naive (cooperative, i.e. non-adversarial) joint objective is not justified just because the plug-in (two-step) estimation is justified. See the response to Q1.

Reviewer comment (Novelty):
Some examples of such existing work include:
- Adapting Neural Networks for the Estimation of Treatment Effects, https://arxiv.org/abs/1906.02120
- RieszNet and ForestRiesz: Automatic Debiased Machine Learning with Neural Nets and Random Forests https://arxiv.org/abs/2110.03031

Author response:
## W1. related work missing

> * (Shi et al., 2019) Adapting Neural Networks for the Estimation of Treatment Effects, https://arxiv.org/abs/1906.02120
> * (Chernozhukov et al., 2022) RieszNet and ForestRiesz: Automatic Debiased Machine Learning with Neural Nets and Random Forests https://arxiv.org/abs/2110.03031

We have added (Shi et al., 2019) in our revision as you mentioned.
The other one (Chernozhukov et al., 2022) is a similar approach based on Neyman orthogonality and (cooperative) joint optimization.
These approaches share the same difficulties when considering their application to our setting.

* Neyman orthogonality itself only guarantees the first-order robustness of nuisance estimation error. Higher-order also matters when the target model and nuisance are complex such as DNNs for CATE estimation rather than the ATE estimation (both of the above methods are for the ATE estimation).
* A naive (cooperative, i.e. non-adversarial) joint objective is not justified just because the plug-in (two-step) estimation is justified. See the response to Q1.

Reviewer comment (Writing):
(EDIT after revision) In the updated draft, the passage "Joint optimization approaches have also been proposed for ATE estimation (Shi et al., 2019), though it may lead to cheating by less weighting to noisy regions especially under noise heterogeneity" is unclear — it is not clear what "cheating" means and what "less weighting" means.

Author response:
## Q1. I am not sure what "cheating" means and what "less weighting" means?

In our revised manuscript, we wrote

> Joint optimization approaches have also been proposed for ATE estimation (Shi et al., 2019), though it may lead to cheating by less weighting to noisy regions especially under noise heterogeneity.

Naive joint optimization risks unintended weighting, which is the reason why we needed adversarial formulation.
The weights are intended to adjust the distribution imbalance, as you know.
However, another motive is introduced unintendedly when jointly optimized.

Consider a simple objective function of the form $\min_{\mu, \theta} \sum_n w^{(n)}_{\mu} \ell^{(n)}$ with weights $w$. 

A trivial solution would be the $\mu$ that realize $w_\mu^{(n)} \to 0$, which of course can be prevented by normalization $\sum w^{(n)}_{\mu}=1$.

On the other hand, when the noise levels differ across regions (the conditional variance $\mathbb{V}[\ell | x]$ is heterogeneous among $x$), the weights $w$ would be skewed toward easy instances (such $x$ that have small $\mathbb{V}[\ell | x]$) and hard instances are to be ignored.
We call this "cheating."

Such biased weights result in a biased loss surface in the target hypothesis space. A hypothesis with optimistic-side error (i.e., the loss is evaluated lower than the true value due to weights) tends to be selected in the training especially when the hypothesis space is large (e.g., complex CATE models) since the existence of such a highly optimistically evaluated hypothesis would be more probable.

As shown in the additional experimental results (see below), naive joint optimization with reweighting makes the training quite unstable.

Reviewer comment (Writing):
The sentence "Although f0 and f1 are also nuisance parameters, the uncertainty of them do not differ among the target parameter space, thus we need not take care" is very hard to understand; it is not complete and certain phrases are not defined, such as "uncertainty does not differ" and "not taking care." It is unclear which uncertainty is being referred to.

Author response:
## Q3. it is not complete and certain phrases are not defined "uncertainty does not differ" and "not taking care". Which uncertainty?

In our revised manuscript, we wrote

> Although $f_0$ and $f_1$ are also nuisance parameters, the uncertainty of them do not differ among the target parameter space, thus we need not take care.
> Excluding the error correction term in (8), a disturbance ($\Delta f_0, \Delta f_1$) can be absorbed by translation $\tau \leftarrow \tau + \Delta f_1 - \Delta f_0$.

The uncertainty we meant was the one that is induced to the target loss $L$ by $f_0,f_1$.
The pessimistic (worst-case) loss surface leads to a different solution from the original (non-pessimistic) loss only when the uncertainty (confidence interval in $L$) differs among the hypotheses (if not, it would be meaningless as $\arg \min_h \max_{c \in [-1,1]} L(h) + c = \arg \min_h L(h)$).
Similarly, the essential part of the loss would be like following.
Consider a scalar case and let $t = f_1 - f_0$ be the target. Then, 
$$\arg \min_\tau \max_{\Delta t \in [-1, 1]} (t + \Delta t - \tau)^2 = \arg \min_\tau (t - \tau)^2$$
due to the convexity and isotropy of $(t - \tau)^2$ on $\tau$.
It does not exactly reproduce the situation in our setting, but we do not consider disturbing $f_0$ and $f_1$ meaningful for essentially the same reason.

The other reason is that unlike propensity scores, where it makes sense (in the sense of equation (3)) to avoid extreme weights at the expense of accuracy for $p(t|x)$, there is no positive advantage to moving them significantly from the pre-trained $f_1, f_0$.
Please also refer to the response for Q3 of Reviewer S5uy.