Paper: SpaCE: The Spatial Confounding Environment (ICLR-2024)

Reviewer comment (Theory):
In equation 1, if R_s is the autocorrelation with neighbors, why do the authors consider only an additive model? What if Y = f(X, a, R)? Does the method fail in the cases of Y = f(X, a, R)?

Author response:
We are thankful for your positive remarks on the structure of the paper and your accurate summary. We have made improvements in Section 4 of the revision based on your comments, marked with red color.

Perhaps there is still some confusion about the counterfactual generation procedure. The process can be summarized in four steps: First, we learn $f$ that best predicts $Y_s$ from $(X_s, A_s)$ using AutoML. Second, we estimate the empirical additive errors $\hat{R}\_s = Y_s - f(X_s, A_s)$ and their joint (spatial) distribution $(\hat{R}\_s)\_{s\in\mathbb{S}} \sim P_R$. Third, we replace these endogenous empirical residuals with an independent similarly distributed exogenous noise $(R\_s)\_{s\in\mathbb{S}} \sim P_R$. Finally, we obtain counterfactuals by varying the treatment while holding constant the confounders and the exogenous noise. We added this summary in Section 4 for additional clarity.

We answer your questions directly.  As quick remarks,  $R_s$ is indeed the exogenous noise (Q3), and the abduction step is indeed what the reviewer suggests as the typical thing to do (Q2).  

The reviewer is concerned about the limitations of additive noise models (ANMs) (Q1, Q3). First, notice that while eq. (1) is additive, SpaCE's benchmark datasets can exhibit complex forms of non-additive noise due to the interactions with the missing confounder, which becomes part of the noise when masked.  The post-masking causal model can be written as $\tilde{Y}_s=f(X^\text{obs}_s, A_s, U_s)$ where the effect of the noise $U_s=(R_s, X_s^\text{miss})$ is likely not additive due to interactions of $X_s^\text{miss}$. Second, it is worth noticing that ANMs remain the focus and leading open challenge in the spatial confounding literature that our paper builds upon (c.f., [2,3,4,5]).  Are the reviewer's concerns motivated by the use and limitations of ANMs in causal discovery tasks [1, 6]? The spatial confounding literature we build upon mainly concerns causal effect estimation, not discovery. While there can be challenges for cause-effect estimation under non-additive noise models too, simple techniques complementing spatial confounding methods for ANMs have been used; for example,  with generalized linear models (e.g., Poisson regression) or outcome transformations (e.g., logarithms) for count-based or binary outcomes [5]. Thus, SpaCE's benchmark datasets are highly relevant to the current state of the art in spatial confounding adjustment. 

Regarding potential failures (Q1), the ANM will be a sensible model when the empirical residuals are approximately symmetric and continuously supported, as exemplified in Fig. 3c. We have documented the outcome transformations and stored histograms of the empirical and synthetic residuals with each environment in the Harvard Dataverse platform to ensure validity. We have also emphasized this in the revision.

*References*

- [1] Janzing & Steudel, 2010. “Justifying additive noise model-based causal discovery via algorithmic information theory”
- [2] Kamal et al. 2023. “Spatial Causality: A Systematic Review on Spatial Causal Inference”. *Geographic Analysis*.
- [3] Dupont et al. 2022. “Spatial+: a novel approach to spatial confounding”
- [4] Paciorek, 2010. “The importance of scale for spatial-confounding bias and precision of spatial regression estimators”
- [5] Urdangarin et al. 2023. “Evaluating recent methods to overcome spatial confounding”
- [6] Peters et al. 2004. "Causal Discovery with Continuous Additive Noise Models."

Reviewer comment (Writing):
Counterfactual inference/generation typically starts from the abduction step that infers the value or distribution of exogenous variables. However, in causal model (1), it is not clear which letter(s) denotes the exogenous variable(s).

Author response:
We are thankful for your positive remarks on the structure of the paper and your accurate summary. We have made improvements in Section 4 of the revision based on your comments, marked with red color.

Perhaps there is still some confusion about the counterfactual generation procedure. The process can be summarized in four steps: First, we learn $f$ that best predicts $Y_s$ from $(X_s, A_s)$ using AutoML. Second, we estimate the empirical additive errors $\hat{R}\_s = Y_s - f(X_s, A_s)$ and their joint (spatial) distribution $(\hat{R}\_s)\_{s\in\mathbb{S}} \sim P_R$. Third, we replace these endogenous empirical residuals with an independent similarly distributed exogenous noise $(R\_s)\_{s\in\mathbb{S}} \sim P_R$. Finally, we obtain counterfactuals by varying the treatment while holding constant the confounders and the exogenous noise. We added this summary in Section 4 for additional clarity.

We answer your questions directly.  As quick remarks,  $R_s$ is indeed the exogenous noise (Q3), and the abduction step is indeed what the reviewer suggests as the typical thing to do (Q2).  

The reviewer is concerned about the limitations of additive noise models (ANMs) (Q1, Q3). First, notice that while eq. (1) is additive, SpaCE's benchmark datasets can exhibit complex forms of non-additive noise due to the interactions with the missing confounder, which becomes part of the noise when masked.  The post-masking causal model can be written as $\tilde{Y}_s=f(X^\text{obs}_s, A_s, U_s)$ where the effect of the noise $U_s=(R_s, X_s^\text{miss})$ is likely not additive due to interactions of $X_s^\text{miss}$. Second, it is worth noticing that ANMs remain the focus and leading open challenge in the spatial confounding literature that our paper builds upon (c.f., [2,3,4,5]).  Are the reviewer's concerns motivated by the use and limitations of ANMs in causal discovery tasks [1, 6]? The spatial confounding literature we build upon mainly concerns causal effect estimation, not discovery. While there can be challenges for cause-effect estimation under non-additive noise models too, simple techniques complementing spatial confounding methods for ANMs have been used; for example,  with generalized linear models (e.g., Poisson regression) or outcome transformations (e.g., logarithms) for count-based or binary outcomes [5]. Thus, SpaCE's benchmark datasets are highly relevant to the current state of the art in spatial confounding adjustment. 

Regarding potential failures (Q1), the ANM will be a sensible model when the empirical residuals are approximately symmetric and continuously supported, as exemplified in Fig. 3c. We have documented the outcome transformations and stored histograms of the empirical and synthetic residuals with each environment in the Harvard Dataverse platform to ensure validity. We have also emphasized this in the revision.

*References*

- [1] Janzing & Steudel, 2010. “Justifying additive noise model-based causal discovery via algorithmic information theory”
- [2] Kamal et al. 2023. “Spatial Causality: A Systematic Review on Spatial Causal Inference”. *Geographic Analysis*.
- [3] Dupont et al. 2022. “Spatial+: a novel approach to spatial confounding”
- [4] Paciorek, 2010. “The importance of scale for spatial-confounding bias and precision of spatial regression estimators”
- [5] Urdangarin et al. 2023. “Evaluating recent methods to overcome spatial confounding”
- [6] Peters et al. 2004. "Causal Discovery with Continuous Additive Noise Models."

Reviewer comment (Theory):
What is the relationship between the residual R and the exogenous variable in the generation of Y? If R is the exogenous variable for Y, the authors need to consider a more general causal mechanism, rather than just the additive noise model (ANM). If R is not the exogenous variable, why can the authors use equation 1 for counterfactual inference?

Author response:
We are thankful for your positive remarks on the structure of the paper and your accurate summary. We have made improvements in Section 4 of the revision based on your comments, marked with red color.

Perhaps there is still some confusion about the counterfactual generation procedure. The process can be summarized in four steps: First, we learn $f$ that best predicts $Y_s$ from $(X_s, A_s)$ using AutoML. Second, we estimate the empirical additive errors $\hat{R}\_s = Y_s - f(X_s, A_s)$ and their joint (spatial) distribution $(\hat{R}\_s)\_{s\in\mathbb{S}} \sim P_R$. Third, we replace these endogenous empirical residuals with an independent similarly distributed exogenous noise $(R\_s)\_{s\in\mathbb{S}} \sim P_R$. Finally, we obtain counterfactuals by varying the treatment while holding constant the confounders and the exogenous noise. We added this summary in Section 4 for additional clarity.

We answer your questions directly.  As quick remarks,  $R_s$ is indeed the exogenous noise (Q3), and the abduction step is indeed what the reviewer suggests as the typical thing to do (Q2).  

The reviewer is concerned about the limitations of additive noise models (ANMs) (Q1, Q3). First, notice that while eq. (1) is additive, SpaCE's benchmark datasets can exhibit complex forms of non-additive noise due to the interactions with the missing confounder, which becomes part of the noise when masked.  The post-masking causal model can be written as $\tilde{Y}_s=f(X^\text{obs}_s, A_s, U_s)$ where the effect of the noise $U_s=(R_s, X_s^\text{miss})$ is likely not additive due to interactions of $X_s^\text{miss}$. Second, it is worth noticing that ANMs remain the focus and leading open challenge in the spatial confounding literature that our paper builds upon (c.f., [2,3,4,5]).  Are the reviewer's concerns motivated by the use and limitations of ANMs in causal discovery tasks [1, 6]? The spatial confounding literature we build upon mainly concerns causal effect estimation, not discovery. While there can be challenges for cause-effect estimation under non-additive noise models too, simple techniques complementing spatial confounding methods for ANMs have been used; for example,  with generalized linear models (e.g., Poisson regression) or outcome transformations (e.g., logarithms) for count-based or binary outcomes [5]. Thus, SpaCE's benchmark datasets are highly relevant to the current state of the art in spatial confounding adjustment. 

Regarding potential failures (Q1), the ANM will be a sensible model when the empirical residuals are approximately symmetric and continuously supported, as exemplified in Fig. 3c. We have documented the outcome transformations and stored histograms of the empirical and synthetic residuals with each environment in the Harvard Dataverse platform to ensure validity. We have also emphasized this in the revision.

*References*

- [1] Janzing & Steudel, 2010. “Justifying additive noise model-based causal discovery via algorithmic information theory”
- [2] Kamal et al. 2023. “Spatial Causality: A Systematic Review on Spatial Causal Inference”. *Geographic Analysis*.
- [3] Dupont et al. 2022. “Spatial+: a novel approach to spatial confounding”
- [4] Paciorek, 2010. “The importance of scale for spatial-confounding bias and precision of spatial regression estimators”
- [5] Urdangarin et al. 2023. “Evaluating recent methods to overcome spatial confounding”
- [6] Peters et al. 2004. "Causal Discovery with Continuous Additive Noise Models."