Paper: A Neural Framework for Generalized Causal Sensitivity Analysis (ICLR-2024)

Reviewer comment (Theory):
It is not very obvious how does the bounds the proposed framework compares with some existing works such as GMSM.

Author response:
The GMSM (proposed by [1]) extends the standard MSM/ CMSM in two major ways: (i) It allows for _mediators_, thus enabling sensitivity analysis for direct/indirect/path-specific effects, and (ii) it allows for the incorporation of _weights_ into the sensitivity model that represent prior knowledge about the confounding structure. Point (i) is not relevant in our setting as we do not consider mediators. Extending NeuralCSA to settings with mediators may be an interesting direction for future work. For point (ii), we kindly refer you to Appendix C: Therein, **we show that the weighted GMSM can be reformulated as a GTSM, which makes NeuralCSA applicable**. For example, NeuralCSA could be used to perform sensitivity analysis for multiple outcomes under a weighted GMSM, which was previously not possible using the bounds in [1]. We would like to emphasize that we do not claim that NeuralCSA archives better performance than existing closed-form solutions (e.g., bounds for the GMSM in single-outcome settings). Rather, we propose our NeuralCSA as a method for causal sensitivity analysis whenever bounds are not analytically tractable (see Table 1). Furthermore, note that the bounds in [1] are not applicable to sensitivity models beyond MSM-type (e.g., $f$-sensitivity models, Rosenbaum’s sensitivity model), yet for which our NeuralCSA is applicable.

Reviewer comment (Writing):
Regarding the color coding in equation (4), does it indicate the parameters for the optimization problem? Does the right supremum also maximize over $P(U|x,a)$?

Author response:
In Eq. (4), the coloring highlights the components of the joint probability distribution that appear in the causal query, i.e., the optimization objective. The optimization happens over the components that appear just below the suprema, that is, both $\mathbb{P}(U \mid x, a)$ and  ${\mathbb{P}(Y \mid x, u, a)\}$ for the right supremum, and $\{\mathbb{P}(U \mid x, a^\prime)\}_{a^\prime \neq a}$ for the left supremum.

    **Action:** We added a clarification below Eq. (4) where we explain the choice behind the coloring.

Reviewer comment (Theory):
I'm not fully understand the two-stage procedure. Could you to provide a more detailed explanation about replace the right supremum with fixed of $\mathbb{P}^*(U|x,a)$ and $\mathbb{P}^*(Y|x,u,a)$?

Author response:
Two-stage procedure: Theorem 1 has two major implications: (1) It is sufficient to fix the distributions $\mathbb{P}^\ast(U \mid x, a)$ and $\mathbb{P}^\ast(Y \mid x, u, a)$; and (2) it is sufficient to choose $\mathbb{P}^\ast(Y \mid x, u, a) = \delta(Y - f^\ast_{x, a}(u))$ as a delta-distribution induced by an invertible function $f^\ast_{x, a} \colon \mathcal{U} \to \mathcal{Y}$, which satisfies the data-compatibility constraint $\mathbb{P}(Y \mid x, a) =  \mathbb{P}^\ast(f^\ast_{x, a}(U) \mid x, a)$. Point (1) implies that we can fix the components in the right supremum of Eq.~(4) and only optimize over the left supremum. Point (2) implies that the causal query $\mathcal{F} \left(\int \mathbb{P}(Y \mid x, u, a) \mathbb{P}(u \mid x) du\right)$ reduces to $\mathcal{F} \left(\mathbb{P}(f^\ast_{x, a}(U \mid x) \right)$. Eq.(5) shows the corresponding Stage 2 optimization for discrete treatments (maximizing over $\mathbb{P}(u \mid x, A \neq a)$ to ensure data compatibility). For continuous treatments, we can directly maximize over $\mathbb{P}(u \mid x)$.

    **Action:** We added a clarification to Sec. 5.2. Please let us know if you have further questions/suggestions regarding the clarification of our two-stage procedure.

Reviewer comment (Writing):
I found Sec 5.1 and 5.2 difficult to read and I think clarity can be improved. What confused me initially was that you suggest fixing $P^*(U|x, a)$ but then the $\sup$ in Eq. 5 is also over the distributions $p(u|x, A)$. Reading it further, the sup is only for $A \neq a$ but I think clarifying that you only fix for the treatment $a$ that enters into $Q$ would be useful. Maybe this is obvious, but it will still make it easier to understand what is being optimized over in the $\sup$.

Author response:
* Thank you for pointing out the ambiguous notation in Eq. (5). You are correct in that we optimize the objective in Eq. (5) over the distribution $\mathbb{P}(u \mid x, A \neq a)$ for fixed $a$, and not directly over $\mathbb{P}(u \mid x)$. The variable $a$ is indeed the fixed treatment intervention that enters the causal query $Q$. Framing the optimization problem in this way ensures that the corresponding joint distribution $\mathbb{P}(u, a \mid x)$ induces the observational distribution $\mathbb{P}_\mathrm{obs}(a \mid x)$, which guarantees sharpness of our bounds without the need to incorporate additional constraints into our learning algorithm. If we would also optimize over distributions $\mathbb{P}(u \mid x)$ that are not compatible with the observational distribution, our bounds may become overly conservative.

    **Action:** We added a clarification to Eq. (5) in order to improve the clarity of Sec. 5.1.

Reviewer comment (Theory):
It would also be nice to have some intuition of the proof of Theorem 1. Also, the invertible function $f^*$ would depend on the fixed $P^*$. Does certain distributions $P^*$ make it easier to determine $f^*$. In practice, how should you determine which $P^*$ to fix?

Author response:
* **Intuition for Theorem 1:** We agree that Sec. 5.1 and 5.2 can be improved by providing further intuition for Theorem 1. We believe that the following two aspects of Theorem 1 may need further clarification (for a rigorous proof, we refer to Appendix B.3):
    * _Why is it sufficient to only consider invertible functions $f: \mathcal{U} \to \mathcal{Y}$, i.e., model $\mathbb{P}^\ast(Y \mid x, a, u) = \delta(Y - f(u))$  as a Dirac delta distribution?_ Here, it is helpful to have a closer look at Fig. 1: Intervening on the treatment $A$ causes a shift in the latent distribution of $U$, which then leads to a shifted interventional distribution $\mathbb{P}(Y(a) = y \mid x)$. The question is now: How can we obtain an interventional distribution that results in a maximal causal query (for the upper bound)? Let us consider a structural equation of the form $Y = g(U, \epsilon)$, where $\epsilon$ is some independent noise. Hence, the “randomness” (entropy) in $Y$ comes from both $U$ and $\epsilon$; however, the distribution shift only arises through $U$. Intuitively, the interventional distribution should be maximally shifted if $Y$ only depends on the unobserved confounder and not on independent noise, i.e., $g(U, \epsilon) = f(U)$ for some invertible function $f$. One may also think about this as achieving the maximal “dependence” (mutual information) between the random variables $U$ and $Y$. Note that any GTSM only restricts the dependence between $U$ and $A$, but not between $U$ and $Y$.
    * _Why can we fix $\mathbb{P}^\ast(U \mid x, a)$ and $f$ without losing the ability to achieve the optimum in Eq. (4)?_ The basic idea is as follows: Let $\widetilde{\mathbb{P}}(\widetilde{U} \mid x, a)$ and $\widetilde{f}$ be optimal solutions to Eq. (4) for a potentially different latent variable $\widetilde{U}$. Then, we can define a mapping $t = {f}^{-1} \circ \widetilde{f} \colon \widetilde{U} \to U$ between latent spaces that transforms $\widetilde{\mathbb{P}}(\widetilde{U} \mid x, a)$ into our fixed $\mathbb{P}^\ast(U \mid x, a)$ (because both $\widetilde{f}$ and $f$ respect the observational distribution). Furthermore, we can use $t$ to push the optimal shifted distribution $\widetilde{\mathbb{P}}(\widetilde{U} \mid x)$ (under treatment intervention) to the latent variable $U$ (see Eq. (46)). In our proof in Appendix B.3. we show that this is sufficient to obtain a distribution  $\mathbb{P}^\ast$ that induces $\mathbb{P}^\ast(U \mid x, a)$ and $\mathbb{P}^\ast(Y \mid x, u, a) = \delta(Y - f(u))$ and which satisfies the sensitivity constraints. For the latter property, we require the sensitivity model to be “invariant” with respect to the transformation $t$. We formalize this via our transformation-invariance assumption (Definition 3).

    **Choice of the fixed distribution $\mathbb{P}^\ast(U \mid x, a)$ in Stage 1:** You are correct in that the function $f^\ast_{x, a}$ depends on the choice of $\mathbb{P}^\ast(U \mid x, a)$ (we sightly abused notation here for improved readability). In practice, Stage 1 reduces to fitting a (conditional) normalizing flow (NF) to learn the observational distribution $\mathbb{P}_\mathrm{obs}(Y \mid x, a)$. **Hence, the problem of choosing  $\mathbb{P}^\ast(U \mid x, a)$ in Stage 1 of NeuralCSA is equivalent to choosing the latent distribution when fitting an NF**. Common choices from the literature are the standard Gaussian or uniform distribution [1]. In our implementation, we chose a standard Gaussian for the following reason: Stage 2 requires fitting a second NF to model a distribution in the latent space with the same support as $\mathbb{P}^\ast(U \mid x, a)$. By using a Gaussian, we avoid the need to hard-code support constraints into our Stage 2 NF. 


    **Action:** We extended our intuition paragraph in Sec. 5.2. and included some of the arguments provided above. Furthermore, we added an extensive discussion to Sec.B.3 in the appendix (see our proof of Theorem 1).



## References

[1] Papamakarios et al. (2021). “Normalizing Flows for Probabilistic Modeling and Inference”. JMLR