Paper: Causality is Invariance Across Heterogeneous Units (ICLR-2024)

Reviewer comment (Writing):
The abstract could helpfully be rewritten to indicate just what is being proposed in this paper. It’s a bit unfocused.

Author response:
**Abstract Revision:** We have rewritten the abstract to better focus on the core proposals of our paper, providing a clearer entry point into our research.

Reviewer comment (Novelty):
There is some history in the literature for identifying invariant causal modules as a way of understanding what causality means, especially for large causal systems and in the context of neural networks interpreted causally. This literature should be identified and reviewed, and the current proposal placed in that context in a precise way. A Google Scholar search should identify relevant papers.

Author response:
**Invariance in Causal Modules Literature:** Regarding the literature on invariant causal modules, we have reviewed relevant papers to situate our work within this context. However, we respectfully note that our approach, while acknowledging the broader literature, introduces a unique perspective, particularly in terms of utilizing neural network structures  based on DiscoSCM for counterfactual predictions.

Reviewer comment (Novelty):
The idea of estimating counterfactual predictions is also not new. Again, the literature on that enterprise should be reviewed, and the current proposal situated in this larger literature.

Author response:
**Counterfactual Predictions Literature:** We respectfully disagree with the suggestion that our approach is not novel. While estimating counterfactual predictions is indeed not a new concept, our method, predicated on the DiscoSCM framework, provides a distinct and practical approach to counterfactual estimation in Layer 3. We have thoroughly reviewed and cited the relevant literature, including Pearl's work, to position our contribution within this larger context.

Reviewer comment (Novelty):
On p. 3, the proposal of representing discrete models using degenerate Gaussian distributions is entertained. A paper from several years ago elaborates this idea in the context of CSL: Andrews et al. Learning high-dimensional directed acyclic graphs with mixed data-types. In The 2019 ACM SIGKDD Workshop on Causal Discovery (pp. 4-21). PMLR. Perhaps this and other such papers should be cited here.

Author response:
**Representing Discrete Models:** Concerning the representation of discrete models using degenerate Gaussian distributions, we have reviewed the suggested paper by Andrews et al. and others. However, our approach diverges significantly in methodology and application, as outlined in our paper.

Reviewer comment (Presentation):
I agree that an empirical example was required for this paper, though not including any results from these experiments in the main text, I believe, was an unforced error for this conference.

Author response:
**Empirical Example and Results:** We understand your concern regarding the absence of empirical results in the main text. We will consider integrating key findings from the supplemental material into the main text in our revised manuscript.

Reviewer comment (Presentation):
For someone who doesn't read the supplement, what conclusions are we to draw from the empirical investigations? No actual results are given in the main text.

Author response:
**Empirical Example and Results:** We understand your concern regarding the absence of empirical results in the main text. We will consider integrating key findings from the supplemental material into the main text in our revised manuscript.

Reviewer comment (Novelty):
The way of modeling unit heterogeneity here is interesting but the ultimate contribution to the conceptual side may not be too big. It is possible to model the same thing within the SCM framework simply by adding more exogenous variables to the main one {U} that represents the choice of individual. For instance new independent variables U' for the different units would allow modeling the same effects as achieved in the Disco framework.

Author response:
1. **Necessity of Decoupling Individual Semantics**: A key aspect of our model is the decoupling of individual semantics from the exogenous variable $U = u$. This approach is fundamental for the practical implementation of our individual/population-level counterfactual algorithms. Without this decoupling, we would be compelled to rely on Pearl's algorithms, necessitating complete knowledge of structural equations and the practical computation of $P(u|e)$, rendering the process intractable. This intractability has been a major hindrance in current literature for practically computing counterfactuals. Our model represents the first practical and reasonable approach to Layer 3 valuations, to the best of our knowledge, even allowing for heterogeneous counterfactual estimation.

Reviewer comment (Theory):
The running example does not seem to make sense. It says a user is observed with a high subsidy and high retention, and asks "were they given a low subsidy, what would be their retention?" It's claimed that consistency implies the retention would also be high, but consistency implies nothing about this scenario, since the intervention of giving a low subsidy is not the same as what was observed (a high subsidy). In the referenced paper (Gong 2023) the example is stated correctly, as "were they again given a high subsidy, what would be their retention?"

Author response:
2. **Response to Specific Questions**:
   - **Running Example and Grammatical Issues**: We are grateful for your observation regarding the issues in our running example and grammar. These have been corrected in the revised version of our manuscript.

Reviewer comment (Writing):
Abstract: "considers is one of his" -> "considers as one of his"

Author response:
2. **Response to Specific Questions**:
   - **Running Example and Grammatical Issues**: We are grateful for your observation regarding the issues in our running example and grammar. These have been corrected in the revised version of our manuscript.

Reviewer comment (Writing):
Section 5.1: "when lack complete knowledge" -> "when lacking complete knowledge"

Author response:
2. **Response to Specific Questions**:
   - **Running Example and Grammatical Issues**: We are grateful for your observation regarding the issues in our running example and grammar. These have been corrected in the revised version of our manuscript.

Reviewer comment (Writing):
"reducible pocket" on pg. 9: I don't understand the metaphor, why is that like a pocket?

Author response:
2. **Response to Specific Questions**:
   - **"Reducible Pocket" Metaphor**: The term "reducible pocket" is indeed a concept borrowed from Stephen Wolfram, who claimed finding pockets of reducibility is the story of science. We use this metaphor to describe the predictive capabilities of our DiscoModel on modeling complex underlying causal mechanisms.

Reviewer comment (Writing):
The model assumption is that $Y$ is independent of $X$ given $T$ and some unobserved $U$. The main idea of the method is to learn a representation for $U$ (denoted as Z) and then use $(Z, T)$ to predict $Y$. The idea is simple but the description in Section 2.1 only makes the method sound complicated.

Author response:
**Clarification of Model Assumptions**: We appreciate your concern about the complexity of our model's assumptions. However, there seems to be a misunderstanding in your interpretation. Our model's foundational assumption is that the outcome $Y_i$ is independent of the treatment $X_i$ given the treatment assignment $T_i$ and the causal representation $z_i$ for any individual $i$. This "unit causal representation assumption" is now explicitly stated in Section 2.1 of our revised manuscript. It is crucial for overcoming computational difficulties in calculating $P(i|e)$. The problem of calculating the probability of selecting a particular user from billions faces numerical overflow challenges, which are alleviated by computing the probability density $P(z_i|e)$ of a high-dimensional Gaussian distribution sample. This assumption is vital and commonly employed in various fields, akin to user embedding in recommendation systems [Bonner, Vasile 2018] or token embedding in language models, from a computational aspect.

Reviewer comment (Theory):
The causal representation $z_{S(e)}$ is not formally defined in the population case. How is it related to Equations (2)-(4)? This should be explained in detail.

Author response:
**Randomness of Causal Representation Variable**: In our model, we denote the posterior of the unit selection variable $S$ as $P(\cdot|e)$, and its corresponding variable as $S(e)$. The randomness in the causal representation variable $z_{S(e)}$ arises from the selection of different units $i$. The DiscoModel's unique approach decouples individual semantics from the exogenous variable $U = u$, using the unit selection variable $S$ for individual-level computations. This enables our model to practically answer Layer 3 queries. We provide a more detailed explanation in Section 2.1 and the appendix of our revised version, comparing to the role of $U = u$ in determining both the selection of individual $i$ and the exogenous uncertainty in SCM.

Reviewer comment (Theory):
There are no identifiability results regarding the causal representations $z_{S(e)}$.

Author response:
**Identifiability of Causal Representations**: We argue that prioritizing identifiability results regarding causal representations is not the primary concern in causal research. Instead, the focus is typically on the identification of causal estimands, often involving the conversion of causal quantities (e.g.  $P(y|do(x), c)$) into expressions that only contain conditional probabilities. The consistency assumption in traditional models bridges potential outcomes with conditional probabilities learnable from data, aiding in identification. Our model's distributional-consistency assumption and independent potential noise assumption serve a similar role, ensuring robust identification.