Teleporter Theory: A General and Simple Approach for Modeling Cross-World Counterfactual Causality

## Abstract

Abstract Leveraging the development of structural causal model (SCM), researchers can establish graphical models for exploring the causal mechanisms behind machine learning techniques. As the complexity of machine learning applications rises, single-world interventionism causal analysis encounters theoretical adaptation limitations. Accordingly, cross-world counterfactual approach extends our understanding of causality beyond observed data, enabling hypothetical reasoning about alternative scenarios. However, the joint involvement of cross-world variables, encompassing counterfactual variables and real-world variables, challenges the construction of the graphical model. Twin network is a subtle attempt, establishing a symbiotic relationship, to bridge the gap between graphical modeling and the introduction of counterfactuals albeit with room for improvement in generalization. In this regard, we demonstrate the theoretical breakdowns of twin networks in certain cross-world counterfactual scenarios. To this end, we propose a novel teleporter theory to establish a general and simple graphical representation of counterfactuals, which provides criteria for determining teleporter variables to connect multiple worlds. In theoretical application, we determine that introducing the proposed teleporter theory can directly obtain the conditional independence between counterfactual variables and real-world variables from the cross-world SCM without requiring complex algebraic derivations. Accordingly, we can further identify counterfactual causal effects through cross-world symbolic derivation. We demonstrate the generality of the teleporter theory to the practical application. Adhering to the proposed theory, we build a plug-and-play module, and the effectiveness of which are substantiated by experiments on benchmarks.

## 1 Introduction

Causal inference is a specialized field that presents promising potential with respect to improve machine learning methods, conventionally encompassing four steps: 1) causal model construction for modeling causality in machine learning applications in a qualitative analysis manner (; ; ); 2) causal model validation, including independence and causality testing, to demonstrate the correctness of the causal model construction (; ; ); 3) causal model-based deconfounding approach implementation, which prospers in various machine learning fields, e.g., eliminating spurious correlations (; ) and performing counterfactual reasoning () in computer vision, learning the intrinsic rationale of the graph (; ) in graph neural networks, overcoming selection bias () and popularity bias () in the recommendation systems; 4) deconfounding approach estimation improvement, focusing on enhancing the accuracy of causal model-based deconfounding (; ). Benefiting from the establishment of graphical models, the advances of the structural causal model (SCM) concentrate greater potential onto exploring the causal mechanisms behind machine learning techniques, e.g., the analysis of independent relationships among variables and the identification of causal effects for various machine learning applications.

In practice, the involvement of derived discrete data with extra stringent structural constraints increases the complexity of machine learning application scenarios, resulting in a lack of adaptability of conventional interventionism causal analysis theories, i.e., single-world SCM-based theory (; ; ). cross-world counterfactuals (; ; ; ) provide a framework to estimate “what-if” scenarios that transcend the observed world, aiding in better-informed causal inference, which is crucial for understanding causal relationships in a more comprehensive manner, as it enables the exploration of causality under different hypothetical conditions (; ; ; ; ). A focal issue is the exclusivity of counterfactual variables and real-world variables in an invariant graphical model, challenging the construction of cross-world counterfactual SCMs. In this regard, Twin networks (; ; ; ; ) demonstrate a symbiotic relationship of graphical modeling in counterfactual and real-world scenarios. Yet, in this paper, we provide multiple scenarios of cross-world counterfactual causal analysis, where the applications of twin networks breakdown, detailed in Section 4.1.

To this end, we propose the teleporter theory to establish a complete graphical representation of counterfactuals, providing a general and simple approach for modeling cross-world counterfactual causality. Concretely, according to the framework of probabilistic causal models, each variable can ultimately trace its changes back to the exogenous nodes that influence it by iteratively applying the structural equations of its parent nodes over a finite number of steps. Variables that have consistent structural equations in both the real world and the counterfactual world possess equivalence, which is determined as a teleporter. Accordingly, we provide sufficient causal analysis from the structural equation perspective, substantiating the theoretical correctness of the proposed teleporter theory. In terms of theoretical applications, we focus on two main aspects: 1) we apply $d$-separation to test the conditional independence between any two cross-world variables of a cross-world SCM constructed by introducing the teleporter theory, which can prove the correctness and generalization of our theory; 2) we use the teleporter theory to build the cross-world SCM and further leverage the cross-world symbolic derivation to compute counterfactual probability, which can avoid the complex calculation of the probability distribution of background variables, demonstrating the effectiveness and simplicity of our theory. Adhering to the proposed theory, we build a practical plug-and-play module to address the intrinsic issue in the field of Graph Out-Of-Distribution (GraphOOD) (; ; ), and the consistency and effectiveness of the proposed module are substantiated by experiments on benchmarks.

Our contributions are as follows:
(1) We provide multiple motivating examples to elucidate the incompleteness of twin networks in certain cross-world counterfactual scenarios with sufficient causal analysis.
(2) We propose a general and simple approach for modeling cross-world counterfactual causality, namely the teleporter theory, which is proved as a complete causal analysis method.
(3) We provide sufficient evidence to prove the theoretical correctness of the proposed teleporter theory by introducing the structural equation analysis.
(4) We conduct extensive validations on commonly adopted benchmarks, demonstrating the generalized applicability of the teleporter theory from theoretical and practical perspectives.

## 2 Related Work

#### Modeling Single-World Causality.

Single-world interventions pertain to the first two levels of Pearl’s causal hierarchy (; ): association and intervention. Once we can model causality using observational data (; ), various methods exist for estimating interventional distributions (; ; ; ), provided identifiability is ensured (). The implementation of interventions transcends simple modeling of data associations, aiming instead to answer scientific questions such as “How effective is $X$ in influencing $Y$?” and thus achieving estimates of causal effects. Numerous works in the machine learning community have benefited from this approach: (1) real user preference in recommendation systems, such as deconfounding (; ; ) and disentangling (), (2) rationale representations in graph neural networks, such as robustness and invariant subgraphs (; ), and (3) invariant representations in domain generalization, such as eliminating spurious correlations (; ).

#### Modeling Cross-World Counterfactual Causality.

Cross-world causality aims to address the top-level query of Pearl’s causal hierarchy (; ): counterfactuals. However, estimating counterfactual causality faces the challenge of conflicts between real-world variable values and counterfactual variable values, making identifiability (; ; ; ) more scarce compared to interventions. Despite this, answering counterfactual queries like “why?” and “what if?” using causal framework can enable personalized and interpretable decision-making and reasoning. This significantly advances several key areas: 1) application in computer vision, e.g., alleviating data scarcity through data augmentation (; ); 2) fairness in legal and policy-making contexts (; ); 3) interpretability in the field of medical health (; ), among others.

## 3 Preliminary

We recap the necessary preliminaries of causal background knowledge relevant to our work. For a more in-depth understanding, please refer to the literature (; ; ).

#### Structural Causal Models.

A SCM (; ) is a causal model in a triple form, i.e., $M=\left\langle X,U,F\right\rangle$, where $U$ presents the exogenous variable set, determined by external factors of the model. $X=\left\{X_{1},X_{2},...,X_{n}\right\}$ presents the endogenous variable set, determined by the internal functions $F=\left\{f_{1},f_{2},...,f_{n}\right\}$. Each $f_{i}$ represents $\left\{f_{i}:U_{i}\cup PA_{i}\rightarrow X_{i}\right\}$, where $U_{i}\subseteq U$, $PA_{i}\subseteq X\backslash X_{i}$, satisfying:

$$ $x_{i}=f_{i}\left(pa_{i},u_{i}\right),\quad i=1,2,...,n.$ (1) $$

$PA_{i}$ denotes the parent nodes of $X_{i}$. Note that, in SCM, uppercase letters conventionally denote variables, and lowercase letters conventionally denote values of the corresponding variables, e.g., $x_{i}$ is the value of $X_{i}$. For ease of discussion, we omit such clarification in the following sections. Each causal model $M$ corresponds to a directed acyclic graph $\mathcal{G}$, where each node corresponds to a variable in $X\cup U$, and directed edges point from $U_{i}\cup PA_{i}$ to $X_{i}$. It is worth noting that exogenous variables $U$ have no ancestor nodes, and each endogenous variable $X_{i}$ is at least a descendant of one exogenous variable.

#### Interventions and Do-calculus.

The causal model $M$ describes intrinsic causal mechanisms, characterized by the observed distribution $P_{M}(X)=\prod_{i=1}^{n}P(x_{i}\mid pa_{i})$. Intervention(^1^11The definition here refers to the atomic intervention (). For brevity, we intervene on only one variable.) is defined as forcing a variable $X_{i}$ to take on a fixed value $x$, modifying the model $M=\left\langle X,U,F\right\rangle$ to $M_{x}=\left\langle X,U,F_{x}\right\rangle$, where $F_{x}=\left\{F\backslash f_{i}\right\}\cup\left\{X_{i}=x\right\}$. This is equivalent to removing $X_{i}$ from its original functional mechanism $x_{i}=f_{i}\left(pa_{i},u_{i}\right)$ and modifying this function to a constant function $X_{i}=x$. Formally, we denote the intervention as $do(x_{i}=x)$, called the $do$-$calculus$. It explores how causal mechanisms will change when external interventions, or experiments, are introduced. We denote the distribution after the intervention as $P_{M_{x}}(X)=P(x_{1},...,x_{n}\mid do(x_{i}=x))$, where

$$ $P(x_{1},...,x_{n}\mid do(x_{i}=x))=\begin{cases}\prod_{j\neq i}P(x_{j}\mid pa_ {j})&x_{i}=x\\ 0&x_{i}\neq x\end{cases}.$ (2) $$

#### Counterfactuals.

If $M_{x}$ defines the effect of the action $do(X=x)$ on $M$, what is the potential change of another endogenous variable $Y$ due to the intervention effect $M_{x}$? We denote $M_{x}$ as the SCM of the counterfactual world () derived by adopting the intervention $X=x$. The potential value of $Y$ influenced by the intervention $do(X=x)$ is denoted as $Y_{x}(u)$, which is a solution to the equation set $F_{x}$, i.e., $Y_{x}(u)=Y_{M_{x}}(u)$. Concretely, $Y_{x}(u)$ presents the counterfactual statement “Under condition $u$, if $X$ were $x$, then $Y$ would be $Y_{x}(u)$.”

#### Path and d 𝑑 d italic_d -separation.

We recap two classic definitions () to help us determine the independence between variables in the SCM graph.

###### Definition 1 .

(Path)
In the SCM graph, the paths from variable $X$ to $Y$ include three types of structures: 1) chain structure: $A\rightarrow B\rightarrow C$ or $A\leftarrow B\leftarrow C$; 2) fork structure: $A\leftarrow B\rightarrow C$; 3) collider structure: $A\rightarrow B\leftarrow C$.

###### Definition 2 .

($d$-separation)
A path $p$ is blocked by a set of nodes $Z$ if and only if:

- 1.
$p$ contains a chain of nodes $A\rightarrow B\rightarrow C$ or a fork $A\leftarrow B\rightarrow C$ such that the middle node $B$ is in $Z$, i.e., $A$ and $C$ are independent conditional on $B$, or
- 2.
$p$ contains a collider $A\rightarrow B\leftarrow C$ such that the collider node $B$ is not in $Z$, and no descendant of $B$ is in $Z$, i.e., $A$ and $C$ are independent without conditions.

If $Z$ blocks every path between two nodes $X$ and $Y$, then $X$ and $Y$ are $d$-separated, conditional on $Z$, i.e., $X$ and $Y$ are independent conditional on $Z$, denoted as $X\upmodels Y\mid Z$.

Figure: Figure 1: Example for breakdown of twin network: Figure (a) represents the real-world SCM, Figure (b) shows the cross-world SCM constructed using the twin network, and Figure (c) illustrates the cross-world SCM constructed using the teleporter theory.
Refer to caption: extracted/5674707/2.png

###### Definition 1 .

###### Definition 2 .

## 4 Teleporter Theory

In this section, we provide a detailed analysis to show the breakdown of twin networks, and further propose a novel theory in cross-world counterfactual causality modeling, which demonstrates sufficient theoretical completeness and generalization.

### 4.1 Theoretical Breakdown of Twin Networks

Twin network () is formed to model cross-world counterfactual causality by connecting the real world and counterfactual world, sharing exogenous variables between them. The constructed sub-networks of real world and counterfactual world are structurally identical, except that the arrows pointing to the intervened variable are removed in the counterfactual sub-network. The specific construction steps are as follows: 1) duplicating the endogenous variables $X=\left\{X_{1},X_{2},...,X_{n}\right\}$ from the real world as endogenous variables $X^{*}=\left\{X_{1}^{*},X_{2}^{*},...,X_{n}^{*}\right\}$ in the counterfactual world; 2) selecting the intervened variable $X_{i}$ and removing all arrows pointing to the counterfactual variable $X_{i}^{*}$; 3) connecting $X$ and $X^{*}$ through exogenous variables $U$ to form the twin network. Figure 1(b) illustrates an example of cross-world SCM constructed by using twin networks.

However, the benchmark twin networks encounter theoretical breakdowns in certain scenarios of modeling cross-world counterfactual causality. Concretely, we explore an example in “Causality” (), as depicted in Figures 1(a) and (b). We aim to test whether $D_{a}$ is independent of $A$ given $B$ or $C$, i.e., $A\upmodels D_{a}\mid B$ or $A\upmodels D_{a}\mid C$. The corresponding twin network of this example causal graph is illustrated in Figure 1(b). To assess the conditional independence between $A$ and $D_{a}$, we determine under which variables $A$ and $D_{a}$ are $d$-separated. Conditional on $C$, the path from $A$ to $D_{a}$, i.e., $A\leftarrow C\leftarrow U\rightarrow C_{a}\rightarrow B_{a}\rightarrow D_{a}$, is blocked by node $C$, and thus, $A\upmodels D_{a}\mid C$ holds. Yet, conditional on $B$, this path is $d$-connected, i.e., $A\nupmodels D_{a}\mid B$. We validate the conclusions obtained from the twin network using Theorem 4.3.1 from (). Both variables $B$ and $C$ satisfy the back-door criterion for $(A,D)$, indicating that for all values $a$ of $A$, given $B$ or $C$, the counterfactual $D_{a}$ is conditionally independent of $A$. The above analysis demonstrates that the twin network erroneously determines $A\nupmodels D_{a}\mid B$, proving that the twin network lacks theoretical completeness in specific cross-world SCMs.

### 4.2 Teleporter Theory for Modeling Cross-World Counterfactual Causality

To remedy the mentioned theoretical deficiency, we employ a probabilistic causal model framework () to expound the teleporter theory. As definitions in Section 3, for SCM $M=\left\langle X,U,F\right\rangle$, a probabilistic causal model is a tuple $\left\langle M,P(u)\right\rangle$, where $P(u)$ is the probability distribution over the set $U$. By the definition of structural equations $x_{i}=f_{i}\left(pa_{i},u_{i}\right)$, the value of an endogenous variable $X_{i}$ can be recursively represented by all possible values of exogenous variables, i.e., $(u_{1},u_{2},...,u_{n})$, meaning each endogenous variable is a function of $U$. For instance, for a certain endogenous variable $X_{i}\in X$, we have $P(X_{i}=x_{i})=\sum_{\left\{u\mid X_{i}(u)=x_{i}\right\}}P(u)$. We formalize the above assertion, and the value of $X_{i}$ can be represented by the following recursively defined function:

$$ $\displaystyle x_{i}$ $\displaystyle=f_{X_{i}}(pa_{i},u_{i})$ (3) $\displaystyle=f_{X_{i}}(f_{X_{i_{1}}}(pa_{i_{1}},u_{i_{1}}),f_{X_{i_{2}}}(pa_{ i_{2}},u_{i_{2}}),...,f_{X_{i_{k}}}(pa_{i_{k}},u_{i_{k}}),u_{i})$ (4) $\displaystyle=g_{X_{i}}(u_{1},u_{2},...,u_{n})$ (5) $$

where $pa_{i}=\left\{X_{i_{1}},X_{i_{2}},...,X_{i_{k}}\right\}\subset X$, and $g_{X_{i}}$ is a function determined solely by the exogenous variables $(u_{1},u_{2},...,u_{n})$ after a finite number of iterations.

Figure: Figure 2: Illustration of cross-world SCM construction using teleporter theory: Figure (a) represents the real world $\mathcal{W}_{r}$, Figure (b) depicts the counterfactual world $\mathcal{W}_{c}$, and Figure (c) shows the cross-world SCM $\mathcal{W}_{m}$ formed by connecting the variables in $\mathcal{W}_{r}$ and $\mathcal{W}_{c}$ through the teleporter $Z$.
Refer to caption: x1.png

Accordingly, we consider the structural equations of variables in the counterfactual world. Suppose the counterfactual world with the intervention $do(X_{i}=x^{\star})$, and the values of variables $X_{j}$ are determined as $x_{j}=f_{X_{j}}^{x^{\star}}(pa_{j},u_{j})=f_{X_{j}}(f_{X_{j_{1}}}^{x^{\star}}(
pa_{j_{1}},u_{j_{1}}),...,f_{X_{j_{k}}}^{x^{\star}}(pa_{j_{k}},u_{j_{k}}),u_{j})$. The value of $X_{i}$ is fixed as a constant, i.e., $x_{i}=f_{X_{i}}^{x^{\star}}(pa_{i},u_{i})=x^{\star}$. In the counterfactual world, intervening on variable $X_{i}$ means removing all arrows pointing to $X_{i}$. Thus, the values of $X_{j}$ that only have $U$ as the parent nodes are $x_{j}=f_{X_{j}}^{x^{\star}}(pa_{j},u_{j})=f_{X_{j}}(u_{j})$. Only the descendants set $W$ of the intervened variable $X_{i}$ is influenced by $x_{i}=x^{\star}$ according to its structural equation, while the set $Z$ that is $d$-separated from $X_{i}$ remains the same as in the original world.

For the convenience of counterfactual notation, we present our teleporter theory by introducing three endogenous variables as uppercase letters(^2^22Please refer to Section 3 for the definition of counterfactual notation.) $X,Y,...,Z$. Accordingly, we provide the detailed theoretical description of our theory.

###### Theorem 1 .

(Teleporter theory for modeling cross-world counterfactual causality)
Suppose we intervene on the endogenous variable $X$. Let $\mathcal{W}_{r}=\left\langle M,u\right\rangle$ denote the real world before the intervention, and $\mathcal{W}_{c}=\left\langle M_{x},u\right\rangle$ denote the counterfactual world, where $u$ denotes the value set of the corresponding exogenous variable set $U$, and $u$ is shared between $\mathcal{W}_{r}$ and $\mathcal{W}_{c}$. In the counterfactual world $\mathcal{W}_{c}$, the variable $Z\subset X\cup U$, that are d-separated from $X=x$, can be determined as the teleporter. $\mathcal{W}_{r}$ and $\mathcal{W}_{c}$ are connected through the teleporter $Z$, forming a cross-world SCM $\mathcal{W}_{m}$. The teleporter theory-based construction of $\mathcal{W}_{m}$ is required to adhere the following rules:

- •
Rule 1: All descendants of $X=x$ in $\mathcal{W}_{r}$, e.g., $D$, have the potential value influenced by the intervention $do(X=x)$ in $\mathcal{W}_{c}$, e.g., $D_{x}$.
- •
Rule 2: Exogenous variables $U_{Z}$ associated with teleporter $Z$ should be removed, while the remaining exogenous variables in $\mathcal{W}_{r}$ are retained. All teleporters need to be determined in $\mathcal{W}_{m}$, including exogenous variables in $U$ that are $d$-separated from $X=x$.

We illustrate the implementation process of teleporter theory by using a classic causal graph as an example. Figure 2(a) investigates the causal relationship between $X$ and $Y$, where $Z$ acts as a confounder (), and all exogenous variables are depicted in the graph, which is treated as the real-world SCM $\mathcal{W}_{r}$. Figure 2(b) represents the intervention $do(X=x)$ on $X$, where all arrows pointing to $X$ are removed, which is treated as the counterfactual world $\mathcal{W}_{c}$. We determine that the structural equation for $Y$ in $\mathcal{W}_{c}$, denoted as $f_{Y_{x}}(X=x,Z,u_{Y})$, is evidently different from the structural equation for $Y$ in $\mathcal{W}_{r}$, denoted as $f_{Y}(X,Z,u_{Y})$. Thus, the meanings and values of $Y$ are different in the two worlds. Therefore, according to Rule 1 of Theorem 1, the descendants of $x$ only consist of $Y$ in $\mathcal{W}_{c}$, denoted as $Y_{x}$.

To derive the cross-world SCM, connecting the real-world and counterfactual worlds, we introduce the teleporter theory. According to Rule 2 of Theorem 1, a teleporter refers to a variable with identical structural equations in $\mathcal{W}_{r}$ and $\mathcal{W}_{c}$. In $\mathcal{W}_{c}$, the only endogenous variable, $d$-separated from $X=x$, is $Z$, with its structural equation denoted as $f_{Z}(pa_{Z},u_{Z})$. $X$ is not the parent node to such variables, and thus they are not influenced by the intervention $do(X=x)$. By utilizing the common teleporter $Z$ shared between $\mathcal{W}_{r}$ and $\mathcal{W}_{c}$ as a connecting node, we obtain the cross-world SCM $\mathcal{W}_{m}$ depicted in Figure 2(c).

###### Theorem 1 .

## 5 Theoretical Application of Teleporter Theory

The SCM $M$ and its corresponding graph $\mathcal{G}$ facilitate the graphical representation of causal variables, enabling us to intuitively test the independence between variables ($d$-separation). This, in turn, allows us to explore the effects of interventions without conducting new experiments, e.g., back-door/front-door adjustments. However, counterfactual variables $Y_{x}$ and real-world variables $X$ cannot coexist in a single graph $\mathcal{G}$ due to involving cross-world considerations. Twin network (; ; ) is the first attempt to address this issue, yet such a method fails in certain scenarios, shows its theoretical incompleteness. In the following two subsections, we demonstrate that the teleporter theory can provide a complete graphical representation of counterfactuals.

### 5.1 Independence between Cross-World Variables

The cross-world independence between counterfactual variables and real-world variables is difficult to derive from the separated real-world and counterfactual SCMs or the corresponding structural equations. The significant advantage of the teleporter theory lies in the graphical representation of counterfactuals, enabling us to analyze the (conditional) independence between any pair of cross-world variables. Concretely, considering the breakdown of twin networks in Section 4.1, we propose to demonstrate the theoretical completeness and generalization of the proposed teleporter theory as follows. In the cross-world SCM $\mathcal{W}_{m}$ of Figure 1(c) obtained through the teleporter theory, we conclude that both $A\upmodels D_{a}\mid B$ and $A\upmodels D_{a}\mid C$ hold. This is because the path from $A$ to $D_{a}$, i.e., $A\leftarrow C\rightarrow B\rightarrow D_{a}$, is blocked by $B$ or $C$. Similarly, upon adding $D$ as a condition, new paths between $A$ and $D_{a}$ are opened through the collider node $D$. Therefore, conditional on $\left\{D,C\right\}$, $A$ and $D_{a}$ are not $d$-separated, satisfying $A\nupmodels D_{a}\mid\left\{D,C\right\}$. However, conditional on $\left\{D,B\right\}$, $A$ and $D_{a}$ are $d$-separated, satisfying $A\upmodels D_{a}\mid\left\{D,B\right\}$.

Thus, the proposed teleporter theory can widely empower the (conditional) independence testing between cross-world variables. To better demonstrate the implementation of independence testing in the cross-world SCM built by using the teleporter theory, we propose the following Theorem 2, summarizing the d-separation theorem for counterfactuals.

###### Theorem 2 .

($d$-separation for cross-world variables under the teleporter theory)
In the cross-world SCM $\mathcal{W}_{m}$ constructed by following the teleporter theory, the path $p$ between the real-world variable $X$ and the counterfactual variable $Y_{x}$ is $d$-separated by the node set $Z$ if and only if:

1. $p$ contains either a chain structure or a fork structure, with intermediate nodes in $Z$, or

2. $p$ contains a collider structure, with neither the intermediate node nor its descendants in $Z$.

The set $Z$ $d$-separates $X$ from $Y_{x}$ if and only if $Z$ blocks all paths from $X$ to $Y_{x}$.

Figure: Figure 3: Figure (a) represents the real world $\mathcal{W}_{r}$, Figure (b) shows the cross-world SCM $\mathcal{W}_{m}$ constructed using the twin network, and Figure (c) depicts the cross-world SCM $\mathcal{W}_{m}$ constructed using the teleporter theory.
Refer to caption: extracted/5674707/3.1.png

###### Theorem 2 .

### 5.2 Cross-World Adjustment

The joint distribution of counterfactual statements requires computation, storage, and utilization of the marginal probability of values of the exogenous variables, i.e., $P(u)$. For example, $P(Y_{x}=y,X=x^{\prime})=\sum_{\left\{u|Y_{x}(u)=y,X(u)=x^{\prime}\right\}}P(u)$. Classic works summarize three steps for estimating the counterfactual $P(Y_{x}\mid e)$ in (), where $e$ denote the observed variable values: 1) abduction: updating $P(u\mid e)$ using the fact $e$; 2) action: updating the SCM $M$ to $M_{x}$; 3) computing $P(Y_{x}\mid e)$ in the counterfactual world $\mathcal{W}_{c}=\left\langle M_{x},P(u\mid e)\right\rangle$. However, obtaining the distribution of exogenous variables $U$ is extremely difficult. The teleporter theory provides a simple method to compute $P(Y_{x}\mid e)$, facilitating cross-world adjustment.

We propose the counterfactual criterion to obtain the (conditional) independence of $X$ and $Y_{x}$:

###### Theorem 3 .

(Counterfactual criterion and cross-world adjustment)
The evidence $e$ represents the values of the variable $E$ in the real world $\mathcal{W}_{r}$. Given an observable variable set $Z$, if $E\cup Z$ causes that conditional on $E\cup Z$, $X$ and $Y_{x}$ are $d$-separated in the cross-world SCM $\mathcal{W}_{m}$, then the counterfactual $Y_{x}$ is conditionally independent of $X$, denoted as $X\upmodels Y_{x}\mid\left\{E,Z\right\}$. The cross-world adjustment formula can be derived as follows:

$$ $P(Y_{x}=y\mid E=e)=\sum_{z}P(Y=y\mid Z=z,X=x,E=e)P(Z=z\mid E=e).$ (6) $$

In comparison to the counterfactual interpretation of the back-door criterion in (), our theoretical approach is proved to be a generalized solution, since the former approach can only treat the back-door path-related scenarios, our approach can achieve cross-world adjustment for any pair of variables. Please refer to Appendix A for the corresponding proofs.

We will now present two examples to illustrate that the teleporter theory is more complete compared to twin networks, as the latter fails in multiple scenarios. The first example demonstrates cases where twin networks incorrectly identify the required variables for adjustment. The SCM of such an example is depicted in Figure 3(a). In the twin network of Figure 3(b), $X$ and $Y_{x}$ are $d$-connected by the path $X\leftarrow C\leftarrow U_{c}\rightarrow C_{x}\rightarrow Z_{x}\rightarrow T_{
x}\rightarrow Y_{x}$. If we acquire to compute $Y_{x}$, the variables for adjustment can only be $C$, since using $Z$ or $T$ for adjustment would open up a collider node, resulting in certain dependence relationships between the parent nodes. However, in the cross-world SCM $\mathcal{W}_{m}$ of Figure 3(c) modeled by using the teleporter theory, the path between $X$ and $Y_{x}$ is $X\leftarrow C\rightarrow Z\rightarrow T\rightarrow Y_{x}$. According to Theorem 3, we can perform the adjustment on any node in $\left\{C,Z,T\right\}$, which is consistent with the empirical conclusion in “Causality” ().

The second example, as illustrated in Figure 4(a), involves computing $P(Y_{x}\mid w)$ given the known evidence $w$. In the twin network of Figure 4(b), $X$ and $Y_{x}$ are connected only through one path: $X\rightarrow\underline{W}\leftarrow Z\leftarrow U_{z}\rightarrow Z_{x}
\rightarrow T_{x}\rightarrow Y_{x}$(^3^33W represents conditioning on W, i.e., W is given.). In this case, we can only perform the adjustment on $Z$, since the adjustment on $T$ would open up new paths, i.e., $Z\nupmodels U_{T}\mid T$. The cross-world SCM $\mathcal{W}_{m}$ constructed by using the teleporter theory is depicted in Figure 4(c), and according to Theorem 3, we can perform the adjustment on both $T$ and $Z$, which well fits the empirical conclusion in “Causality” (). The above theoretical application analysis sufficiently demonstrate the generalization and applicability of our teleporter theory.

Figure: Figure 4: Figure (a) represents the real world $\mathcal{W}_{r}$, Figure (b) shows the cross-world SCM $\mathcal{W}_{m}$ constructed using the twin network, and Figure (c) depicts the cross-world SCM $\mathcal{W}_{m}$ constructed using the teleporter theory. Grey nodes indicate conditioning on that variable
Refer to caption: extracted/5674707/4.png

###### Theorem 3 .

## 6 Practical Application of Teleporter Theory

To demonstrate the practical applicability of teleporter theory, we utilize our theory to model the set of variables in GraphOOD (; ; ), and propose a method to perform the counterfactual conditional probability estimation between cross-world variables.

Preliminary of GraphOOD. The area of OOD learning deals with scenarios in which training and test data follow different distributions. Furthermore, GraphOOD problems focus on not only general feature distribution shifts but also structure distribution shifts (). Accordingly, Graph neural networks (GNNs) (; ) are designed based on node features and adjacent matrix to pass messages, which perform well in solving GraphOOD problems. Previous GNN-based methods (; ; ) aim at disentangling the invariant part and the environment part of the input graph to find rationales, thereby addressing the problem of domain shift. The common learning paradigm is shown in Figure 6, where the input graph is processed and split into two latent variables, i.e., the invariant representation $z^{Inv}$ and the environment-dependent representation $z^{Env}$, and only $z^{Inv}$ is used for label prediction, which is achieved by leveraging the empirical observation encompassing the available labeled samples.

### 6.1 Cross-World Counterfactual Causality Modeling via Teleporter Theory

The obtained invariant representation unavoidably contains significant environment-dependent information due to the inherent inductive bias arisen from the learning paradigm of benchmark methods. On the contrary, the desired invariant representation is expected to solely contain pure environment-agnostic predictive information. However, such a representation can barely be acquired in the real world yet feasibly obtained in the counterfactual world.

To this end, we propose to explore the causal mechanism behind both real-world and counterfactual variables, which is accomplished by modeling the cross-world counterfactual causality. Concretely, we establish an SCM at first, as depicted in Figure 5(a), which elaborates on the causality among the variables in GraphOOD in the real world. In Figure 5(a), there exist four endogenous variables in the real world: the input graph $X$, the learned representation $R$ of $X$, the predicted label $Y$ and the environment-dependent information $E$. $U_{X}$, $U_{R}$, $U_{E}$ and $U_{Y}$ are four exogenous variables corresponding to the endogenous variables. According to Theorem 1, the variable $E$ can be determined as the teleporter, so the cross-world SCM is demonstrated in Figure 5(b), where $x$ represents the intrinsic causal subgraph, $R_{x}$ denotes the environment-agnostic invariant representation, and $Y_{x}$ denotes the predicted label corresponding to the graph $x$, which is also the true label, since ideally, $x$ and $R_{x}$ only include environment-agnostic task-dependent information in the counterfactual world.

Figure: Figure 5: SCM for GraphOOD. Figure (a) denotes the real-world SCM. Figure (b) denotes the cross-world SCM.
Refer to caption: x2.png

### 6.2 Counterfactual Conditional Probability Estimation via Multi-Scale Mixup Scheme

According to the cross-world SCM in Figure 5(b), we determine that our objective is to derive the ideal predicted label $Y_{x}$ by only using the available $X$. Such an objective can be formalized as follows: computing the counterfactual probability $P(Y_{x}=y\mid X=x^{\prime})$, where $x^{\prime}$ denotes the available value of $X$, and $y$ denotes the true label of $x^{\prime}$. Deriving $P(Y_{x}=y\mid X=x^{\prime})$ is equivalent to calculating the conditional probability of $X$ on $Y_{x}$ in the cross-world SCM. Such a computation can be approximated by using neural network-based methods. Adhering Theorem 2’s $d$-separation for cross-world counterfactuals, we can directly obtain $X\upmodels Y_{x}\mid E$ from the cross-world SCM in Figure 5(b). The calculation of $P(Y_{x}=y\mid X=x^{\prime})$ can be acquired as follows:

$$ $P(Y_{x}=y\mid X=x^{\prime})=\sum_{e}P(Y_{x}=y\mid X=x,E=e)P(E=e\mid X=x^{ \prime})$ (7) $$

As demonstrated in Figure 6, to acquire $P(Y_{x}=y\mid X=x^{\prime})$, we propose to design a fine-grained method, which can derive the invariant part $z^{Inv}$ and the environment-dependent part $Z^{Env}$ from the input graph $x^{\prime}$, thereby predict the true label $y$ by leveraging $Z^{Inv}$. According to Equation 7, $P(Y_{x}=y\mid X=x^{\prime})$ can be estimated by summing the conditional probability of $P(Y_{x}=y\mid X=x,E=e)$ with respect to different environment-dependent information $E$, i.e., $z^{Env}$. Following (), we utilize a contrastive learning module to estimate $P(Y_{x}=y\mid X=x,E=e)$, where $z^{Inv}$ is firstly concatenated by a shuffled batch of $z^{Env}$, and then projected into $\tilde{z}^{Inv}$ via a MLP-predictor $\rho$. Ultimately, $z^{Inv}$ and $\tilde{z}^{Inv}$ are used to measure the similarity for contrasting. Accordingly, expanding the available value set of $E$ can widely obtain a more precise estimation of $P(Y_{x}=y\mid X=x^{\prime})$ in Equation 7. Hence, we introduce a Multi-Scale Mixup Scheme (MsMs) to enrich the available data of $E$, which is achieved by leveraging a hyperparameter $scale$ in concatenating the shuffled environment $z^{Env}$. Furthermore, we further expand the available value set of $E$ by the scaled mixup scheme by $M$ times.

Figure: Figure 6: GraphOOD learning paradigm with Multi-Scale Environment Mixup Scheme.
Refer to caption: x3.png

### 6.3 Experiments on GraphOOD

The detailed descriptions of the benchmarks and the baselines are in Appendix B.1 and Appendix B.2, respectively.
To ensure reproducibility, the intricate details of our method’s architecture, and our hyper-parameter settings are detailed in the Appendix B.3. The empirical results on the GOOD and DrugOOD benchmarks are presented in Table 1. By enhancing the available value set $E$ with MsMs, our method places the best in four of six datasets, and shows the best average ROC-AUC score among the baselines, which indicates the effectiveness of our proposed method and further emphasizes the practical generalization of teleporter theory.

**Table 1: Evaluation performance on GOOD () and DrugOOD () benchmark. The best is marked with boldface and the second best is with underline. ${\dagger}$ denotes the reproduction results.**
| Method | GOOD-HIV | DrugOOD | Average |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| scaffold-covariate | size-covariate | IC50-assay | IC50-scaffold | EC50-assay | EC50-scaffold |  |  |
| DIR () | 68.44(2.51) | 57.67(3.75) | 69.84(1.41) | 66.33(0.65) | 65.81(2.93) | 63.76(3.22) | 65.31 |
| GSAT () | 70.07(1.76) | 60.73(2.39) | 70.59(0.43) | 66.45(0.50) | 73.82(2.62) | 64.25(0.63) | 67.65 |
| GREA () | 71.98(2.87) | 60.11(1.07) | 70.23(1.17) | 67.02(0.28) | 74.17(1.47) | 64.50(0.78) | 68.00 |
| CAL () | 69.12(1.10) | 59.34(2.14) | 70.09(1.03) | 65.90(1.04) | 74.54(4.18) | 65.19(0.87) | 67.36 |
| DisC () | 58.85(7.26) | 49.33(3.84) | 61.40(2.56) | 62.70(2.11) | 63.71(5.56) | 60.57(2.27) | 59.42 |
| MoleOOD () | 69.39(3.43) | 58.63(1.78) | 71.62(0.52) | 68.58(1.14) | 72.69(1.46) | 65.74(1.47) | 67.78 |
| CIGA () | 69.40(1.97) | 61.81(1.68) | 71.86(1.37) | 69.14(0.70) | 69.15(5.79) | 67.32(1.35) | 68.11 |
| iMoLD †() | 73.54(1.33) | 65.87(1.98) | 71.23(0.14) | 67.30(0.35) | 76.03(1.66) | 66.41(1.88) | 70.06 |
| iMoLD+MsMs | 74.43(1.96) | 66.19(2.32) | 71.70(0.62) | 67.77(0.48) | 77.29(0.65) | 67.79(0.84) | 70.86 |

## 7 Conclusions and Limitations

We strive to explore graphical representation of counterfactuals and propose the teleporter theory to address the challenge of simultaneously representing real-world and counterfactual variables in a single SCM. The cross-world SCM constructed by using the teleporter nodes can well avoid the theoretical breakdown of twin networks in various cross-world counterfactual scenarios, thereby demonstrating the completeness and generalization of the teleporter theory.
However, the rules that teleporter variables are required to adhere are quite stringent, and introducing such constraints increases the complexity of constructing cross-world SCMs. In future work, we will explore to simplify the proposed rules for determining teleporter variables and attempt to apply our theory in the scenarios involving multiple counterfactual worlds.

## Appendix A Proof of Theorem 3

Below, we use the calculation of the counterfactual statement $P(Y_{x}=y\mid E=e)$ as an example to illustrate that once the conditional independence of relevant variables is obtained through cross-world SCM, cross-world adjustment can be achieved using simple algebraic derivations:

$$ $\displaystyle P(Y_{x}=y\mid E=e)$ $\displaystyle=\sum_{z}P(Y_{x}=y\mid E=e,Z=z)P(Z=z\mid E=e)$ (8) $\displaystyle=\sum_{z}P(Y_{x}=y\mid X=x,E=e,Z=z)P(Z=z\mid E=e)$ (9) $\displaystyle=\sum_{z}P(Y=y\mid X=x,E=e,Z=z)P(Z=z\mid E=e).$ (10) $$

Equation 9 holds because $X\upmodels Y_{x}\mid\{E,Z\}$. Equation 10 holds due to the consistency condition: $X(u)=x,Y(u)=y\rightarrow Y_{x}(u)=y$.

## Appendix B Experimental Settings

### B.1 Benchmarks

We employ two real-world GraphOOD benchmarks, i.e. GOOD [] and DrugOOD [] to exam the performance of our method. GOOD is a systematic benchmark which is tailored specifically for graph OOD problems. We adopt one molecular dataset GOOD-HIV for the graph prediction task, where the objective is binary classification to predict whether a molecule can inhibit HIV.
DrugOOD is an OOD benchmark for AI-aided drug discovery, which provides
two measurements (IC50 and EC50) and their environment-splitting strategies (assay, scaffold, and size). According to the split strategy, we choose four datasets, e.g., IC50-assay, IC50-scaffold, EC50-assay, EC50-scaffold as the benchmarks. Due to the chosen task of GOOD-HIV and DrugOOD are both binary classification, we adopt the ROC-AUC score as the evaluation metric.
The details of benchmark are shown in Table 2.

**Table 2: Benchmark statistics. BC denotes Binary Classification.**
| Dataset | Task | Metric | #Train | #Val | #Test | #Tasks |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| GOOD | HIV | scaffold-covariate | BC | ROC-AUC | 24682 | 4133 | 4108 | 1 |
| size-covariate | BC | ROC-AUC | 26169 | 4112 | 3961 | 1 |  |  |
| DrugOOD | IC50 | assay | BC | ROC-AUC | 34953 | 19475 | 19463 | 1 |
| scaffold | BC | ROC-AUC | 22025 | 19478 | 19480 | 1 |  |  |
| EC50 | assay | BC | ROC-AUC | 4978 | 2761 | 2725 | 1 |  |
| scaffold | BC | ROC-AUC | 2743 | 2723 | 2762 | 1 |  |  |

### B.2 Baselines

To compare our method with other methods, we include three interpretable graph learning methods (DIR [], GSAT [] and GREA []) and five GraphOOD algorithms (CAL [], DisC [], MoleOOD [], CIGA [] and iMoLD []) as baselines.
Note that iMoLD performs not stable on DrugOOD benchmarks, so we reproduce the results using the official code on github.
The descriptions and the github links of the baselines are listed as follows:

- •
DIR [] identifies an invariant rationale by performing interventional data augmentation to generate multiple distributions from the subset of a graph.
[https://github.com/Wuyxin/DIR-GNN](https://github.com/Wuyxin/DIR-GNN)
- •
GSAT [] introduces an interpretable graph learning method that leverages the attention mechanism. It injects stochasticity into the attention process to select subgraphs relevant to the target labels. [https://github.com/Graph-COM/GSAT](https://github.com/Graph-COM/GSAT)
- •
GREA [] identifies subgraph structures called rationales by employing an environment replacement technique. This allows the generation of virtual data points, which in turn enhances the model’s generalizability and interpretability. [https://github.com/liugangcode/GREA](https://github.com/liugangcode/GREA)
- •
CAL [] introduces a causal attention learning strategy for graph classification tasks. This approach encourages GNNs to focus on causal features, while mitigating the impact of shortcut paths. [https://github.com/yongduosui/CAL](https://github.com/yongduosui/CAL)
- •
DisC [] takes a causal perspective to analyze the generalization problem of GNNs. It proposes a disentangling framework that learns to separate causal substructures from biased substructures within graph data. [https://github.com/googlebaba/DisC](https://github.com/googlebaba/DisC)
- •
MoleOOD [] investigates the OOD problem in the domain of molecules. It designs an environment inference model and a substructure attention model to learn environment-invariant molecular substructures. [https://github.com/yangnianzu0515/MoleOOD](https://github.com/yangnianzu0515/MoleOOD)
- •
CIGA [] proposes an information-theoretic objective that extracts the desired invariant subgraphs from the causal perspective. [https://github.com/LFhase/CIGA](https://github.com/LFhase/CIGA)
- •
iMoLD  [] propose a first-encoding-then-split method to disentangle the invariant representation and the environment representation via a residual vector quantization skill and a self-supervised learning pattern. [https://github.com/HICAI-ZJU/iMoLD](https://github.com/HICAI-ZJU/iMoLD)

### B.3 Hyper-Parameters

We reproduce iMoLD with the best hyper-parameters provided in the paper. As for the MsMs part, we choose $scale$ from {0.3, 0.7, 1.0}, $M$ from {1, 3, 5}.

### B.4 Experiments Compute Resources

Experiments are conducted on one 24GB NVIDIA RTX 4090 GPU.

## Peer Reviews

confidence: 4
contribution: 1
initial_rating: 3
presentation: 3
questions: ## Questions:
* To check whether $A \not\perp D_a | B$, why did we condition on $B$ but not $B_a$? Since $B$ and $B_a$ have the same set of parents and exogenous noise, aren't they the same variables? Should the authors condition on $B_a$ for determining such independency?
* How is $A \perp D_a | B$ different from $A \perp D_a | B_a$ or $A \perp D_a | [B, B_a]$? What would be the meaning of each such expression in terms of counterfactuals?
* What is the intuitive meaning of $P(D_0, A=1)$?
* What queries become counterfactually identifiable [1] that were non-identifiable before? Can the authors discuss on that?

[1] Shpitser, Ilya, and Judea Pearl. "Complete identification methods for the causal hierarchy." (2008).
soundness: 3
strengths: The paper contains appropriate citations. The paper is quite well written. It discusses different topics intuitively. The application of counterfactuals in GraphOOD is quite interesting.
summary: This paper points out the issues with existing frameworks for counterfactual representations, such as twin networks or SWIG. The authors propose the teleporter theory, which offers a simpler graphical representation and algebraic derivation. Finally, they show the application of their proposed algorithm on the GraphOOD learning task, where the goal is to learn the invariant features of a graph.
weaknesses: Below I provide my feedback. I hope these comments become useful to the authors.

## Minor weakness:
* The authors discussed many contributions/applications in the introduction. It might be a little hard to focus.
* Line 161: Since the authors are dealing with counterfactual variables, they should intuitively explain what $A \perp D_a | B$ or $A \perp D_a | C$ imply.
* Line 167: Theorem 4.3.1 and its intuitive explanation should be provided in the paper (maybe in the appendix). The authors can explicitly discuss how B, C satisfy the backdoor criterion for $(A, D)$.
* Line 200: The notation $Y(x)$ should be explained. The authors should also discuss why SWIG failed in such a scenario.

## Major weakness
* I **highly recommend** the authors check Shpitser (2008). They have already discussed the issues with twin networks (Page 1957, Figure 9, Figure 11): i) it being complicated, ii) d-separation criterion being misleading.
* Please check Lemma 24, Lemma 25, and Figure 10 (make-cg algorithm). They discuss how to detect which random variables are the same variables in the twin network and how to merge them. To my understanding, these lemmas and the algorithm do the same thing the proposed teleporter theory does in this paper. I would request the authors clearly specify their novelty compared to this work.

---
References:
* [shpitser2008complete] Shpitser, Ilya, and Judea Pearl. "Complete identification methods for the causal hierarchy." (2008). (link: https://www.jmlr.org/papers/volume9/shpitser08a/shpitser08a.pdf )
* Shpitser, Ilya, and Judea Pearl. "What counterfactuals can be tested." arXiv preprint arXiv:1206.5294 (2012).

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: 1. Section 5.2 paragraph 1. "However, obtaining the distribution of exogenous variables $U$ is extremely difficult". The term "difficult" is vague here. Suppose causal sufficiency is assumed in this paper, why is it difficult to learn the distribution of $U$ from data? You also need to compare Theorem 3 with the existing identification methods in [Shpitser & Pearl, 2008].
soundness: 2
strengths: Most parts of the paper are clearly written and easy to follow. Examples are provided for the definitions and theorems. Proofs of the theorems were provided in the appendix. The paper contains some empirical studies and applications. The paper also summarized some previous works and highlighted the improvements made by this work.
summary: The paper proposed the teleporter theory for computing counterfactual queries. The theory mainly improves the original twin networks (or parallel networks) by merging certain duplicates into teleporters. The paper showed that the cross-world SCM constructed based on the teleporter theory may capture more independencies between variables in different worlds, leading to a more general adjustment formula for identifying counterfacutal queries under evidence.
weaknesses: Here are my two major concerns:
1. While not mentioned, it seems that the paper assumes causal sufficiency (no hidden confounders) for all examples and Theorem 1 Rule 3. This can be a huge limitation when compared to the existing work.
2. The identification of counterfactual queries has been studied in [Shpitser & Pearl 2008] Section 4. This previous paper observed that the twin (parallel) networks may not capture the independencies and proposed "merge-cg" operation to merge variables in the twin (parallel) network. To prove the novelty, I think a thorough comparison between the method in [Shpitser & Pearl 2008] and the method in this paper is required. 

Here are some issues regarding the formulation of theoretical results:

3. The notion of "cross-world SCM under teleporter theory" was never formally defined. In fact, I think Theorem 1 in its current form should really be a Definition not a Theorem. I'm also not sure what Theorem 1 is trying to prove here.
4. Similarly, in Theorem 2, d-separation has never been defined on cross-world SCMs under teleporter theory yet, so the notion is not well-defined. It's possible that the authors are unaware of the difference between independence and d-separation. d-separation is a notion defined only on DAGs, whereas (conditional) independencies are more generally defined for variables.
5. I found it difficult to penetrate Theorem 3. You may simply say "If a variable set $Z$ satisfies that $X$ and $Y_x$ are d-separated by $E \cup Z$, then the counterfactual query can be computed as follows: ..."

Comments on the experimental section:

6. The experimental section (Section 6) is quite confusing in general. I think the authors are stacking too many concepts together, e.g., the paragraph below Equation (6). I think the task/setup needs to be clarified further.
7. From my understanding, the goal of the experiments is to show that Equation (6) can lead to better counterfactual identification. If that's the case, I think the paper should explicitly state which methods in Table 1 actually consider Equation (6) and which ones don't. Also, I don't think methods based on neural networks are the best to demonstrate the effectiveness of Equation (6) -- the performance of neural networks can be quite arbitrary.

References:
- Ilya Shpitser, Judea Pearl. Complete Identification Methods for the Causal Hierarchy. J. Mach. Learn. Res. 9: 1941-1979 (2008).

---

confidence: 3
contribution: 4
initial_rating: 6
presentation: 3
questions: My questions generally concern the weaknesses stated above. In particular, I would like to ask the authors to comment on the following questions:

1) Regarding point 1: The proof for Theorem 2 mentions a relation to Markov compatibility by preserving topological structure. While hold true for sets $Z,Z^*$ is this also true for changing variables $D,D^*$? Could the authors elaborate this further, and comment on the above mentioned relation to faithfulness (which I believe is implied by Markov compatibility)?

2) Could the authors comment generally on the previous point 2? In particular, (1) would keeping $U_Z$ as parents of $Z,Z^*$ invalidate any of the presented results, and (2) would the application of rule 3 still be compatible for networks containing  $U$s that are shared between multiple variables?

3) Regarding point 3: Could the authors please provide more details on the proposed Multi-Scale Mixup Scheme (e.g. in the form of a pseudo algorithm)?
soundness: 2
strengths: The paper contains important theoretical contributions, by identifying and fixing issues of Markov compatibility of previous approaches. The presented improvements open up potentials for better utilization of data across many practical applications. The presented experimental evaluation seems to underline the theoretical claims. The presented teleporter networks --in particular with theorem 3-- allow for generalized cross-world adjustments between any two variables and therefore generalize upon backdoor adjustments within twin networks.

The identified issues regarding representation of independencies within twin network and (while I'm not an expert in) SWIGs, are inferred clearly from examples. The presented theorems on teleporter theory and cross-world adjustment are stated clearly and, to the best of my knowledge, are sound.

Apart from the clear theoretical benefits, the authors seem to show that their approach can perform consistently above other baseline methods for graph OOD experiments and simple image datasets.
summary: In their work, the authors demonstrate the inability of twin network and SWIGs the truthfully represent conditional independencies with regard to some underlying counterfactual distribution. In the following, the authors propose a 'teleporter theory', a novel graphical representation that is able to alleviate these problems and allows for generalized adjustment within cross-world inference.

The authors define two categories of variables that change ($D\neq D^*$) or remain constant ($Z=Z^*$) between the factual and counterfactual world. As variables $Z, Z^*$ adapt the same values in the observed and counterfactual domain, the authors propose to merge both sets into unique 'teleporter' variables that are then shared between both worlds. The resulting 'teleporting networks' are claimed to be faithful with regard to the underlying counterfactual distribution. Due to the obtained faithfulness, a generalized cross-world adjustment between any two variables is defined. As a result, the authors demonstrate that broader sets of variables can be used for counterfactual cross-world adjustment than what was available with previous twin networks.

The authors propose to train neural based predictors via a contrastive learning approach that leverages their newly presented adjustment method. The models are tested to two graph OOD datasets and for simple image scenarios. Results are compared against several baselines.
weaknesses: While the paper contains intriguing novel theoretical results, some minor details on the presented theory are left open. My main concern is with the description of the experimental evaluation:

1) The identified limitations in section 3.1 imply that $(X \text{ indep } Y | Z)_p \implies (X \text{ indep } Y | Z)_G$ may not always hold for twin networks. To the best of my knowledge, this condition is usually referred to as faithfulness of the graph. While the proof of theorem 2 mentions Markov compatibility, a discussion and explicit mention of faithfulness (or differences to it) could improve the section.

2) The requirement of rule 3 in theorem 1 is not obvious to me. Removing the previously shared $U_Z$, but fixing $Z$ instead, on one hand it simplifies the graph, on the other hand possible information about $U_Z$ is lost. To the best of my understanding, there would be no downside in the following proofs of keep $U_Z$ and having rule 3 as an optional step.

3) Description of model training is held very short. The authors propose a Multi-Scale Mixup Scheme (MsMs) on top of an existing algorithm that leverages their cross-world adjustment. While the general idea is well introduced, details and the exact working of the MsMs are lacking.