Causal Graph Transformer for Treatment Effect Estimation Under Unknown Interference
Authors: Anpeng Wu
Haiyi Qiu
Zhengming Chen
Zijian Li
Ruoxuan Xiong
Fei Wu
Kun Zhang
Venue: ICLR 2025 Poster (2025)
Keywords: Causal Graph Transformer
Networked Interference
Unknown Interference Graph
Peer Effects
Treatment Effects Estimation
Primary area: causal reasoning

Abstract: Networked interference, also known as the peer effect in social science and spillover effect in economics, has drawn increasing interest across various domains. This phenomenon arises when a unit’s treatment and outcome are influenced by the actions of its peers, posing significant challenges to causal inference, particularly in treatment assignment and effect estimation in real applications, due to the violation of the SUTVA assumption. While extensive graph models have been developed to identify treatment effects, these models often rely on structural assumptions about networked interference, assuming it to be identical to the social network, which can lead to misspecification issues in real applications. To address these challenges, we propose an Interference-Agnostic Causal Graph Transformer (CauGramer), which aggregates peers information via $L$-order Graph Transformer and employs cross-attention to infer aggregation function for learning interference representations. By integrating confounder balancing and minimax moment constraints, CauGramer fully incorporates peer information, enabling robust treatment effect estimation. Extensive experiments on two widely-used benchmarks demonstrate the effectiveness and superiority of CauGramer. The code is available at https://github.com/anpwu/CauGramer.

Published as a conference paper at ICLR 2025
CAUSAL GRAPH TRANSFORMER FOR TREATMENT EF- FECT ESTIMATION UNDER UNKNOWN INTERFERENCE
Anpeng Wu Haiyi Qiu'
Zhengming Chen Zijian Li
Ruoxuan Xiong
Fei Wul Kun Zhang 2,5m
ZZejiaang University MBBZAA
*Guangdong University of Technology
*Emory University Carmegie Mellon University
{chenzhengming) leiiign)) (eaeiiiiiinngmmmillii. @gmail.com, nuoxuan.xiong eaonggeeemoouuuuu kunzl @ emuuedu
(anpwu, haiyiqiu, wufei) @zju.edu..cc
ABSTRACT
Networked interference, also known as the peer effect in social science and spillover effect economies, has drawn increasing interest across various domains. This phenomenon arises when a unit's treatment and outcome are influenced by the actions of its peers, posing significant challenges to causal inference, particu- larly in treatment assignment and effect estimation in real applications, due to the violation of the SUTVA assumption. While extensive graph models have been developed to identify treatment effects, these models often rely on structural as- sumptions about networked interference, assuming it to identical the social network, which can lead to misspecification issues in real applications. To address these challenges, we propose an intefereenceAAnostte Causal Graph Transformer (CauGramer), which aggregates peers information via L-order Graph Transformer and employs cross-attention to infer aggregation function for learning interference representations. By integrating confounder balancing and minimax moment straints, CauGramer fully incorporates peer information, enabling robust treatment effect estimation. Extensive experiments strate the effectiveness and superiority of CauGramer. The code is available at https: / / thub. com/ anpwu/ CauGramer.
widely-used benchmarks demon-
INTRODUCTION
Causal inference has drawn increasing interest across networked scenarios, such epidemiology (Barkley et al., 2020; Benjamin- Chhnn et al., 2018), marketing (Chae et al., 2017; Krishna & Rajan, 2009), economies (Mas & Moretti, 2009), and social science (Aronow & Samii, 2017; Cornelissen et al., 2017; Wang et al., 2018). However, one primary problem in causal inference on networked data stems from networked interference! in which each unit's treatment and outcome are influenced by the actions of its peers (interference nodes). This violation of the Stable Unit Treatment Value Assumption (SUTVA) poses significant challenges for treatment assignment and effect estimation (Cai al., 2023; Jiang & Sun, 2022). For example, in COVID studies (Fisher, 2020; Latkin et al., 2021; Matrajt al., 2021), individuals' decisions to get vaceinated (treatment) not only protect themselves from COVID but also reduce infection risk (outcome) within their social networks, thereby influencing the outcomes of others in the community. In this case, both individual and community immunity responses are influenced by the overall vaccine distribution, making individual- levell information insufficient to identify causal effects. Similar to Jiang & Sun (2022) and Matrajt et al. (2021), given observational data, aim to identify treatment effects and answer the counterfactual question: Would community immunity be stronger treatments were assigned to different group of people?
To estimate causal effects under networked interference, conventional network studying typically assumes that the structure of interference graph?, is known and identical to the social network (Cai
"Corrsponndng Authors
Interferencc refers to interactions between individuals where one person's actions can affect others. "The interference graph represents interactions hetween individuals, with edges indicating these interactions. It may have edges not present the social network, interference occur between non-connected individuals.

---

Published as a conference paper at ICLR 2025
Network
Network
A
Observed Variables (2 Dreg(UUriggaarr).TT... T.X.
Direct (Causal) Effects
A
K
X Keeighöor
xsetr
Xpeerz
To the Same As
Not the Same As
B
Tivers
Taxty
Ysets
Tsets
Easty
Interferences
(a) Conventional Networked Settings
(b) General Networked Settings
Interferences
Causal Diagrams in Networked Data. (a) Conventional networked settings: interference graph equals social network with known peer-treatment effect on outcomes; (b) General networked settings: interference differs from social network with unknown peer-treatment effect.
Figure 1:
et al., 2023; Forastiere et al., 2021; Qu et al., 2021; Liu et al., 2023; Ma et al., 2021; Ma & Tresp, 2021). As shown in Figure I(a), given known interference graph and assuming conditional independence between a unit's 's treatment and its neighbors' treatments, some semi-pat semi-parametric estimators propose networked propensity
methods to infer causal effects (marked in red line) of both the unit's treatment and its neighbors' treatments (Chin, 2019; Ma & Tresp. 2021). I). Moreover, inspired by representation learning Shalit al. (2017), recent works develop representation balancing methods and targeted maximum likelihood estimators for better effect estimation on networked data (Jiang & Sun, 2022; Cai al., 2023: Chen al., 2024). However, these works still follow the structural assumption that the interference graph is identical to the social network, which induces estimation bias in woald networks, like how influenza viruses do not spread strictly along social networks.
Additionally, most existing works assume that the interference of neighbors' treatments on unit's outcome is captured through a specified summary function, commonly defined as neighborhood exposure Z = Agg Neighbor) (Forastiere et al., 2021; Qu et al., 2021; Jiang & Sun, 2022; Ma & Tresp, 2021; Cai et al., 2023; Chen et al., 2024). Typically, this aggregation is performed using a sum average function, which mismatches the true data generation process (DGP) in real- real-world applications, leading to model misspecification and additional bias in causal effect estimation.
Therefore, in this paper, we study causal inference problems under unknown interference graph and an unknown summary function, as illustrated in Figure 1(b). In such scenarios, a unit's s treatment may be influenced by peers' treatments (marked in dashed line). To achieve robust causal estimation under the unknown interference setting, we develop an interference-gnnosti Causal Graph Transformer (CauGramer) framework. First, although the interference graph is unknown, all interference nodes must exist within the T-ooddrr neighbor network (T. is sufficiently large). Thus, we can aggregate peer information (interference) using an L-layer Graph Transformer. Second, the complex interaction between two nodes is unknown. Unlike traditional methods that consider only self-attention, we model the interference function by treating self and peer features as linear- linear-query and GCN-key key pairs, designing cross- gross-attention mechanism to capture these interactions, which also broader the receptive field to learn complex sequential interference representations. Finally, by integrating confounder balancing and minimax moment constraints, our CauGramer fully incorporates peer information, enabling robust treatment effect estimation. Extensive experiments on two widely-used benchmarks demonstrate the superior performance of CauGramer. The contributions summarized as follows:
We formalize the treatment effects estimation problem under an unknown interference graph, which is not the same as the social network, and the structural function is unknown, noting that the interference graph may have edges not present in social network.
To achieve robust causal inference under unknown interference, propose interference- agnostic CauGramer, which aggregates peer information through L-order networks and designs cross-attention capture complex-sequential intervention functions. Moreover, balancing and bridge moment constraints are developed to ensure robust causal prediction.
To illustrate the effectiveness and superiority of CauGramer, experiments on two real-world networks with pseudo-real in scenarios involving unknown interference, latent variables, and limited budgets treatments.
conduct extensive
2

---

Published as a conference paper at ICLR 2025
Table 1: Characteristics of Representative Algorithms on Observational Networked Data.
Settings
Effects
Prior
Sub-Medules
Method
Interference Unmeasured
Main Peer Total [Graph Summary Reweigghing Representation
Attention
CNE (Vettch 2019)
P
NedDccomf (Gue at., 2020)
eammrr (I cung & Loupos, 2022)
Ne-TMLE (Ogbum et al., 2024)
(Khatami
2024)
G-HSIC (Ma & 2021)
SPNet (Huang al. 20233
P
<<
RRNet(Cai et 2023)
NetEst (Jinng & Sun, 2022)
Uncertainty (Hhattachorya al., 2020)
UNITE et al., 2024)
CanGramer (Ours)
In observatioal networked dais. we cunsder two settings with networked Interference Unmeasured Confounders, three causall effects (Main, Peer, and Total effects) two forms of peior knowledge (intrfersace Graph and Summary function). and thece sub-modules (Reweighting, Representation, A Atteation), whtrr indicates that the mothods consider the cornesponding term. For the unmessured confounders setring, "I" denotos that CNE, NeiDeeonf, and SPNett only adjust partial unmeasured confounders embedded in the stractural network and unit information.
2 RELATED WORK
In the causal inference on i.i.d data, conventional methods typically utilized prognostic or propensity scores for matching. stratification, reweighting, and doubly robust techniques control confounding effects (Bang& & Robins, 2005; Hansen, 2008; Lee & Lee, 2022; Li et al., 2016; Rosenbaum, 1987; Rosenbaum & Rubin, 1983; Stuart et al., 2013; Yao et al.. 2021a). With the advances in deep learning. CFRNet (Johansson et al., 2016; Shalit et al., 2017) first propose learning balanced representations by directly minimizing the distribution distance between treated and control groups using the Integral Probability Metric (IPM). Similar to Inverse Propensity Weighting (IPW) (Cai et al., 2023; Forastiere et al., 2021; Lee & Lee, 2022; Liu et al., 2023; 2016; Qu et al., 2021; Veitch et al., 2019; Zhang et al., 2023), representation with IPM has inspired numerous representation-be based methods in both i.i.d. data (Hassanpour& & Greiner, 2019; 2020; Wu et al., 2022; Yao et al., 201 8: 2021b) and networked data (Cai et al., 2023; Guo al., 2020; Zhao al., 2024; Jiang & Sun, 2022; Ma & Tresp, 2021). Following this, we summarize representative algorithms on observational networked data in Table 1.
Under the assumption of no interference effects, CNE (Veitch al., 2019) uses semi-supervised prediction to learn node embeddings for main effects estimation, and NetDeconf (Guo et al., 2020) leverages graph convolutional networks (GCNs) captures the hidden confounders for main effect estimation. However, both approaches overlook interference and peer effects. To address this, DRLearnrr (Leung & Loupos, 2022) proposes a doubly robust estimator and explicitly models interference within a social network. Using targeted maximum likelihood estimation and double machine learning techniques, Net-TMLE (Ogburn al., 2024) and GDML (Khatami et al., 2024) rely on pre-specified exposure mappings and assume interference is limited to direct neighbors. These strong assumptions often fail to hold real-world applications. (Ma & Tresp, 2021), SPNet (Huang et al., 2023). RRNet (Cai et al., 2023), and NetEst (Jiang & Sun, 2022) explore causal effects under networked interference but either focus on main causal effects or require prior knowledge of a summary function. A common limitation across these approaches is the assumption that the interference graph is known and identical to the social network- condition that difficult satisfy in practical scenarios.
addition, methods such as G-HSIC
Under unknown interference graph, Sävje et al. (2021) and Cortez et al. (2022) rely on randomized controlled trials (RCTs) study causal effects. Hoshino & Yanagi (2023) assume that the interference graph is a subgraph of the social network and relies on the random assignment of instrumental vari- ables (IVs) to ensure valid causal inference. In observational data, Bhattacharya et al. (2020) assumes that the true teta-generating process perfectly corresponds (satisfying Markov and faithfulness condi- tions) to some unknown chain graph with two restrictions. However, in practical scenarios, structure learning algorithms often face implementation issues and produce unreliable results due to the size of the graph and unmet assumptions. Although Lin et al. (2024) proposes the UNITE framework, which uses L0-norm regularization and GCNs for accurate treatment effect estimation, it only supports the estimation of AME and IME. Besides, they do not consider that peer treatments can influence the treatment choices of others in real applications. To address these issues and estimate causal effects under an unknown interference graph, we propose incorporating GCNs with cross-attention in the I-llayer CauGramer embedding to aggregate L-order neighbor infornation.
3

---

Published as a conference paper at ICLR 2025
Notably, our CauGramer differs from Graphormer (Ying al., 2021), CT (Melnychuk et al., 2022), CAL (Sui et al., 2022), and SPNet (Huang et al., 2023). Graphormer (Ying et al., 2021) just integrates the encoder of Transformer (Vaswani et al., 2017) into node, edge, and graph embedding learning for graph classification, without considering the Transformer decoder. CT (Melnychuk et al., 2022) is specifically designed for estimating counterfactual outcomes time not for networked data. CAL (Sui et al., 2022) designs two soft mask graphs for node attention and edge attention and conducts implicit backdoor adjustment for graph classification, which suffers from networked interference and unmeasured confounders. For SPNet (Huang et al., 2023), it utilizes an adaptive learning weight function based on concatenation of neighbor features and self-features for reweighting, which fundamentally is not Transformer framework.
3
TREATMENT EFFECT ESTIMATION UNDER UNKNOWN INTERFERENCE
3.1 PROBLEM SETUP AND NOTATIONS
We first describe our setting by taking the COVID vaccination case study (Fisher, 2020; Latkin et al., ; Matrajt et al., 2021). Let X, T, Y denote a unit's features (e.g.. physical fitness), treatment (e.g.. receiving vaccination), and observed outcome (e.g., risk of infection), respectively. As shown in Figure 1(b). in the presence of networked interference, a unit's COVID infection risk is not only influenced by their own vaccination status but also by that of their peers. Therefore, when we study the causal effects of different vaccine assignments (i.e., treatment) on the community immunity level (i.e., outcome), we have to take the interference (i.e., peer effects) into consideration. However, in real applications, the interference graph typically is unknown and is not the same as the social network: COVID transmission is not strictly limited to social connections, as individuals without social ties can still interact and potentially spread the virus.
To model this problem, we consider the networked data as D = (fæ,, t,, Yr 1 A, E), where A and E represent the adjacency matrices of the observed social network and unknown interference graph, respectively, and N denotes the total number of units. For each unit i, we have d-dimensional features æ; X' where X' Rdd binary treatment assignment f; T where T 10,1),
1}, and the corresponding outcome yi E Y where y C R. To distinguish between social networks and interference graphs, we denote the first-order 'neighbors' of in the social network A observed N,; and the interacting nodes of i in the interference graph E as unknown 'peers' P.. Then, we denote the treatment and feature vectors received by unit i's peers as tp, and xp, Similarly, we define tv. and æx-- Under networked interference, the potential outcome of a unit is influenced not only by its treatment but also by its peers' treatments, denoted by y(z;, xP, f, tp), where (1, tp) without subscript represents the manipulated interventions on the treatment. Then, the causal estimand of interest can be defined through three parameters: main effects (ME), peer effects (PE), and total effects (TE) (Cai et al., 2023; Jiang & Sun, 2022). We denote tp Op as the full-zero treatments. Definition (Individual Main Effects (IME)). IME denotes the effects of self-treatment, i.e., (I,) = y(zz, æp, 1, 0p) y(2,, ZP, 0, 0p).
Definition 2 (Individual Peer Effects (IPE)). IPE denotes the effects of peers' treatments, i.e. Tire(x,, tp) y( xp, 0, tp) y( 2,,,0 0, 0p) for any tp E TIP.
Definition (Individual Total Effeett (ITE)). ITE denotes the combination main and peer effects, i.e., Tim tx) y( Tp, 1, tp) y Tp. O, 0p) for any tp TPP...
Similarly, average main effects (AME), average peer effects (APE), and average total effects (ATE) of specific treatment distribution T тr be defined, e.g., ATE E, [Tne( tp)]. Given the observational DD (( f;, yi j== A) with unknown interference E, we aim to identify three treatment effects and seek to answer the counterfactual question "would the community immunity level be stronger if we assigned treatments to a different group of people"?
3.2 CAUSAL IDENTIFICATION ON UNKNOWN INTERFERENCE GRAPH
To precisely estimate the three treatment effects, i.c., ME, PE, and TE, we first discuss causal identification under the standard causal assumptions on networked data (Jiang & Sun, 2022). Assumption 1 (Positivity). The probability of a unit with their peers to receive any treatment pair (1, t)) is always positive, ie.. O P(4 = I, t,,p, tp Di, I.P.) ) < for any Tc.
4

---

Published as a conference paper at ICLR 2025
Assumption 2 (Consistency). The potential outcome is the same the observed outeome under the same delf-treatment and peer-treatment i.e., Y1 = y( up,,, t,, tp,) ))o treatment pair (t,tp,)). Assumption Unconfoundedness), The self-treatment and peer- peer-treatments are independent of the potential outcome given self and peer' features. i.e., y( U, t,,p) TL (t, tp) (I,, pp,))
Theorem (Causal Identification). Given these assumptions, while the Stable Unit Treatment Value Assumption (SUTVA) does not hold under networked interference, the treatment effects are identified as long as we can control the confounders Di and the peers pP:
Proof. As shown in Figure 1, when we consider the (unknown) peer interference graph, all common causes (2); Dp,) of treatment pair (1z; tp,)) and outcomes yi have been discovered. Thus, have:
TITE (x;; tp) = Ely Tp. 1, bp) Di, pp,] Ely pp, 0p) 2i, xp,
Ely TP. 1, tp)
I4,, 2p, t, tp] (z ap, 0, 0p) 24, 0, 0p]
(1)
= Elgh, Is 2p, t, tp] E 2;; xp, 0, Op),
(2)
where Mt.to is the observed outcome when the unit and its peers have features T ap, and receive the treatment pair (t, tp). Eq. (1) holds under the Uncounfoundedness, i.e., y( p,,, F, tp) 11 (t,tp)
(av,, Z.,)). ). Eq. (2) holds under the Consisteney Assumption, i.e., St.tp y(a, 2p, t, tp). Theorem
I holds for any treatment pair (t, (p) under Positivity Assumption, i.e., P(C, = L, tp = (p) > 0. O
However, since the interference graph is unknown, we cannot directly model the expectation function Elyt.p Ki, Fp,, t, tp). Nevertheless, under the Unconfoundednes assumption, we know that y(,; pP, ,,,,,,,4 (p) I {4i bp (ai, XP, Therefore, if we control for all observed confounders [o:)_ we can infer to y(;; PP. 1p) 1 {4i, J {x: } The interaction information in the interference graph is embedded within the node and network features. To capture this, we propose using two functions, 9x and gt, to represent peer feature and peer treatment information, respectively.
Assumption& 4 (Representation). The treatment vector tp, of peer nodes can be captured by a peer treatment function 9t. and the confounder vector xP, by a peer confounder function 9w-
Proposition (Interference). If the unknown interference graph E is latent in full graph information {x, t. A}, then, the outcome Elyy,,p Dp, t, tpl is identified.
Proof. Denote the other non-peer nodes of unit i in the interference graph E as O,, i.c., all units in the community can be represented by {i, P., O.), i.e..r {a:, Tp,, mo,) } for any i. Then, we have:
(b) Elye, 9u (zx, A), 1),.( (t, A)), (3)
Elyt,ep I2 TP. tp]
Elyi,ep Dc, TP, DO. t, tp. tol
where Ia (xr, A) a covariate embedding which captures peer features, and gr (t. A) is a treatment
embedding which captures peer treatments. Eq. (a) holds because (mo,,; do not interfere with the outcomes of unit i. Eq. (b) holds when gu (æ, A) and (t, A) can fully embed peer information. O
In this paper, we do not directly recover the interference graph. Instead, we aim to learn two representation functions and to aggregate interference information. To avoid over-learning we will use a balaneing decomposing constraint remove much irrelevant information possible from Ir and gr that does not belong to peer interference nodes.
4 CAUGRAMER: CAUSAL GRAPH TRANSFORMER WITH CROSS-ATTENTIE
Following the identification results from the previous sections, under unknown interference graph E, we propose an interference-agnossic framework, called Causal Graph Transformer (CauGramer), to learn interference representations R. = 92 (z, A) and R2 = gc (t, A) from observed social network A and units features (x, t). Despite this advancement, there remain several questions and challenges in accurately estimating the main effects, peer effects, and total effects of treatment on outcomes:
What are the characteristics of the interference representations R, = 9z [x A) and R; = (t, A), and how can we learn them?
What is the difference between self-treatment and peer-treatment? How do we balance confounders for self-treatment and outcomes, as well as for peer-treatment and outcomes?
How do we unbiasedly estimate the counterfactual regression outcomes?
5

---

Published as a conference paper at ICLR 2025
Objective Function: min max (1 YY (MSE(V. v)
RTT)) + B Cross Entropy(T. r)) +y wwie P)
Embedding
R2 Embedding
Contal
MI.P
Layer
XL. Layer
R₁
MLP
Feed Forward
MLP
Forward
T= 1 +1-7)x minm (( P) WaC Wa
XM Head
XM Head
MatMal
Mathiul
Bridge Function W
SaftMax
SafMMa
MatMal
MatMal
Scale
Query
Query
Sigmood
Wg
Wx
Wi
Wq
Wix
WW
Lin
GCNIX.
GUNC, A
Linear(R)
GCNOR,
Query
Key
We
Linar((K)
R Rosty
Rg
R
Concat
Rg
Xeri
Nasighher
Agenwork
Resightar har
Akeewark
Figure 2: An illustration of the proposed CauGramer with cross- cooss-attention Since the interference graph E is unknown, we use L-layer CauGramer embedding to learn interference representations R. and R, from L-order neighbors in network A with features X. Then we design a bridge function as minimax constraints to estimate potential outcomes. A denotes A with diagonal elements as 1.
These questions motivate the development of our CauGramer framework. Next, will introduce CauGramer and provide insights into addressing these challenges. Notably, in this paper, we focus on estimating treatment effects through the two interference representations but do not reconstruct the interference graph, as identifying it is pretty hard and unnecessary for identifying treatment effects.
4.1 GCNs WITH BROSS-ATTENTION FOR INTERFERENCE REPRESENTATION LEARNING
Motivation for Cross-Attention In networked data, traditional GCNs often struggle with limited receptive fields and information homogenization, making it challenging to capture long- -isttaeee interference. Consequently, conventional studies often only account for interference among first- order neighbors (Cai et al., 2023; Forastiere et al., 2021; Guo et al., 2020; Jiang & Sun, 2022; Qu et al., 2021). While Transformer's ' variants provide broader perspective in graph learning (Rampášek al., 2022: Sui et al., 2022: Ying al., 2021), their use of GCN embeddings input and attention modeling confined to self-embedding still limits the receptive field, hindering long- long-distance interference learning in the topology. To address this, we extend the Transformer to an L-order neighbor network and use pross-attention to broaden the receptive field for GCNs, enabling them to interference representations. use GCNs with different adjacency matrices to capture topological information and construct local cross-atted cross-attention for longer interaction sequences, allowing exploration of a larger receptive field and learning heterogeneous information.
learn complex-sequente
our CauGramer, as shown in Figure 2, we
Under networked interference, each unit's action and outcome are influenced not only by their own features but also by actions and features of (unknown) peers. For learning interference representation Rz, as shown in Figure 2, we model cross-attention (QKV pairs (qi, k; v;)) on self-features (æ;), nhighbor-fettrre (.,,), and combined features (æ;, Zp.) capture interactions between nodes
within and beyond the local neighborhood. Let (1) = x denote the input of the first layer, and Á represents the adjacency matrix A with diagonal elements 1. For each attention head in layer h,
T (1+1)) Attention (l) v(h)
Softmax
ViA)
Va
(b) Linear (h) (wy K(h) = GCN (r
(AW V(M) = GCN!N (r (N)
AWW (5)
In the L-layer CauGramer embedding with M-head attention, neighbor information and topology structure, while cross-attention provides broader receptive field that allows GCNs to embed complex-sequential interference representations, i.e., R. = 9z (x, A).
GCNs to aggregate L-order

---

Published as a conference paper at ICLR 2025
Then, the final output (D) the input of the first layer of the peer-treatment representation network, i.., ri (1) re As shown in Figure 2, we can similarly construct attention heads in layer h of the treatment representation by considering neighbors' treatments into GCNs.
(k+1) Attention (h) t
v!()
Soflmax
@
v!()
(6)
0 Linear (r) w
K. GCN (T) A. D)W GCNN (r) .A. A)W/" (7)
Then, we learn peer-treatment representations through L layers with M- cross-attentions, i.e. R. = gett, A). As the network's 's depth L increases, the scope our GCNs with cross-attention expands, enabling the embedding of all potential interference peers into representations R2 and R.-
4.2 REPRESENTATION BALANCING FOR ADJUSTING CONFOUNDING BIAS
Motivation for Representation Balancing: The interference representations Rg and R., embedding the information of self-features Ri, peer features æp, and peer treatments tp,, are common cases of self-treatment and outcome of interest, which would confound the causal effect estimation. To estimate main effects and peer effects, inspired by Cai et al. (2023); Guo et al. (2020); Huang et al. (2023); Jiang & Sun (2022); Ma & Tresp (2021), we also learn balanced representations to adjust confounders by minimizing IPM of treated and control groups (Johansson et al., 2016; Shalit et al., 2017). Let
( r, be the concatenation of rz and we use Wasserstein (Arjovsky et al., 2017; Cuturi & Doucet, 2014) to measure the discrepancy of distributions:
IPM(r. t) = Wass( or (1)))==)))
(8)
where (Tea==e) refers to the concatenated representation distribution o == Tz @ rt on the samples with E 13. By minimizing IPM( rx, t). we can learn a balanced representation T; and use it to identify the main effects for self-treatment 4;- However, the relationship between peer-treatment tp; and outcome y, remains confounded by the common causes r,, leading to biased estimates of peer and total effects. Moreover, the peer-treatment tp, is unknown. Thus, we reformulate the balancing constraint r; IL tp, to } IL t4. Recall the GCN structure, where the representation r. serves as a weighted proxy of peer representations {ryjeP, }. hence, we maximize predictive power of {rijep, } on relf-treatment t,, while minimizing IPM(r2, T2, ensure {rijee,, 1 t,:
Cross Entropyf,, t) N ENN [t, (p:) + (1 - -1)) log (1 p:)],
-1),
where p: = (ff })) is feed forward neural networks to estimate self teffttreatmett t,, and is the Sigmoid function. With the balanced r., the objective function of potential outcome estimation:
by MSE(Ú. v) + IPM(r, + B Crros__Enropp) t),
(10)
where ú, = t.- f (r.) + (1- fr()), (r.) and f. (r+)) are ferwaro neural networks estimating potential control and treated outcomes. a and 3 are hyper-parameters to trade off the representation balancing, treatment, and outcome regression, discussed in Appendix D and E.
We embed interference information into representations but do not need to identify or reconstruct it. For representation networks r(æ, A) and outcome regression network ý(r, t)., input the observed variables and use the latent representation r(æ, t, A) to predict the outcome, i.c. Ely] (z, t, A] (r, t). By maximizing the predictive power of r on y. (i.e., minimizing the estimation error (MSE(g. y))), we can force the interference information is embed to r(x, t, A).
4.3 BRIDGE MOMENT CONSTRAINTS FOR POTENTIAL OUTCOME PREDICTION
Motivation for Bridge Moment Constraints: There exists trade-off between representation balancing and treatment regression. To avoid model misspecification and ensure unbiased estimation, we construct conditional moment constraints Ely - ÿ x, t] = O, ensuring the residual approaches zero given and t. Furthermore, reformulate it as unconditional maximum moment problem:
y" arg min max E((y 9)q(r, t)], ((r, t) Sigmoid (Q (K),,
(11)
gEQ
where Q = Linear(r): W and K = Linear(r): W Then, q(-) is termed as a "bridge function", transformation links eliminate confounding effects from the confounders in causal identification.
7

---

Published as a conference paper at ICLR 2025
Table 2: Results of Constant Treatment Effects Estimation on BlogCatalog (BC) and Flickr Datasets.
BC
Effects
CFRNet
DRLeamer NetDeconf G-HSIC
SPNet
CAL
Graphormer
RRNet
NotEst
CauGramer
AME
2.058 0.210+0.05 0.07
=.070 2.006
0.083
0.08620.05
0. -0000
0.070 =.00440008
APB
0.117
0.351 a..m2
0.387 an.u 0.20 2.223
0.370
0.436.m.m
0.229
0.078tno2
0.034
ATE
0. 12330.077
4.004
00.00
60.00
0.35
20.0s 0.203
0.3533a0.s
0.340
0.296
0.065
0.039
IME
0.006
loos 1.00
0.119
Lovos 0.132 10.0: 0.100
0.131 Lo.0s
0.211
0.152
1o.07
0.0991000
0.075 0..7
IPE
0.122au.s
0.220 0. g.305
0.410anas
0.230 0.3841go.7
0,457 au.us
0,238.n.00
0.002
0.0441-..02
ITE
0.147=2007
0.51 440.01 0.351 0.38 1,38 0.2134p.00 0.3700a0.s
0.42990.04
0.311-0005
0.117
0.063
Flickr
Effects
CFRNei
DRLeamer NetDevon!
G-IISIC
SPNet
CAL
Graphormer
RRNet
NetEat
CauGramer
AME
0.066 a.
2.088 0.086_00.0 00.0 2.054 2..000g.0s
0.030
0.360.0.0
0.063gno4 0.0282n.0
APE
0.115
0.21 20.00
0.345:0..0
0.354=0.04
0.121 20.00
0.30220.04
0.409=0.03
0.268=0.09
0.055
0.019=0001
ATE
0.144g0.06
0.117
g.351 0.3000g..0
g.13
0.310gg..s
0.3822n.01
0.374.o.11
0.07 0.032g0.0
IME
0.
199
0.517
0. 13440.05
0. =2.29=0.02
0. 0.137 0.079
0.08620.03
0.229=a.04
0 0.006
0.047=0.04
V PERE
IPE
0.128 0.240
0.38220.07 0..055...4
0.131 =0.05 0.3322=005
0.45940.03
0,29420.10
0.070
0.03220.01
ITE
0. 177.00s 0.531 40.02
0. 0.363 0.14220.06 0.3344=0.0 0.44320..4 0.405
p.100 0.051
Table 3: Results of Heterogeneous Treatment Effects Estimation with/without Unconfoundedness on Bltalo (BC) and Flickr Datasets. The best is boldface while the second best is underlined.
Fifects
CFRNe
DRI
GDMI.
SPNa
NetFat
CEVAR
CNE
UNITE
Coutiey
CG
Canmmmer
AME
0.366
0.083
1. 0.077am
0.087
0.069 g0069
0.0012
0.07 ao.cs
0.40544a
0.519
0.007
s0ic4
0:537 0.243...1
0.108
0.349
1.127.00
0.077
0.44400s
0.046.804
IM6
0.15
2.005 2.0r
2.194
0:110ar.00
0,229
041llane
0.266.8.0
0.145
0.42Panm
0.352.01e
0.125at.2
0.117-0.02
0.218
0,529 o2
0.007 t0
0.279
0.1773aaee
0.308
0.810
0:149aca0
0.125 20.00
Flickr
Bilcets
CFRNet
DRLearser
GDML
SPNN
NotEat
CEVAB
CNE
UNITE
C.G.UC)
CGGNC)
Cantinaerr
AME
0509
0.135
0. 2.230
0.096ssas
0.0302000
0.0684caa
0. 0.168_0
0.04
0.000
0.773
0 ia.cs
APE
0.100:3:0
0.236
0.106:.107
(1067
0. 0.562 0.4822nan
0.007atm1
0.088
0.030.2.0
ATB
0.20
g.131
0.205
0. 0.
0.380
0.056.r
0142
IME
0.174
0. 1.520-000
0.354
0.147
0. 0.139anaa
0.212
2.14
0.007
0.090
IPE
0. 2007
0. m.
0.488
0.2152mm
0. 150
0.505aras
0. 0.837-000
0.1 6128
0.112
01111
FT1
0.274_3.0
0,547
0.716,a00
0.25218.77
0,185
0.47816a4
0,969
0.150
0.127 2000
0.325.0.00
The detailed explanations of bridge functions are deferred to Appendix D.4. If the model is correctly specified and properly optimized, the residual
should be independent of covariates æ, ie., E 1 {x, t}. We have Eig(z, (x, t)- <| = Elq(x: x. t)] Eie] = for any bridge functions q(z, t). The minimax bridge moment constraint in Eq. (11) serves as a residual correction term, ensuring the residual E و q(2r, t)] = 0 for any g(22, t)-
y
By incorporating this constraint, the objective function for potential outcome estimation becomes:
arg min mig max L (1 (11-)) 7) (MSE(). v) IPM(r, t) + 9- Cross_Fntropyyf. f)) + Blg(r, t) ú),, (12)
to rEQ
where is hyper-parameter serving assumption, when 7 = O, the model degrades to trade-off' regression, and when 7 = 1, it degrades to a Conditional Moment model. In the presence of unmeasured confounders, we will collect negative control variables and embed them into the Eqs. (11) and (12), generalizing CauGramer to CauGramer(NC) to address unmeasured confounding bias. The details are deferred to Appendix D.4.
interpolation coefficient. Under unconfoundedness
The implementation details, pseudo-code, and advances of CauGramer are provided in Appendix D. Discussion and optimization of Hyper-Paraameters fa, 3, 7) can be found in Appendix E.
EMPIRICAL EXPERIMENTS
To evaluate the performance of our CauGramer on networked data, we conduct extensive experiments
across eight dimensions: (1) Constant Treatment Effects (Table 2), (2) Heterogeneous Treatment Effects (Table 3), (3) Unknown Interference Graph (Figure 3), (4) I. Limited Budgets (Figure 4), and Optimization (Figure 6 and Table 5). (7) Time Cost (Table 6), and (8) Various Interference Scenarios, including Interference Strength (Figure 7), Interference Functions (Table 7), and Unmeasured Confounding (Table 8). We include comparisons against more baselines in Tables 9& 10 in Appendix E.
(5) Ablation Study (Table 4) in main text; (6) Hyper-Paramee
Datasets. In real-world applications, the ground-truth causal relations are unknown, and thus it is impossible to evaluate the causal estimation performance of CauGramer directly. Therefore, following existing works (Cai et al., 2023; Guo et al., 2021; 2020; Jiang & Sun, 2022; Ma et al.,
8

---

Published as a conference paper at ICLR 2025
Five Competing Methods:
CFRNet
SP
NetEst
CauGramer(NC)
CauGramer
Favg en AME Estimation
0.2
SPEIE on IME Estimation
0..
FAvG on AME Esimation
0.3
VEEEE IMC Estimation
0.2
0.2
0.1
0.1
0.1
Known S.GIN
SAG.2N Kandom
Known S.G.N 8.GNN Kandom
Known S.G.N S.GEN Kandom
Known S.GNN S.G2N Handem
0.6
Estimation
0.6
IPE Excimation
0.6
Favo APE Estimasion
0.6
IPE Estimation
0.4
0.2
0.0
0.0
Knowa S.G.IN S.G.N Randuns
Known S.G.IN S.G.2N Random
Known S.G.IN S.G.2N Random
Known S.G.IN SGGNN Random
Cavo
0.6
VEysue Escimatioon
Cavs
VSPSHR
0.2
0.2
0.2
0.0
0.0
0.0
0.0
S.G.IN
Randocs
Known S.G.IN S.G.NN
S.G.IN
Knowa
Random
BlogCaralog Daraset with Linknown Interference Graph
Flickr Datnset with Unknown Interforenee Graph
Figure 3: Results on General Networked Data with Unknown Interference Graph and Unknown Summary Functions: 'Known' graph the same social networks; 'S.G.IN' and 'S.G.2N' sampled SubGraphs from 1st and 2nd-order neighbors graph; 'Random' denotes random graphs.
Cava ATE Estimation BlogCatalog
Save
Estimation Flickr
0.6
Competing Methods
0.4
CFRNet
NetFar
CauGramen(NC)
CanGramer
0.0
0.0
Assigned
2udgg20%
Rudig00% Radg800% Budg1000
Assigned
Budg220%
Budg50% Bodgg005 Budgi 00%%
Figure 4: Results (eAVG) of ATE Estimation on Limited Budget for Treatment Distribution: 'Assigned'
denotes observed treatment, 'BudgX%' indicates varying budget allocations for treatment assignment.
2021; Veitch et al., 2019), we use pseudo-real datasets from BlogCatalog (BC) and Flickr, where the features (ar) and social networks (A) are real, while treatments (t), outcomes (y), and interference (E) are simulated. We use the same data simulation by Jiang & Sun (2022) to evaluate CauGramer's performance original data with known intervention graphs and summary functions for constant treatment effects. Then, we design a series of generalized heterogeneous treatment effects datasets with unknown summary functions across various settings. Furthermore, we simulate unmeasured confounding by removing features. The generation details are deferred to Appendix E.
Baselines. We summarize the characteristics of all competing baselines in Table Section In the traditional data (Jiang & Sun, 2022), we compare our methods CauGramer with causal methods (CFRNet (Shalit et al., 2017), DRLearner (Leung & Loupos, 2022), NetDeconf (Guo al., 2020), G-HSIC (Ma & Tresp. 2021 1), SPNet (Huang et al., 2023), RRNet (Cai et al., 2023), and NetEst (Jiang Sun, 2022)) and two Transformer methods (CAL (Sui et al., 2022) and Graphormer (Ying et al., select top-4 baselines and add four models (GDML (Khatami et al., 2024), CEVAE
2021)). Then, (Louizos et al., 2017), CNE (Veitch et al., 2019) and UNITE (Lin et al., 2024)) in general network settings. For the sake of faimess, modify all non-interference methods by incorporating neighbors' treatment and social networks as additional input. For unmeasured confounders, we provide two variants: CUG.(UC) randomly selects and removes two features from the input unmeasured confounders and C.G.(NC) incorporates NCs as proxies for address unmeasured confounding effects.
Evaluations. In the experiments, aim estimate the ME, PE, and TE of treatments individual and population level. In population, we use the Mean Absolute Error (MAE) on AME, APE, and ATE metric, i.e., EAVG = || 7|I, where and are the true and estimated average effects. In individual, we use rooted Precision in Estimation of Heterogeneous Effect (PEHE) on IME, IPE, and
ITE, i.e. PEEEE
D (+ 2. We conduct 10 replications to report (mean results.
We compare our method with the baselines for estimating treatment effects with known interference graphs in Tables 2 and 3. First, the proposed CauGramer consistently outperforms all
Results.

---

Published as a conference paper at ICLR 2025
Table 4: Ablation studies on CauGramer (a, () by setting (no balancing constraints),
B 0 (no peer treatment constraints), and = 0 (no moment constraints). "w/o CA' indicates no
cross-attention. We report cAVE on AME, APE, ATE, and / /CEEEE on IME, IPE, ITE.
(Bo)Catalog(BC)
AME
APE
ATE
IME
IPE
ITE
CauGramer (0.0,0.0.0.0)
0.003
0.072+0.02
20.06
0.147
0.139 20.01
0.189 20.00
CauGramer (0.0,0.4.0.2)
0.118 0.11810.10
0.061
0.11610.07
0.209
0.21610.06 0.13310.02
CauGramer (0.8,0.0,0.2)
0.085 10.08
+0.04
0.066i=0.0G
0.12610.11
0.132 0.120
CauGramer (0.8,0.400.0)
1.000
0.06411.04
10.05
0.12310.12
0.121
0.14210.03
CauGramer (0.8.0.4.0.2) wio CA
0.104
0.067
0.072
0.154
0.125
0.142
CauGramer (0.8,0.4.0.2)
0.07320.06
0.057
0.045.0.04
0.090 #0.05
0.118go0.2
0.125
Flickr
AME
APE
ATE
IME
IPE
ITE
CauGramer (0.0.0.0.0.0)
0.080=0.07
0.05040.05
0.131+0.0G
0.127
0.130=0o.2
0.209
CauCramer (0.0.0.4.0..)
0.087=0.07
0.21340.15
0.309
0,146=0.00
0.303
0.38440.18
CauGramer (0.8,0.0,0.2)
0.028 0.066
0.028
0.1 1,118
0.112+0.01
0.128
CauGramer (0.8,0.4.0.0)
0.078
0.041
0.051:0.03
0.149
0.141
0.16020.01
CauGramer (0.8.0 8.0.4.0.22 w/o CA
1.100
0.135 16
0.174:1011
0.190
0,265 0.221
CauCramer (0.8.0.400.2)
0.058=0.05
0.030
0. 0.025=00.0
0.096 0G
0.111=0.01
0.125+0..0
baselines, because our Causal Graph Transformer with Minimax Moment Constraints improves the computation graph with Cross-Attention and provides double guarantees to address confounding bias. Second, when NCs are available and latent variables exit, while C.G.(UC) suffers from unmeasured confounding, C.G.(NC) significantly improves performance, even approaching that of CauGramer, In this case, the moment constraint in C.G.(N)) acts as a bridge function. By comparing our C.G.(NC) and CauGramer method to the optimal baseline in Tables 2 and 3, when the interference graph is known, we observe a decrease in evG and εPEнE of (39%, 56%) ) and (46%, 51% on constant total effect estimation for BlogCatalog and Flickr datasets, a decrease in evG and epEhe of (55%, 80%) and {28%, 32% on heterogeneous total effect estimation, highlighting the superiority of our method.
Abalation Studies. As shown in Table 4, we denote different ablation versions as CauGramer(a,, B, 7), where hyperparameters a controls balaneing constraints, 3 handles peer treatments constraints and y manages moment constraints. "w/o CA' indicates CauGramer without the cross-attention module. The results in Table 4 demonstrate that each component contributes effectively to causal effect estimation. In Appendix E.2, further discuss the hyperparameter optimization for CauGramer.
Scaling to Unknown Interference Graph and Limited Budgets. Figure 3 shows the performance of our methods on unknown interference across BlogCatalog and Flickr datasets. Across 1tt and 2nd-order SubGraphs ('S.G.IN' and 'S.G.2N') and 'Random' Graphs, CauGramers consistently outperform all baselines. Especially on IS', our method shows a TE estimation improvement of over 35% In Figure 4, across treatment assignment budgets from 20% to 100%, we observe robust performance with ATE estimation errors below 0.1 on both datasets. Conversely, other methods display notably increasing estimation errors as the budget changes.
Time Complexity Analysis. CauGramer suffers from the same limitation as other graph transformers, namely the quadratic complexity of the attention computation. However, unlike Graphormer and its variants, which rely on computationally expensive full graph attention mechanisms through self- attention. CauGramer computes cross-attention only from its 1-hop neighbors' embeddings. Its time cost decreases as node degree decreases. Empirically, we report the average training times of various methods a single execution on both datasets in Table in Appendix E.3. CauGramer significantly reduces single execution training time compared to most baselines, keeping it under I 88 seconds.
6 CONCLUSION
In networked interference settings, many graph models developed to identify treatment effects rely on the structural assumption a known interference graph and aggregation function, which may not hold true in real-world applications and can result in model misspecification. To address this issue, we study causal inference problems under unknown interference and propose a CauGramer, which learss interference representations and infers causal effects without relying structural assumptions. By integrating confounder balancing and minimax moment constraints, CauGramer fully incorporates peer information, enabling robust treatment effect estimation.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
This work was supported by the National Natural Science Foundation of China (62441605, 62441617, 62376243, 62037001), and the Starry Night Science Fund Shanghai Institute for Advanced Study (Zhejiang University).
REFERENCES
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pp. 214 223. PMLR, 2017.
Peter M Aronow and Cyrus Samii. Estimating average causal effects under general interference, with application to a social network experiment. Annals of Applied Statistics, 1914):112211447, 2017.
Heejung Bang and James M Robins. Doubly robust estimation in missing data and causal inference models. Biometrics, 61(4):962-97 2005.
Brian G Barkley, Michael G Hudgens, John D Clemens, Mohammad Ali, and Michael Emch. Causal inference from observational studies with clustered interference, with application cholera vaccine study. The Annals of Applied Statistics, 14(3):1432-144) 2020.
Jade Benjamin-Chung Benjamin F Arnold, David Berger, Stephen P Luby, Edward Miguel, John M Colford Jr, and Alan Hubbard. Spillover effects in epidemiology: parameters, study designs and methodological considerations. International journal of epidemiology, 47(1):332-347, 2018
Rohit Bhattacharya, Daniel Malinsky, and Ilya Shpitser. Causal inference under interference and network uncertainty. Uncertainty in Artificial Intelligence, pp. 1028-1038. PMLR, 2020.
Ruichu Cai, Zeqin Yang. Weilin Chen, Yuguang Yan, and Zhifeng Hao. Generalization bound for estimating causal effects from observational network data. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pp. 163-172, 2023.
Inyoung Chae, Andrew T Stephen, Yakov Bart, and Dai Yao. Spillover effects in seeded word-of- mouth marketing campaigns. Marketing Science, 36(1):89-104, 2017.
Dexiong Chen, Leslie O'Bray, and Karsten Borgwardt. Structure-aware transformer for graph representation learning. In International Conference on Machine Learning, pp. 3469-3489. PMLR. 2022.
Weilin Chen, Ruichu Cai, Zeqin Yang, Jie Qiao, Yuguang Yan, Zijian Li, and Zhifeng Hao. Doubly robust causal effect estimation under networked interference via targeted learning. In Forty-first International Conference on Machine Learning, 2024. URL https:/ /ppeneeview.nerm net/ Forum?id=511 4dw. dws.
Alex Chin. Regression adjustments for estimating the global treatment effect in experiments with interference. Journal of Causal Inference, 7(2):20180026,
80026, 2019.
Thomas Cornelissen, Christian Dustmann, and Uta Schönberg. Peer effects in the workplace. American Economic Review, 107(2):225-46 2017.
Mayleen Cortez, Matthew Eichhor,, and Christina Yu. Staggered rollout designs enable causal infer- under interference without network knowledge. Advances Neural Information Processing Systems, 35:7437-7449, 2022.
Marco Cuturi and Arnaud Doucet. Fast computation of wasserstein barycenters. In International conference on machine learning, pp.
PMLR, 2014.
Dean Fekles, Brian Karrer, and Johan Ugander. Design and analysis experiments in networks: Reducing bias from interference. Journal of Causal Inference, 2(1):2015 2017.
Kiva A Fisher. Factors associated with cloth face covering among adults during the covid-19 pandemic-unt states, april and may 2020. MMWR. Morbidity and mortality weekly report, 69, 2020.
11

---

Published as a conference paper at ICLR 2025
Laura Forastiere, Edoardo M Airoldi, and Fabrizia Mealli. Identification and estimation of treatment and interference effects in observational studies on networks. Journal of the American Statistical Association, L16(534):901-915 2021.
Ruocheng Guo, Jundong Li, and Huan Liu. Learning individual causal effects from networked observational data. In Proceedings of the 13th international conference on web search and data mining. pp. 2222200,2202 2020.
Ruocheng Guo, Jundong Li, Yichuan Li, K Selçuk Candan, Adrienne Raglin, and Huan Liu. Ignite: A minimax game toward learning individual treatment effects from networked observational data. In Proceedings of the Twenty-Ning -inty International Conference on International Joint Conferences Artificial Intelligence, pp. 4534-4540 2021.
M Elizabeth Halloran and Claudio Struchiner. Causal inference in infectious diseases. Epidemiology. pp. 142-151, 1995.
Ben B Hansen. The prognostic analogue of the propensity score. Biometrika, 95(2):481 2008.
Negar Hassanpour and Russell Greiner. Counterfactual regression with importance sampling weights In IJCAI, pp. 5880-5887, 2019.
Negar Hassanpour and Russell Greiner. Learning disentangled representations for counterfactual regression. In International Conference on Learning Representations, 2020.
Tadao Hoshino and Takahide Yanagi. Causal inference with noncompliance and unknown interference. Journal of the Americun Statistical Association, pp. 1-12, 2023.
Jie Kate Hu, Eric J Tchetgen Tchetgen, and Francesca Dominici. Using negative controls adjust for unmeasured confounding bias in time series studies. Nature Reviews Methods Primers, 3(1):66, 2023.
Qiang Huang, Jing Ma, Jundong Li, Ruocheng Guo, Huiyan Sun, and Chang. Modeling in- terference for individual treatment effect estimation from networked observational data. ACM Transactions on Knowledge Discovery from Data, 18(3):1-21, 2023.
Michael G Hudgens and M Elizabeth Halloran. Toward causal inference with interference. Journal of the American Statistical Association, 103(482):832 842, 2008
Song Jiang and Yizhou Sun. Estimating causal effects networked observational data via repre- sentation learning. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pp. 852-861, 2022.
Fredrik Johansson, Uri Shalit, and David Sontag. Learning representations for counterfactual inference. In International conference machine learning, pp. 3020-3029, PMLR, 2016.
Nathan Kallus, Xiaojie Mao, and Masatoshi Uchara. Causal inference under unmeasured confounding with negative controls: A minimax learning approach. arXiv preprint arXiv:2103. 14029, 2021.
George Karypis and Vipin Kumar. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM Journal on scientific Computing, 20(1):359-39) -92, 1998.
Seyedch Baharan Khatami, Harsh Parikh, Haowei Chen, Sudeepa Roy, and Babak Salimi. Graph neural network based double machine learning estimator of network causal effects. arXiv preprint arXiv:2403. 1133. 2024.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint 1609 0290000977,... 2016.
Aradhna Krishna and Uday Rajan. Cause marketing: spillover effects of cause-related products in 2009.
product portfolio. Management Science,
Carl Latkin, Lauren A Dayton, Grace Yi, Arianna Konstantopoulos, Ju Park, Catherine Maulsby, and Xiangrong Kong. Covid-19 vaceine intentions in the united states, a social-ecological framework. Vaccine, (9(16))2288-229 2021.
12

---

Published as a conference paper at ICLR 2025
Myoung-Jae Lee and Sanghyeok Lee. Review and comparison of treatment effect estimators using propensity and prognostic scores. The international journaal of biostatistics, 18(2):35-330, 2022.
Michael P Leung and Pantelis Loupos. Unconfoundedness with network interference. arXiv preprint arXiv:2211.07823, 6, 2022.
Sheng Li, Nikos Vlassis, Jaya Kawale, and Yun Fu. Matching via dimensionality reduction for estimation of treatment effects in digital marketing campaigns. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pp. 3768-3774 2016.
Xiaofeng Lin, Guoxi Zhang, Xiaotian Lu, and Hisashi Kashima. Treatment effect estimation under unknown interference. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 28 42. Springer, 2024.
Jie Liu, Fangjuan Ye, and Yang Yang. Nonparametric doubly robust estimation of causal effect on networks in observational studies. Stat, 12(1):e549, 2023.
Lan Liu, Michael Hudgens, and Sylvia Becker- Becker-Dreps. On inverse wrebahelity-weighhedd estimators in the presence of interference. Biometrika, 103(4):829 8420-4422222 2016.
Christos Louizos, Uri Shalit, Joris Mooij, David Sontag, Richard Zemel, and Max Welling. Causal effect inference with deep latent-variable models. Advances in neural information processing systems, 30, 2017.
Jing Ma, Ruocheng Guo, Chen Chen, Aidong Zhang, and Jundong Li. Deconfounding with networked observational data in a dynamic environment. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pp. 166-174, 2021.
Yunpu Ma and Volker Tresp. Causal inference under networked interference and intervention policy enhancement. In International Conference on Artificial Intelligence and Statistics, pp. 3700-3708. PMLR, 2021.
Alexandre Mas and Enrico Moretti. Peers at work. American Economic Review, 2009.
Laura Matrajt, Julia Eaton, Tiffany Leung, and Elizabeth R Brown. Vaccine optimization for covid-19; Who to vaccinate first? Science Advances, 7()) 2021.
Valentyn Melnychuk, Dennis Frauen, and Stefan Feuerriegel. Causal transformer for estimating
counterfactual outcomes. In International Conference on Machine Learning, pp. 15223-11322 PMLR, 2022.
Wang Miao, Xu Shi, and Eric Tchetgen Tchetgen. A confounding bridge approach for double negative control inference on causal effects. arXiv e-prints, pp. arXiv-1808, 2018.
Elizabeth L. Ogburn, Oleg Sofrygin, Ivan Diaz, and Mark Van der Laan. Causal inference for social network data. Journal of the American Statistical Association, 119555))597-611 2024.
Julius Owusu. Randomization inference of heterogeneous treatment effects under network interfer-
ence. arXiv preprint arXiv:2308.. 00202, 2023.
Judea Pearl. Causality. Cambridge university press, 2009.
Zhaonan Qu, Ruoxuan Xiong, Jizhou Liu, and Guido Imbens. Efficient treatment effect estimation in observational studies under heterogeneous partial interference. arXiv preprint arXiv:2107.122: 12420, 2021.
Ladislav Rampášek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Do- minique Beaini. Recipe for general, powerful, scalable graph transformer. Advances in Neural Information Processing Systems, 35:14501-14515 2022.
Paul R Rosenbaum. Model-based direct adjustment. Journal of the American statistical Association, 82(398):387 1987.
13

---

Published as a conference paper at ICLR 2025
Paul R Rosenbaum and Donald B Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41-55, 1983.
Fredrik Sävje, Peter Aronow, and Michael Hudgens. Average treatment effects in the presence of unknown interference. Annals of statistics, 49(2):673, 2021.
Uri Shalit, Fredrik D Johansson, and David Sontag. Estimating individual treatment effect: general- ization bounds and algorithms. In International conference machine learning, pp. 3076-3085. PMLR, 2017.
Xu Shi, Wang Miao, and Eric Tchetgen Tchetgen. A selective review of negative control methods in epidemiology. Current epidemiology reports, 7:190-202, 2020.
Michael E Sobel. What do randomized studies of housing mobility demonstrate? causal inference in the face of interference. Journal of the American Statistical Association, 101(476):198-11477 2006.
Elizabeth A Stuart, Brian K Lee, and Finbarr P Leacy. Prognostic based balance measures can be a useful diagnostic for propensity score methods in comparative effectiveness research. Journal of clinical epidemiology, (6(8):S84-S9)) 2013.
Yongduo Sui, Xiang Wang, Jiancan Wu, Min Lin, Xiangnan He, and Tat-Seng Chua. Causal attention for interpretable and generalizable graph classification. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1696-1705, 2022.
Costanza Tortu, Irene Crimaldi, Fabrizia Mealli, and Laura Forastiere. Modelling network interference with multi-valued treatments: the causal effect of immigration policy on crime rates. arXiv preprint arXiv:2003, 10525, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Victor Veitch, Yixin Wang, and David Blei. Using embeddings correct for unobserved confounding in networks. Advances in Neural Information Processing Systems, 32, 2019.
Ming-Te Wang, Noona Kiuru, Jessica L. Degol, and Katariina Salmela-Aro. Friends, academic achievement, and school engagement during adolescence: social network approach to peer influence and selection effects. Learning and Instruction, 58:148-11 160, 2018.
Anpeng Wu, Junkun Yuan, Kun Kuang, Bo Li, Runze Wu, Qiang Zhu, Yueting Zhuang, and Fei Wu. Learning decomposed representations for treatment effect estimation. IEEE Transactions Knowledge and Data Engineering, 35(5):4989-500 2022.
Liuyi Yao, Sheng Li, Yaliang Li, Mengdi Huai, Jing Gao, and Aidong Zhang. Representation learning for treatment effect estimation from observational data. Advances neural information processing systems, 31, 2018
Liuyi Yao, Zhixuan Chu. Sheng Li, Yaliang Li, Jing Gao, and Aidong Zhang. A survey causal inference. ACM Transactions on Knowledge Discovery from Data (TKDD), 15(5):1-46 2021a.
Liuyi Yao, Yaliang Li, Sheng Li, Mengdi Huai, Jing Gao, and Aidong Zhang. Sci: subspace learning based counterfactual inference for individual treatment effect estimation. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 3583-3587, 2021b.
Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Yin Liu. Do transformers really perform badly for graph representation? Advances in neural information processing systems, 34:28877-28888, 2021.
Christina Lee Yu, Edoardo M Airoldi, Christian Borgs, and Jennifer T Chayes. Estimating the total treatment effect in randomized experiments with unknown network structure. Proceedings of the National Academy of Sciences, 119(44)):2208975119 2022.
14

---

Published as a conference paper at ICLR 2025
Yuan Yuan and Kristen M Altenburger. A two-part machine learning approach characterizing network interference in a/b testing. arXiv preprint S.tv:2300.0099900 2023.
Yilan Zhang, Hanlin Zhang, Zachary Chase Lipton, Li Erran Li, and Eric Xing, Exploring transformer backbones for heterogeneous treatment effect estimation. Transactions on Machine Learning Research, 2023.
Ziyu Zhao, Yuqi Bai, Ruoxuan Xiong, Qingyu Cao, Chao Ma, Ning Jiang, Fei Wu, and Kun Kuang. Learning individual treatment effects under heterogeneous interference in networks. ACM Transactions Knowledge Discovery from Data, 18(8):1-21, 2024
Ziyu Zhao, Anpeng Wu, Kun Kuang, Ruoxuan Xiong, Bo Li, Zhihua Wang, and Fei Wu. Networked instrumental variable for treatment effect estimation with unobserved confounders. IEEE Trans- actions Knowledge and Data Engineering, 37(2):823 836, 2025. doi: 10.1 2022. 3491776.
15

---

Published as a conference paper at ICLR 2025
A BROADER IMPACTS
This paper proposes a novel Causal Graph Transformer algorithm, named CauGramer, which extends the computation graph of Transformer networked data and uses GCN with cross-attention to learn complex intervention representations. By fully embedding peer information, the balancing constraints and minimax moment constraints provide double guarantees for causal prediction. This advancement in Transformer improves the interpretability and generalization performance of the model across various networked scenarios, such as epidemiology, marketing, and social science. For instance, in COVID vaccination, when vaccine resources are limited, CauGramer can identify the causal effects of different treatment distributions on groups and then allocate vaccinations precisely, maximizing community immunity with limited resources. Additionally, it supports evaluating the impact of community skills training on the labor market and estimating the effects of public policies on social groups, such as housing distributions community stability and economic development.
However, in real social problems, treatment distributions that maximize treatment effects on the population might not be the optimal choice for certain individuals. When individual treatment effects and community treatment effects conflict, we should make decisions based on the specific situation rather than relying solely on model results. Overall, CauGramer advances the Transformer framework for networked data, providing new lines for further research on causal Transformers in the machine learning field. We will also explore its application more ML tasks, using accelerate large models with Transformers and eliminate large model illusions through causal prediction.
B LIMITATIONS AND FUTURE DIRECTIONS
Unknown Social Network: CauGramer relies on a known social network for information aggregation. In practice, however, such networks may not always be accessible due privacy concerns or application-specffc constraints. Nevertheless, as demonstrated in Figure 3, CauGramer remains robust even when the provided network entirely unrelated to the 'Random' interference graph. still learns causal effects effectively and outperforms existing methods. suggesting that any connected graph-even one that randomly generated and independent of the interference graph- can help aggregate interference information. In future work, we aim to address this limitation by developing strategies to construct efficient connected graphs directly from the data. This would further reduce dependence a predefined social network and enhance the method's applicability to scenarios without network availability.
Negative Controls Availability: In the presence of latent confounding variables, although the proposed CauGramer can readily generalize to address unmeasured confounding by constructing bridge functions of negative controls, identifying negative controls remains challenging task, requiring expert knowledge and extensive experimental data support. Fortunately, with increasing attention from practitioners to this technique, negative controls are becoming more accessible through web technologies, user profiles, and digitized records (Hu et al., 2023). To further study unmeasured confounding, we plan to explore techniques like instrumental variable (IV) regression, variational autoencoders (VAEs), and sensitivity analysis in future work.
C
THE DIFFERENCE OF OUR ALGORIHTMS WITH TRADITIONAL METHODS
Our CauGramer is different with the following algorithms:
1. Leung & Loupos (2022): This paper introduces a non-parametric approach for estimating causal effects, specifically treatment and spillover effects, using observational data from known social network. It proposes a doubly robust learner to model causal effect where in- terference decreases with network distance, accounting for peer influences both outcomes and treatment selection. Unlike prior studies that rely on low-dimensional representa- tions of confounding, the authors introduce high-dimensional network unconfoundedness condition, utilizing graph neural networks (GNNs) to estimate these confounders effectively,
2. Ogburn et al. (2024): This paper introduces a semiparametric approach for handling de- pendencies caused by information transmission and latent similarities between nodes. It assumes a known interference graph with interference limited to direct neighbors and relies on prior knowledge of two structural functions, se and 8x summarize covariate and
16

---

Published as a conference paper at ICLR 2025
treatment interferences. Given these strong assumptions, Net-TMLE leverages structural equation modeling and targeted maximum likelihood estimation to estimate causal effects, supporting inference for static, dynamic, and stochastic interventions in social networks.
3. Khatami et al. (2024): Assuming the interference graph and exposure mapping known, this paper proposes constructing a focal set, where all units' neighborhoods are non- overlapping, to ensure independence among analyzed units and enable consistent estimation of causal effects in social networks. By combining graph machine learning approaches with the Double Machine Learning (GDML) framework, this method represents significant advancement in causal inference. It integrates Graph Neural Networks (GNNs) to esti- mate propensity scores and outcome models while addressing challenges like interference, nigh-dimensional confounding, and model misspecification.
4. Sävje et al. (2021): This paper study average treatment effects estimation in randomized experiments with unknown and arbitrary interference among units. The authors propose the Expected Average Treatment Effect (EATE) as a generalized estimand that incorporates potential spillover effects by marginalizing all possible treatment assignments. While their study provides theoretical and experimental analysis using existing methods, it does not introduce new algorithms to address unknown interference. Furthermore, their analysis is based on randomized controlled trials (RCTs). In contrast, our paper focuses on addressing the challenge of unknown interference observational data.
5. Cortez et al. (2022): Under unknown network structures, Cortez et al. (2022) extend the work of Yu et al. (2022) by generalizing causal inference methods from linear to polynomial models, enabling the estimation of total treatment effects (TTE) under more complex and non-linear network interactions. They introduce a staggered rollout experimental design, which incrementally applies treatments and collects observations, eliminating the need for prior knowledge of the network structure. However, limitation of their approach is its reliance on multiple randomized trials (i.e., the staggered rollout design) to gather observations at different stages.
6. Hoshino & Yanagi (2023): This paper studies causal inference models in which individuals interact within a social network and may not comply with the assigned treatments. It poten- tially assumes that the interference arises through the social network, where the structure of interference may depend on local or approximate neighborhood interactions. Based on this, the paper introduces Instrumental Exposure Mapping (IEM), a low-dimensional representation of potentially complex spillover effects, to estimate Intention-to- Treat (ITT) and Local Average Treatment Effects (LATE) for compliers, accounting for both direct and indirect effects while explicitly handling the challenges of noncompliance and unknown interference. However, this paper assumes that the interference graph is a subgraph of the social network and relies on the random assignment of instrumental variables (IVs), as well as the independence of IVs across individuals, to ensure valid causal inference.
7. Bhattacharya et al. (2020): Under interference and network uncertainty, this paper proposes methods that integrate structure learning and causal inference techniques to estimate causal effects. Their algorithms rely on structure learning to first learn graphical models and then estimate the Population Average Overall Effect (PAOE), variant of the Average Total Effect. They address network uncertainty under partial interference through structure learning and then the auto-g- satto-g-complie computation algorithm to estimate the PAOE. This method rely on the process corresponds perfectly (satisfying Markoy
assumption that the true and faithfulness conditions) to some unknown chain graph, with two restrictions. This paper does not release its code.
8. Lin et al. (2024): Under unknown interference, the UNITE algorithm uses a Graph Structure Learner (GSL) to infer the hidden interference structure by constructing a complete graph and imposing L0-norm regularization to identify significant connections. With the inferred structure, Graph Convolutional Networks (GCNs) are employed to learn aggregation func- tion and model the interference from neighboring units, effectively capturing the interference patterns and enabling accurate treatment effect estimation.
DRLeamer (Leung & Loupos, 2022) proposes doubly robust estimator and explicitly models interference within a social network. Using targeted maximum likelihood estimation and double machine learning techniques, Net-TMLEE (Ogburn et al., 2024) and GDML. (Khatami et al., 2024) rely
17

---

Published as a conference paper at ICLR 2025
on pre-specified exposure mappings and assume interference is limited to direct neighbors. Under the unknown interference graph. Sävje et al. (2021) and Cortez et al. (2022) rely on randomized controlled trials (RCTs) to study causal effects. Hoshino & Yanagi (2023) assume that the interference graph is a subgraph of the social network and relies on the random assignment of instrumental variables (IVs) to ensure valid causal inference. In observational data, Bhattacharya al. (2020) assumes that the true data-generating process perfectly corresponds (satisfying Markov and faithfulness conditions) to some unknown chain graph with two restrictions. However, practical scenarios, structure learning algorithms often face implementation issues and produce unreliable results due to the size of the graph and unmet assumptions. Although Lin et al. (2024) proposes the UNITE framework, which uses LO-norm regularization and GCNs for accurate treatment effect estimation, it only supports the estimation of AME and IME, Besides, they do not consider that peer treatments can influence the treatment choices of others in real applications. To address these issues and estimate causal effects under an unknown interference graph, we propose incorporating GCNs with cross-attention in the L-layer CauGramer embedding to aggregate L-order neighbor information.
D IMPLEMENTATION DETAILS, PSEUDO-CODE, AND ADVANCES
In this paper, inspired by the powerful Transformer (Vaswani et al., 2017), we propose an interference- agnostic Causal Graph Transformer (CauGramer) framework, which extends the computation graph of the Transformer to an L-order neighbor network and uses cross-attentie to provide a broader receptive field for GCNs (Kipf & Welling, 2016), enabling them learn interference representations. Under the generalized networked setting with Unconfoundedness (Figure 5(a)). by fully embedding peer information, the proposed balancing constraints and minimax moment constraints can provide double guarantees for causal prediction to estimate the main, peer, and total effects. However, when the Unconfoundedness Assumption is violated, the outcome regression (Eq. (I)) would be confounded by unmeasured confounders (denoted by (U)). To generalize CauGramer for settings with unmeasured confounders (UCs), we propose incorporating negative control variables (NCs) into the unconditional maximum moment optimization and integrating this into the objective function (Eq. (12)). This adaptation directly extends CauGramer to CauGramer(NNC) for such settings, Appendix D.1 describes the key definitions and assumptions of CauGramer, Appendix D.2 details its implementation; Appendix D.4 introduces the negative controls and bridge functions for Cau CauGramen(NC)) and we discuss CauGramer's 's advances and limitations in Appendices D.5 and B.
D.1 KEY DEFINITIONS AND ASSUMPTIONS OF GRAMER
In this subsection, we will clarify the definitions of confounders and potential outcomes, as well as the required assumptions in CauGramer. We start by reviewing the causal diagrams of the proposed generalized networked setting. As shown in Figure 1, in social networked data, both the self-treatment and peer-treatment of unit influence the self-outcome. We define the effect of self-treatment on self-outcome as main effect, denoted as TM (æ,, xP,) {x;, Tp, + ys. which is a function of self-features and peer-features. Next, we define the peer-treatment effect of peer on unit i's outcome as the heterogeneous peer effect, denoted as (T(x,, 22,) {x; æi} - where j E P,, which is a function of the features of unit i and peer j. Finally, we further define the potential control outcome under (t;, tr.) = (0, 0p,) as (o( æz5) fxx; pp.} + y;- The formal definitions are as follows:
Definition (Potential Control Outcomes). y(a,, xp., 0, 0p) = 150 ( (D), 2.z).
Definition (Potential Treated Outcomes). y( Tp., tú tp,) = po(a;, m.) + t. TM (#i, xp.) ΣjEP, It, (T( )-
Definition (Observed Outcomes). y; = 4 y(m,, Tp,, l4 tp,) + (1 - (.) y(.,, Tr,, O, Op).
In networked interference, the challenge is that we can only observe the outcome for the treatment assignment that was actually applied, and not for all potential treatment assignments simultaneously. This means we cannot directly observe what would have happened under different treatment distri- bution. Besides, in observational data, as shown in Figure 1(b), some variables are common causes of both the treatment and the outcome of interest. These variables confounders extrancous variables whose presence affects the relationship between the variables being studied and introduces confounding bias (Pearl, 2009) when imbalanced confounders exist.
18

---

Published as a conference paper at ICLR 2025
Definition (Confounders to Main Effect). Confounders to Main Effect variables that influence both self- treatment and self-outcome, potentially creating a spurious association that can distort the main effect estimation. In this paper, Confounders of t, and y are (p, tp,
Definition (Confounders to Peer Effect). Confounders to Peer Effect are variables that influence both peer-treatment and self-outcome, potentially creating spurious association that can distort the peer effect estimation. In this paper, Confounders of Ep, and yi are {z,, Tp..t,t.) t,).
To eliminate confounding bias for estimating main effects and peer effects, inspired by Cai et al. (2023); Guo et al. (2020); Huang et al. (2023); Jiang & Sun (2022); Ma & Tresp (2021), we propose representation balancing to adjust confounders by minimizing IPM (Eq. (10)) of treated and control groups (Johansson et al., 2016; Shalit et al., 2017):
by MSE(Ú. y) + a IPM(r, t) + B Cross_Entropyy t),
where a and 33 are hyper-parameters to trade off the representation balancing and outcome regression.
To precisely estimate treatment effects, i.e., ME, PE, and TE, we first make the standard causal assumptions on networked data (Forastiere et al., 2021; Jiang & Sun, 2022).
Assumption (Positivity). The probability of a unit with their peers receive any treatment pair (4, (p)) is always positive, i.e., < P(4 = Lp. = hp 2; Tp, < for any Ct.
Assumption (Consistency). The potential outcome is the same the observed outcome under the same self-treatment and peer-treatments, i.e., = ",,,,
tp,) for treatment pair (t,, tp,). TP,, t,
Assumption (Unconfoundedness). The self-treatment and peer-treatments are independent of the potential outcome given self and peer' features, i.e., y(æ;, Tp,, L, (p) IL (t, tp) (æ;, tP,).
Assumption (Representation). The treatment vector Lp, of peer nodes can be captured by a peer treatment function 9t, and the confounder vector Ip; by peer confounder function Ja-
As shown in Figure under unconfoundedness assumption, all confounders have been observed. When we control {æ,, PP, J. the main effect of t; on is identified; when we control {x,,, Dp,, } the peer effect of bp, on y; is identified.
Positivity assumption requires that the probability of receiving any treatment among a unit and their peers is always positive, i.e., there are no circumstances under which a treatment cannot be applied. It's fundamental to causal inference because if there are conditions where a treatment is completely unavailable, becomes impossible to draw any inference about its effects from the data. Consistency assumption states that under the same self-treatment and peer-treatments, the observed outcome consistent with the potential outcome. This assumption is also crucial in causal inference as provides the basis for inferring causal effects from observational data.
D.2 IMPLEMENTATION DETAILS AND PSEUDO-CODE
As shown in Figure 2, the overall architecture of our model consists of the following components:
The interference representation R. (x., A) of self-features and peer-features. In this paper, we propose I.-layers (default: 2) M-haads (default: 3) cross-attention GCN to learn the representation R2 = gu A). In each attention head, all neural networks consist of one layer comprising 32 hidden units. We then perform cross-attention computation, which yields the concatenation of M head embeddings. followed by feed-forward network to output a 32-dimensi representation.
The interference representation R, = (t, of self-treatments and peer-treatments. In this paper, we propose a L-layers (default: L = 1 for main results and L = 2 for unknown interference graph in Figure 3) M-heads (default: 3) cross-attention GCN to learn the representation R² = (t, A). The process for learning the treatment interference representation R, similar to that of the feature interference representation R, except that the GCN employs treatments t to mask node information in each attention. The feed-forward network then outputs a 32-dimmenla representation.
The treatments and outcomes regression networks {fe, fo First, concatenate the representations R R, 2 R. Then, we use three two-layer linear networks, where each layer comprises 64 hidden units, to regress treatments T and potential outcomes (Y, Yi).
19

---

Published as a conference paper at ICLR 2025
Algorithm 1 CauGramer: Causal Graph Transformer
Input:
The observed features Zi, treatment i,, and outcomes y4: Social network A; Hyper- parameters {a, 9, 7): ; The num of training epochs - 300; Steps (K, {150, 2}.
Representations (ga, Sr one bridge function y, one treatment estimators ft, and two potential outcome estimators {5gr fy f.]]-
Output:
Loss Function: L - (1 7) (MSE( 9. y) + a IPM( 6) + 3 Cross Entropy(i, 6)) + B[ 0)}.
Initialize networks {g 9t. 9, fo. fgg fy). for i 1,2,... do
r= (2, A), Ft = (t, A);
r Tu 3 TG;
WA = q(r t), t offi(tofgee H)), 90= (r(r) and Φ1 = (r));
ÿ = t. ÿ1 + (1-t). t..t))8. ÿo:
Cross Entropy
En [t. log (f.) + (1 t.) log (1 f.]]; IPM(r, Wass (ris, n) [r),,= MSE(Ú: y) N Σi (0i -y:)²:
if <i<Kan K or % S = then
Update (9 It, fer fg. } + AC)
else Update {Ø} + Adam(- - N En (g(ri. (.) (0i y;))); end if
end for return Networks {9z, gt, q, Scr Sg.
The bridge function W g(r; t). We input the concatenated representation R into a single-layer linear network to obtain Query and Key separately. Then, they are fed into MatMul operations to obtain the attention- based bridge function.
In the whole architecture CauGramer, we use the ReLU activation function and set the dropout rate to 0.1 to mitigate overfitting. Then, the objective function is:
arg min max L = (1 7) (MSE(Ú. v) + a- IPM(r, t) + 8 Entroppy t)) + + B[ - -)] 9
WA
where WA = q(r, t), i a (fe(( H)), 9; = . (r:) + (1 -4))))(r)))) Then, we adopt Adam optimization with a learning rate 0.01 and set epochs 300 alternately train WA A and (T. Yo. ) During the optimization of Wa. we keep the parameters the representation networks gc)()) and treatment and outcome regression networks {ft(), ff)()) (-) fixed. Conversely, we keep the parameters in g(-) fixed while optimizing the representation networks and treatment and outcome regression networks. The pseudo-code is placed in Algorithm 1.
(r:).
Hardware used: (1) MacBook Pro with Apple M2 Pro. (2) Ubuntu 16.04.3 LTS operating system with 2 Intel Xeon E5-2660 v3 @ 2.60GHz CPU (40 CPU cores, 10 cores per physical CPU, threads per core). 256 GB of RAM, and 4 GeForce GTX TITAN X GPU with 12GB of VRAM.
Software used: Python 3.9 with numpy 1.26.4, scipy 1.10.0, pandas 2.2.2, torch 2.3.0, scikit-learn 1.4.2, openpyxl 3.1.2, torch_geometric 2.5.2, torch-scatter 1.1.0.
D.3 BRIDGE FUNCTION: ENFORCING RESIDUAL INDEPENDENCE
The core intuition behind the bridge function is that it enforces independence between the residual y ý and the variables (-, l):
1. Independence (u - (xx, I. (xr, is equivalent (y ((x,))) q(x, t) for all q())
2. This cquivalence implies that Pr(y t(x,t).g(2z Pr(g(z, t)) for all q(.).
Pr(y (zz x
3. As a result, we have: Fl(y Úa, t)) q(>,, 1)) Flyy - Ú(., t)] Flg(a, t)] 0 for all 4())
4. This leads to the minimax formulation: ming maxyeq Fl((y 9) q( 1)). where:
20

---

Published as a conference paper at ICLR 2025
max detects any dependency between the residual and the covariates. ming adjusts the predictions eliminate this dependency.
"The model being correctly specified and properly optimized" is sufficient condition for "the residual independent of covariates z". Conversely, the residual is not independent {z, t), this indicates that the model suffers from misspecification. The bridge function serves to mitigate such misspecification issues by enforcing the independence the residual and the covariates.
D.4
BRIDGE FUNCTIONS WITH NEGATIVE CONTROLS FOR CAUGRAMER(NC)
Xvetr
Xpeerz
Xasts
Xpeers
NeGiveers
Treers
2
W
Truers
Taet
Yenly
Tseif
Ysefs
(n) Our Settings with Unconfoundedness
Uperr Upeers Ugtoout
Observations
Latents.
(b) Causal Dingram for Negative Controls
(c) Our Settings without Unconfourdednesss
Figure Causal Diagram for Negative Controls and Generalized Networked Settings.
However, in real applications, numerous latent variables may exist, leading to violation of the Unconfoundedness Assumption. These unmeasured variables can influence both the treatment outcome, posing additional challenges for causal inference. When this assumption is violated, negative control variables (NCs) typically are used for detection, reduction, and correction of confounding bias (Kallus al., 2021; Shi al., 2020). As shown in the causal diagram in Figure 5(b), negative control variables are variables that are not direct causes of the outcome or are not caused by the treatment, and they are becoming more accessible through web technologies, user profiles, and digitized records (Hu et al., 2023).
Definition (Negative Controls (NCs)). Negative Controls comprise two types: NC exposures (NCE, denoted by 2) and NC outcomes (NCO, denoted by w). NCE z cannot directly affect the outcome and neither NCE z nor the treatment can affect the NCO w. i.e. 1L y (x, u), z 1L (x, u). t IL w (x, u) and (z, w) J.. (z,
From the causal diagram and definition of negative controls, NCEs Z are proxy variables that linked to the treatment T through the same unmeasured confounders U, and NCOs W are influenced by the same unmeasured confounders U the outcome Y. Then, Miao al. (2018) and Kallus et al. (2021) construct a bridge function g(z, t, æ)) for the treatment regression using NCE and bridge function luw, t, x) for the outcome regression using NCO w:
E æ] E u
E 2]g(2, t, x)) (t, x u)), Eiy t. x, u] ((h(w,t,a) t, u]. (13)
The main goal of bridge functions is to learn transformation links between observed variables and negative controls to eliminate confounding effects from the same unmeasured confounders in causal identification. In the presence of unmeasured confounders, propose using bridge functions optimization to generalize the unconditional maximum moment optimization in Eq. (11). This directly extends our CauGramer to CauGramer(NC) for addressing unmeasured confounders.
Theorem 2. When negative control variables are available, the treatment effects identified from the bridge functions q(2, t, x) and A(w,& t, x) (Kallus et al., 2021).
h = arg min max E (q( t, x) (h( t) -y1- NEB
(14)
rEQ'
i = arg min max E| 1) ())(2,, -f Nww, x, | d)))(())
(15)
TEQ
KEE
where Jt is the base measure associated with treatments and HH and hypothesis spaces.
21

---

Published as a conference paper at ICLR 2025
The proof can be found in Kallus et al. (2021). In CauGramer, design cross atts-atttntionn moment constraint through a minimax game to make residual independence from input features:
y arg min max (ig(r, (y 9) g(r,t) Sigmoid (Q K'),
qeQ
The negative control variables (2, w) can be directly embedded into our maximum moment optimiza- tion in the above moment constraint and generalized it follows:
y" arg min max (lg(r(z, æ) (y j(r(w, x), t))l ((((z, x), = Sigmoid (Q K'), (16)
This allows to transform the moment optimization into bridge optimization problem, thereby generalizing CauGramer to CauGramer(NC) and enabling it to address unmeasured confounders without incurring additional computational costs.
D.5 ADVANCES OF CAUGRAMER
In networked data, traditional Graph neural network (GNN) and its variants (e.g., GCN) often face limitations related to the size of the receptive field and suffer from information homogenization.
Receptive Field Size: In GNNs, the receptive field refers the scope of neighboring nodes or edges that influence the representation of a unit. As the network deepens, the receptive field may not expand sufficiently capture information from distant nodes or edges, leading to limited attention and limited receptive field.
GNNs can lead to the problem of information homogenization, where the representations of different nodes become increasingly similar as they pass through multiple layers. This can cause loss of important structural or relational information present in the graph.
Information Homogenization: Deepening
These constraints pose challenges for traditional GNNs and GCNs in capturing long- interfer- ence information within graph topologies. Consequently, conventional network interference studies often only account for interference among first-order neighbors (Cai et al., 2023; Forastiere et al., 2021; Guo et al., 2020; Huang et al., 2023; Jiang & Sun, 2022; Lee & Lee, 2022; Liu et al., 2016; 2023; Ma & Tresp, 2021; Qu et al., 2021; Veitch et al., 2019; Zhao et al., 2025; 2024).
Transformers provide a broader perspective to GNNs, overcoming the traditional GNN limitation of receptive field size and enabling the capture of distant node information through their attention mechanism. While recent Transformer variants have achieved superior performance in graph learning (Rampášek al., 2022; Sui al., 2022; Ying et al., 2021), they rely on modeling full graph attention through self-attention, which models attention (Query, Key and Value) within the self-embedding. which poses further issues:
Using embedding from GNNs input for Transformers, the self-attention mechanism may still struggle to capture the long-distance information of the graph.
By modeling self-attention mechanisms only on the self-embedding of each unit, these models may fail to capture networked interference.
Storing and processing the attention weights for every pair of nodes or edges in the graph requires substantial memory resources.
Modeling full graph attention using self-ateention mechanisms incur substantial compu- tational costs, particularly for large graphs.
These works still suffer from the limited receptive field and high memory and computation costs, and they cannot address the networked interference problem. To address these issues, we propose an interference-agnostic Causal Graph Transformer (CauGramer) framework, which extends the computation graph of the Transformer to an L-order neighbor network and uses cross-altention to provide a broader receptive field for GCNs (Kipf & Welling, 2016), enabling them to learn complex- sequential interference representations. As shown in Figure 2, in each embedding layer, construct cross-attention to learn Query q, Key k, and Value v, from self-features (æ,)), neighbor- (arp,), and combined features (æ,, 2p,) respectively, rather than just modeling self-attention only
22

---

Published as a conference paper at ICLR 2025
each unit's own embedding. This allows us capture interactions between nodes within and beyond the local neighborhood, rather than focusing solely on each unit's 's own embedding.
In the L-layer CauGramer embedding with M-head attention, neighbor information and topology structure, while cross-attentioo provides a broader receptive field in GCNs for learning complex-sed seeeentilll interference representations. As the network's 's depth L increases, the scope of our GCNs with cross-attention expands, enabling the embedding of all potential interference peers into representations R2 and R. Furthermore, we reformulate the Transformer decoder with cross-altention minimax constraints to precisely estimate the main, peer, and total effects. When negative control variables are available, the minimax constraint the objective function (Eq. (12)) of CauGramer acts as the bridge function in Theorem 2, thereby empowering CauGramer to generalize effectively and address the challenges posed by unmeasured confounders.
GCNs to aggregate L-order
E DATASETS, SUPPLEMENTAL EXPERIMENTS, AND HYPER- SEPEE--PANE PARAMETERS
In this section, we introduce the dataset details used in this paper, conduct mization analysis (Figure 6 and Table 5), and time-cost analysis (Table 6). Additionally, we present supplementary experiments to demonstrate the robustness of CauGramer under various scenarios, including interference strength (Figure 7), interference functions (Table 7), unmeasured confound- ing (Table 8). Furthermore, we include comparisons against additional baselines, including graph Transformers and conventional networked methods, in Tables 9 and 10.
opti-
E. DATASETS DETAILS
In real-world applications, the ground-truth causal relations are unknown, and thus it impossible to evaluate the causal estimation performance of CauGramer directly. Therefore, following existing works (Veitch et al., 2019; Jiang & Sun, 2022: Guo al., 2020; 2021: Ma al., 2021; Cai al., 2023), we generate semi-synthetic data from BlogCatalog (BC) and Flickr datasets 3, where the features (x) and social networks (A) are real, while treatments (t), outcomes (y), and interference (E) are simulated:
BlogCatalog (BC) (Guo et al., 2020; Jiang & Sun, 2022). BlogCatalog (BC) is an online community that offers blog services, where each blogger represents study unit within the dataset. The relationships between these units form the social network, with each edge denoting social link. The features are bag-of-words representations of keywords in the bloggers' descriptions.
Flickr (Guo al., 2020; Jiang Sun, 2022). Flickr is an online social network providing image and video sharing services. The dataset is constructed by forming links between images that share common metadata. In this dataset, each instance is a user, and each edge represents the social relationship between two users. The features of each user represent list of tags indicating their interests.
We use the same linear discriminant analysis (LDA) and the same data simulation by Jiang & Sun (2022) on BlogCatalog and Flickr datasets. Moreover, Jiang & Sun (2022) use METIS (Karypis & Kumar, 1998) to partition the original networks into three sub-networks as train/valid/ttes data with 2482, 2461, and 2358 samples in Flickr, and 1784, 1716, and 1696 samples in BlogCatalog. In the testing data, we will randomly sample binary treatments from a Bernoulli distribution with probability of 0.5 to simulate manipulation operations. Then, we use the same data simulation by Jiang& & Sun (2022) evaluate CauGramer's performance original data with known intervention graphs and summary functions for constant treatment effects. Then, we design a series of generalized heterogeneous treatment effects datasets with unknown summary functions across various settings.
For the conventional networked setting with known interference graph and known summary function (Table 2), similar Jiang & Sun (2022), we generate the treatments and potential outcomes with known intervention graphs (A = E) and summary functions (2, [. ΣjEN, ty). as follows.
Let p a (w P'x WT ESEN,
a (w!? .x). 1 Ejen, P where A.
'hhe BlogCatalog and Flickr datasets are available at: https: http://...oo/oooogoooooo.. Causal- Infeencce-eee on- Networked-Daaa.
23

---

Published as a conference paper at ICLR 2025
{wzz wx random coefficient vectors sampled from uniform distribution (1-1100,11) 1.0).
(1) (2)
f = 1(p + PM > PN.)
Σ
=
+.5p'x + +
(17)
JEN.
where 1 is indicator function, P and Piv, denote the average value of pt and pig:" The variable (0, 0.1) denotes a measurement enror.
For the generalized networked setting with known interference graph and unknown summary function (Table 3), under interference graph E, we use the same functions by Cai et al. (2023); Jiang Sun (2022) to define:
p = ow x.),
pb.
p).
p = o(w -x)), Ppp
1
Σ
P
P
jEP.
jEP:
2
NCE ((w xx.)
1
a aw (3)
xz), wNoo o(w xxi)
1
Σ
o zz)):
(w
IP
P
jEP,
jEP;
where { (wa wT w W (4)
are random coefficient vectors sampled from uniform distribution NCE and w. NcO are NCs. Then, generate treatments based the priority level
(2)
4(-1.00 1.0), 2,
8: = 2; + x.
(1)
(2)
where v; and x;
(2) denotes the 1st and 2nd variable in the feature of únit
m: pl + pp,,, b m
tj 1 8:), 4 1 m.
(18)
P
jEP.
In the treatment generation process, we simulate treatment interference based on the priority level s; to avoid cycles. Then, we simulate outcomes with heterogeneous effects under networked interference.
1/4 4 T/ IME
TT.T
IPE +p
0.5p%;
64:
(19)
IME Ti
v 1 p. -P'
IPE
1
tj
xz 12
(20)
P
jEP,
where ||,; xx denotes the squared Euclidean distance, and p E (0.2, 0.4, 0.6, 0.8, 1.0, 1.5, 2.0) represents the different interference level in the Figure 7.
Moreover, to simulate an unmeasured confounder setting (Table 3), we will randomly select two variables as latent variables and remove them in the feature unit In Table CauGramer(Ul (UC) directly employs models with missing variables to estimate treatment effects, while CauGramer(NCC) uses {2NCE NgNCO) as NCs identify treatment effects.
In the generalized networked data with unknown interference graph and unknown summary functions (Figure 3), we design three interference graphs E which are not the same as the social network A. 'SN and 'S.G.2N" represent subgraphs from graphs composed of 1st and 2nd-
order neighbors marked by Euclidean distance (0.1 (155 x 11 < 0.2), respectively, while
(:5)
'Random' denotes randomly generated graphs by Euclidean distance (0.20 < ||>: (..)) x (:5) 1/ 0.25). Additionally, for comparison, we denote the 'Known' graph as the social network. Then, we use these interference graphs to simulate data by the above simulation process (Eqs. (18) and (19)).
Then, in the limited budget experiments (Figure 4), treatment sources like COVID vaccinations are limited, requiring treatment assignment to adhere to varying budget constraints ('BudgX%) X%)) rather than predefined treatment assignment policies ('Assigned'). Given the training data generated by equations (18) and (19), in testing data generation, we randomly sample binary treatments from Bernoulli distribution with probability of (20%, 50%, 80%, 100%) to simulate different budget on treatment sources. This enables us to evaluate the performance of the proposed CauGramer in estimating treatment effects under varying budget constraints.
For different interference levels (Figure 7)), we adjust the coefficient p ranging from (0.2, 0.4, 0.6, 0.8, 1.0, 1.5, 2.0) in equation (19) simulate various levels of interference.
24

---

are a, 8,7 = 0.8, 0.4, 0.2 for both Published as a conference paper at ICLR 2025
Evv on ATE Estimation with varying {e,
FvvG on ATE Estimation with varving Y
1.0
0.097
0.060
0.122
0.088
0.024
0.024
0.15
0.279
0.279
Param, B
0.2 0.6
0.100
0.215
0.125
0.347
0.103
0.076
0.059
0.030
0.026
0.030
0.024
0.018
0.10
CAVG
0.2
0.309
0.129
0.138
0.052
0,025
0.015
0.134
0.173
0.169
0.218
0.053
0.027
0.017
0.05
0.10.51
0.051
0.225
0.151
0.145
0.055
0.028
0.018
0.025
0.032
0.00
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Hyper-Paramettrr
Hyper-Parametery y
(EEEEE ITE Estimation with varying {a, [3
/EPEHE on ITE Estimation with varying y
0.670
1.0
0.202
0.150
0.234
0.178
0.131
0.129
0.25
0.6
0,597
Param, B
0.2 0.6
0.305
0.209
0.217
0.399
0.224
0.163
0.142
0.156
0.130
0.129
0.129
0.130
0.20
VEPBHE
0.489
0.4
0.384
0.220
0.249
0.164
0,,125
0.129
0.15
0.282
0.278
0.371
0.149
0.127
0.134
0.383
0,258
0.264
0.150
0.128
0.130
0.10
0.20.160
0.125
0.144
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Hlyper-Purametee a
Figure 6: Hyper- Pareer-arrameter Optimization on Flickr Dataset: CauGramer optimizes three hyper- parameters, 8,, 7) } & (0.0, 0.2, 0.4, 0.6, 0.8, 1.0]. The minimum regression error on the validation data indicates the optimal hyper-parane hyper-parameters (fa, B, 7} f0.8, 0.4, 0.2}) for ITE estimation, marked by the red dashed box and lines. A higher value on a might improve ATE estimation performance.
E.2 PYPER-PARAMMTTES OPTIMIZATION
We first review the objective function (Eq. (12)) of the proposed CauGramer:
arg min max C = (1 7) (MSE(). v) a -.-IMM( () 4 8- Cross Entropy)t)))) + B/y(,,r,, t) 9)).
v
In our CauGramer, there are three Among them, and /3 jointly control representation balancing for adjusting confounding bias in the estimation of main treatment effects and peer-treatment effects. Additionally, 3 serves for aggregating neighbors' representations by maximizing the predictive power of TyJEP, on self- treatment 4,. Through learning representations with reduced Inter-Profile Matching (IPM) distance between treated and control units, a and /3 facilitate a bias-variance trade-off. The Y serves interpolation coefficient, where 7 = 0 corresponds to outcome regression with representation balancing, and y = represents a conditional moment model. By adjusting y, we integrate the Conditional Moment model into the objective function, providing dual guarantees to mitigate confounding bias effectively.
i.e., fa. E 0.4.0..2,4440666..0 1.0).
In this paper, we adopt the minimum regression error on the validation data determine the optimal hyper-parameters {a, 7) E (..0, 0.4, 0.6, 0.8, 1.0). First, We search for a, E 0.0, 0.2, 0.4, 0.6, 0.8, 1.0 identify the hyper-parameter combination corresponding the minimum validation error, while fixing y = 0.2. Subsequently, utilizing the optimal parameters ,3 determined from the previous search, further explore 7 0.0, 0.2, 0.4, 0.6, 0.8, 1.0 to identify the hyper- parameter combination corresponding to the minimum validation error. Considering the Flickr dataset the
as an example, as depicted Figure 6, we determine the hyper-paramette parameters that correspond smallest validation regression error, which also indicates the smallest /EPEHE on the Flickr dataset, marked by the red box and lines. The optimal the BlogCatalog and Flickr datasets.
Furthermore, as shown in Table 5, we can use a grid search on the validation dataset to choose the optimal L value and M value in CauGramer with L-layer and M Mllayer. In this paper, we propose a L-layers (default: 2) M M-heads (default: 3) cross-attention GCN to learn the representation R. In (x. A). For interference information, we propose L-layers (default: L = 1 for main results and L 2 for unknown interference graph in Figure 3) M -heads (default: 3) cross-attention GCN to learn the representation R2 (t, A).
25

---

Published as a conference paper at ICLR 2025
Table 5: Hlyper-Parameee Optimization on Flickr Dataset: CauGramer with L-layer and M -headds.
Flickr
AME
APE
ATE
IME
IPE
ITE
T=l, M=
0.03910.02
0.32010.10
0.290_o.00
0.06610.02
0.358 _..00
0.339
L=2, M=2
0.09340.05
0.230=0.11
0.203
0.137+0.07
0.282-0.00
0,281
L-2, M=3
0.07440.05
0.223
0.203
0.11440.07
0.270 0.222
1 M=4
0.048
0.260=a.09
0.228
0.083
0.305
0.288
L=3, M
0.11210.07
0.437 10.08
0,458
0.16610.00
0.467 _9.07
0,488 1088810000
Table 6: Training Time(s) of Various Methods in A Single Execution on Both Datasets.
Method
BlogCatalog
Flickr
Method
BlogCatalog
Flickr
CFRNet
20.9s
35.3s
DRlearner
29.7s
33.9s
NetDeconf
53.9s
101,6s
G-HSIC
508.75 7s
1054.3s
SPNet
460.2s
815.8s
CAL
108.8s
204.1s
Graphormer
788.5s
1225.4s
RRNet
628.4s
1119.1s
CEVAE
74.0s
106.5s
NetEst
1226.5
2125.15
GraphGPS
66.4s
124.4s
SAT
104.3s
225.7s
GDML
8.4s
24.4s
UNITE
76.38
111.7s
CNE
66.5s
105.9s
CauGramer
79.7s
155.5s
E.3 TIME COST ANALYSIS
For each experiment, conduct 10 independent replications to analyze the average running time (in seconds) for the proposed model single execution and compare it to baselines. From the results in Table 6, observe that among all transformer-based methods (CAL, SPNet, Graphormer), CauGramer exhibits the shortest running time (79.7s on BlogCatalog data, 155.5s on Flickr data). Unlike traditional Transformer variants, which rely on computationally expensive full graph attention mechanisms through self-attention, CauGramer adopts more streamlined approach. By redesigning the computation graph of the Transformer, CauGramer effectively models sequential-struee struuuurrrs inter- ference representations using GCN with aateetttten,on,, bypassing the need for full graph attention through self-attention. Moreover, compared the optimal baseline NetEst, CauGramer achieves notable speedup. being 12 times faster. This highlights CauGramer's performance and computational efficiency in graph-based modeling tasks.
E.4 SUPPLEMENTAL EXPERIMENTS ACROSS VARIOUS INTERFERENCE FORMS
The experiments presented Tables 2 and 3, as well as Figures 3 and 4, the main text, have demonstrated the superiority of CauGramer over existing methods, whether in conventional networked settings or in generalized networked settings with or without unmeasured confounders, and under limited budget constraints. To simulate additional experiments on varying interference levels E (0.2, 0.4, 0.6, 0.8, 1.0, 1.5,1.5,.00 2.0}.
complex real-world applications, further conduct
We report the estimation errors of Average Total Effect (ATE) and Individual Total Effect (ITE) of treatment on BlogCatalog and Flickr datasets across varying interference levels E (0.2, 0.4, 0.6, 0.8, 1.0, 1.5, 2.0). As shown in Figure 7, the interference level increases, the pro- posed CauGramer algorithm consistently outperforms existing methods. When NCs are accessible, CauGramer(NC) effectively eliminates unmeasured confounding bias, approaching the performance of CauGramer trained on full data. For causal effect estimation, we notice that all baselines' errors increase with higher interference levels. However, CauGramer and its variant CauGramer(NC) exhibit superior performance in predicting the individual total effects of treatment. Moreover, in ATE estimation, both CauGramer and CauGramer(NCC) remain robust, with errors consistently below 0.1 or 0.05 on the BlogCatalog and Flickr datasets. This demonstrates that our CauGramer can learn complex intervention representations. By fully embedding peer information, the balancing constraints and minimax moment constraints provide accurate causal prediction. Consequently, even under large interference, the proposed CauGramer maintains low estimation errors, showcasing its causal robustness and effectiveness in mitigating confounding bias.
26

---

Published as a conference paper at ICLR 2025
Competing Methods:
CTRNet
SPNet
NetEst
CauGramer(NC)
CanGramer
ATE Estimation
PSrEEE ITE Estimation
ATE Estimation
ITE Estimation
0.3
0.4
0.4
0.3
0.2
0.3
0.2
0.2
0.2
0.1
0.1
0.1
0.2 0.4 0,6 0.8 1.0 1.5
2.0
0.2
0.4 0.6 0.8 1.0 1.5 2.0
0.2 0.4 0.6 0.8 1.0 1.5 2.0
0.2 0.4 0.6 0.8 1.0 1.5 2.0
Interference Level
Interference Level
Interference Level
Interference Level
BlogCatalog Dataset with Varying Interference Level
Flickr Dataset with Varying Interlerence Level
Figure 7: Results (<AVG and (PPEHE) of Total Effect Estimation on BlogCatalog and Flickr Datasets across Varying Intervention Level {0.2, 0.4, 0.8, 1.0, 1.5.
Table 7: Results of the Proposed CauGramer across Various Interferences: (1) aggregation interference
Jiang & Sun (2022) (Eq. (17): 7 M.. EjeN, 4)) (2) interaction interference (Eq. (20): T;
PPT EjEp, 1 * Im: 2 11)) (3) external interference (Eq. (21): = PP.1 EjEP, sin(t, | 12)))
BC
Interference
AME
APE
ATE
IME
IPE
ITE
SPNet
Eq. (17)
0.06640.01
0.223
0.203_o.10
0.100 0.213+0.09 0.230
NetEst
Eq. (17)
0.07640.02
0.078
0.065_o.03
0.099
0.11740.01 0.092+0.01
CauGramer
Eq. (17)
0.05440.03
0.034
0.039_0.04
0.075
0.04440.02
0.063
SPNet
Eq. (20)
0.083
0.243 0.212
0.131
0.246
0.279
NetEst
Eq. (20)
0.08940.03
0.077
0.109_0.00
0.147 20.03
0.145
0.173
CauGramer
Eq. (20)
0.073
0.057
0.04
0.090
0.118
0.125
SPNet
Eq. (21)
0.273 0.12140.05 06
0,273_o.10
0.17922.08
0.283
0.28440.10
NetEstimator
Eq. (21)
0.083
0.064
0.072_0.02
0,121 #0.05
0.078
0,10840..0
CauGramer
Eq. (21)
0.06340.07
0.039
0.052_o.03
0.108
0.051
0.07440.08
Flickr
Interference
AME
APE
ATE
IME
IPE
ITE
SPNct
Eq. (17)
0.05440.03
0..01
0.131_0.05
0.079
0.131 +0.05
0.142
NetEst
Eq. (17)
0.003
0.005
0.073_o.00
0.095
0.070
0. 0.105
CauGramer
Eq. (17)
0.028
0.019
0.032_
0.047
0.032
0.051
SPNet
Fq. (20)
0.096
0.203 0.166
0.147
0.213
0.252
NetEst
Eq. (20)
0.06940.05
0.067
0.123_n.07
0. 0.136=0.06
0.156
0. 0.00
CauGramer
Fq. (20)
0.05844.05
0.030
0.025_a.03
0.096_00.0
0.111 +0.01
0. 0.125
SPNet
Eq. (21)
0.088 04
0.204_o.04 0.202
0.136
0.219
0. 0.223
NetEstimator
Eq. (21)
0.06040.01
0.073
0.073_o.04
0.102
0.093
0. 0.11440.02
CauGramer
Eq. (21)
0.057
0.010
0.046_o.01
0.097 20.03
0.040
0.086
Additionally, in Tables and 3, simulate two different forms of interference using Eqs. (17) and (20). Eq. (17) represents aggregation interference, commonly used in traditional networked interference works (Cai et al., 2023; Jiang Sun, 2022; Ma Tresp, 2021). Eq. (20) simulates heterogeneous interaction interference, based on the difference between self- and peer attributes within the k-hop subgraph. In this section, we simulate an external interference using only peer information within the k-hop subgraph, mimicking constant external influences:
IPEE Ti
Σ ty JEP.
1
(21)
P
We integrate experiments across various interference forms (Eqs. (17), (20), and (21)) into Table 7. Under various interference forms, the proposed CauGramer remains the best method for estimating the main effects, peer effects, and total effects of treatment.
27

---

Published as a conference paper at ICLR 2025
Table 8: Results of the Proposed CauGramer under Various Unmeasured Confounders (UCs). Differ- ent subscripts on [...] denote masked variables, while NULL indicates all variables are observable.
BC
UCs
AME
APE
ATE
IME
IPE
ITE
CauGramer(UC))
0.128
0.073
0.09440.04
0.131 0.182
0.166
CauGramer(UC
1110000771
0.111
0.080 0.07440.03
04
0.125 0.167=0.10
0.157:20.08
CauGramer(UC)
2,6,7,8
0.111
0.07440.04
0.087 +0 06
0.163
0.127
0.1622:0.04
CauGramer(UC)
[[,,9]
0.109
0.077 0.067
0.159
0.125
0.149940.03
CauGramer(NC)
0.092.2006
0.068
0.057 1..04
0.134
0.122
0.134410.00
CauGramer(NC))
#11,3,5771
0.099=0.06
0.061 0.047=0.08
0.061
+0.04
0.148
0.109
+0.03
C))
86,,7,8.9
0.085
0.057
1.002
0.138 10.10
0.123
10.08
CNuGramer(NN)
[[6,9]
0.09222007
0.055
0.047 1.1.0.0.000
0.139
0.117
0.129
SPNet
NULL
0.083
0.131 0.131 0.2433 0.21224.11
+0.07
0.246640.10
0.279
NetEst
NULL
0.07710.02 0.089
0.109 os
0.147 10.03
0.145
0.173
CauGramer
NULL
0.073 06
0.057
0.04511.04
0.090
0.118
0.1 0.125
Flickr
UCs
AME
APE
ATE
IME
IPE
ITE
CauGramer(UUC)
0.082.2004
0.055.0.03
0.080 04
0.125
0.153 0.124
CauGramer(UUC)
111.3.5.71
0.096g0004
0.06224.01
0.065
0.130 0.140
0.155
CauGramer(UCC
17..7,8.9
0. 0.153
0.071 0.063
0.220
0.127
0.159
CUUGramer(UC)
[[1.9]
0.091
0.067
0.05610.04
0.141 10.08
0.128
0.150
CauGramer(NC)
0.081 81..66.7.8.8
0.0354..02
0.051 4.0.02
0.124
0.114
0.140
CauGramer(NC)
111.00071
0.063
0.00844.02
0.062+0.06
0.101 +0.03
0.115
0.146440.04
CauGramer(NC)
T6,77,89
0.067=0.03
0 0.034
0.043
0.110
0.113
0.134::0.01
CauGramer(NC)
[[8.9]
0.073_0000
0.038
0.032 +082+00.0 03
0.107 20.06
0.112
0.1277+0.01
SPNet
NULL
0.096=0006
0, 0.166
0.203
0.147 :0.07
0.213
0.252.0.07
NetEst
NULL
0.069
DJ06671..0
0.123
0. 0.136
0.156440.0
0.185
CauGramer
NULL
0.058
0.030.zo.o
0.025+0.os
0.09620.06
0.111
0.125
E.5 SUPPLEMENTAL EXPERIMENTS ACROSS VARIOUS FORMS OF LATENT VARIABLES
In the presence of unmeasured confounders (UCs), we propose using bridge functions of negative control variables (min max Elg(z, 2, æ, (y G(w, , x, t))] in Eq. (14)) to generalize the unconditional maximum moment optimization in Eq. (11) as min max Elg(r(x:), t) (y (r((x), t))] This extends our CauGramer to CauGramer(NC) for settings with latent variables. Then, we mask different variables (2[1899: 7.6.7,8.9) 8.9] 1,,,,,,77] "11,3, from the observational data, where the subscripts x ..... denote masked variables, to simulate various unmeasured confounders settings. The results in Table 8 demonstrate the superiority of CauGramer(NC), Even with more than five latent variables, the estimation error of CauGramer(NC) remains lower than that of the two most competing methods, SPNet and NetEst.
E.6 SUPPLEMENTAL EXPERIMENTS WITH MORE GRAPH TRANSFORMER BASELINES
We further conduct comparisons with four competing graph Transformer methods, including SAT (Chen et al., 2022), GraphGPS (Rampášek et al., 2022), CAL (Sui et al., 2022), and Graphormer (Ying et al., 2021), and reported evv of AME, APE and ATE and /EPEHE of IME, IPE and ITE on the BlogCatalog (BC) and Flickr datasets in the Table
SAT (Chen et al., 2022) incorporates structural information into self-attention by extracting a subgraph representation rooted at each node before computing attention.
GraphGPS (Rampášek et al., 2022) proposes a hybrid MrNN*Transformer architecture, in which we use GCNs to support the MPNNs.
CAL (Sui et al., 2022) designs two soft mask graphs for node attention and edge attention and conducts implicit backdoor adjustment for graph tasks.
Graphormer (Ying et al., 2021) integrates the encoder of Transformer (Vaswani et al., 2017) into node, edge, and graph embedding learning for graph tasks.
28

---

Published as a conference paper at ICLR 2025
Table 9: More Baselines Combining Transformer and GCN, For the sake of fairness, we add deconfounder module (+Deconf) to these frameworks for treatment effect estimation.
HloCCatalog(BC)
AME
APE
ATE
IME
IPE
ITE
Graphormer+Deconf
0.06840.02
0.365
0.362=00.8
0.153
0.3944u.o2
0.418
CAL+Deconf
0.15120.03
0.308
0.364:10.07
0.230
0.333
0.385
StaphGPS+Deconff
0.09110.09
0,195
0.247
0.142 10.12
0.22410.17
0.279
SAT+Deconf
0.176.0.15
0.102
0.188 20.15
0.251
10.17
0. 0.155
0.305
CauGramer
0.073+00.0
0.057_n.00
0.045_n.m
0.090
0.118 02
0.125
Flickr
AME
APE
ATE
IME
IPE
ITE
Graphormer+Deconf
0.05510.00
0.421
0.388
0.112
0.482+0.02
0.465
CAL+Deconf
0.109=0.04
0.340
0.365
0.177
0.388:0.00
0.402
GraphGiSSDDeconf
0.138
0.261
0.325
0.199
0.307
0.362.40.000
SAT+Deconf
0.089 20.0s
0.21 +0.05
0.178
0. 0.168
0.259
0.294 +0.1 18
CauGramer
0.058
0.030_nn.o
0.025
0.096=0.06
0.111+0.01
0.125 01
Table 10: Recent Baselines Networked Datasets.
AME
APE
ATE
IME
IPE
ITE
DRLeamer
0.19620.07
0.18320.00
0.099
00.07
0.544_o.03
0.216 0.03
0.529_o.oz
GDML
0.166
0.371
0.537 20.09
0.265
0.411 _0.0
0.607
GDML w/o FSet
0.069
0.34420.04
0.412:20.04
0.152_0.04
0.384_0.04
0.471
UNITE
0.06910.00
0.345520.01
0.263=20.00
0.1941=0.0
0.3466oo.os
0.2699 -0.01
CauGramer
0.073.2000
00.04
0.045
0.090 00.09
0.118 20002
0.125
Flickr
AME
APE
ATE
IME
IPE
ITE
DRLearner
0.135
0.216
0.131 +0.07
0.529
0.2661 -0.02
0...17=0.02
GDML
0.239
0.381=0.02
0.620
0.359
0.448-0.02
0.716_0.00
GDML w/o FSet
0.067
0.363
0.428
0.174_0.03
0.426_o.07
0.511=0.08
UNITE
0.043
0.335=2.01
0.297
0.212_o.o0
0.335_n.01
0.287
CauGramer
0.058 20.05
0.030=o.08
0.025=n.0a
0.096=n.08
0.111=n.01
0.125=n.01
For the sake of fairness, we add a deconfounder module (+Deconf) to these frameworks for treatment effect estimation. The results in Table 9 demonstrate the superiority of CauGramer, which achieves the best performance across all metrics on both datasets.
E.7 FURTHER DISCUSSION ON RECENT BASELINES ON NETWORKED DATA
In this subsection, we further discuss the comparisons with four competing non-i.i.d. methods on networked data, including DRLearner (Leung& & Loupos, 2022), GDML and GDML w/o FSet (Khatami et al., 2024), and UNITE (Lin al., 2024). Then reporte EvvG of AME, APE and ATE and VEPEHE of IME, IPE and ITE on the BlogCatalog (BC) and Flickr datasets in the Table 10.
(1) DRLearner performs well for population-level estimates but exhibits significant errors in individual-level estimates. This is because Doubly Robust methods were originally designed to address confounding bias at the population level, rather than to capture fine-grained individual-level heterogeneity. (2) GDML relies on a focal set where all units' neighborhoods are non-overlapping. To construct the focal set, we randomly select a node, remove it along with its second-order neighbors, and repeat this process until no more nodes can be added. However, in densely connected graphs, the focal set contains very few nodes. On the two datasets we used, the focal set for GDML included only about 100 nodes, making it difficult to achieve accurate causal effect estimation. (3) To address this limitation, we provided an additional baseline, GDML w/o FS, which does not use a focal set and instead encompasses the entire dataset. While GDML w/o FS performs well in estimating the average main effect, it falls short on the other five metrics compared to our proposed CauGramer. (4) The UNITE provided by Lin et al. (2024) only supports the estimation AME and IME. While UNITE shows a slight improvement over method in terms of the Average Main Effect (AME), its performance significantly drops at the individual level. In contrast, our method consistently excels across all six metrics, demonstrating more balanced capability in capturing both average and individual-level effects. These results
From Table 10, we have the following observations:
29

---

Published as a conference paper at ICLR 2025
underscore the strengths of CauGramer, particularly its ability to handle complex individual-level heterogeneity and interference effects while maintaining superior performance across all metrics.
P.S. UNITE (Lin et al., 2024) uses all units features X and treatments T to learn a Graph Structure Learner (GSL). then employs two learned outcome predictors to predict treated and control outcomes. If the inputs are changed, the learned graph structure and its associated weights will also change, leading to potential additional bias in the fixed outcome predictors without further assumptions. These results show that UNITE struggles with peer effects (APE/IPE) and total effects (ATE/TTE) due to changes in the graph structure and its weights, which likely render the fixed outcome predictors ineffective. Consistent with the original UNITE (Lin al., 2024) paper, which focuses on average treatment effects and individual treatment effects, we report only AME and IME for UNTTE in the main text.
F SUPPLEMENTARY RELATED WORK IN NETWORKED INTERFERENCE
Unmeasured Confounders. Our work is related the unmeasured confounders. As shown in Table 1, CNE (Veitch et al., 2019), NetDeconf (Guo et al., 2020), and SPNet (Huang et al., 2023) only consider partial unmeasured confounders embedded in the structural network and unit information, and cannot address the missing key variables, which can influence both treatment and outcomes, within self-features or peer-features. our CauGramer, we incorporate bridge moment constraints to reduce confounding effects. Moreover, when negative controls are available (Miao al., 2018; Hu et al., 2023; Shi et al.. 2020), CauGramer can extend to address unmeasured confounders by bridges.
Partial Interference. The partial interference, characterized by partition of units into numerous disjoint clusters (e.g.. households), with interference confined to units ("neighbors") within the same cluster, was initially explored in the works of Halloran & Struchiner (1995); Sobel (2006); Hudgens & Halloran (2008) under random treatment assignments in two-stage randomized experiments. Following this, a series of works account for nonrandom treatment assignments observational data under partial interference (Barkley et al., 2020; Liu et al., 2016; Qu et al., 2021). Rather than utilizing the framework employing the a-allocation strategy, Qu et al. (2021) define the estimands under the conditional exchangeability framework and derive the semi-parn cemi-parametric efficiency bound. In our CauGramer, we relax the partial interference assumption and study heterogeneous interference effects under an unknown interference graph.
Exposure Mapping. This paper is also related exposure mapping, which another general way to deal with network interference through experimental designs or treatment assignments (Aronow & Samii, 2017: Tortu et al., 2020). Aronow & Samii (2017) the first to introduce the exposure mapping framework, pioneering the estimation of average unit-level causal effects from randomized experiments under networked interference. The exposure mapping relates treatments assigned to exposures received based inverse probability weighting (Aronow & Samii, 2017). Unlike traditional methods relying on randomized trials (Aronow & Samii, 2017; Owusu, 2023; Yuan & Altenburger, 2023), Tortu et al. (2020) extend the exposure mapping framework about causal inference under network interference in observational studies, allowing for multi-valued treatments and weighted interference networks. While Tortu al. (2020) proposes a parametric estimation approach, our CauGramer automatically model unknown interference effects.
Unknown Network Structure. The most related work to our paper are those by Yu et al. (2022); Cortez et al. (2022). Under unknown network structure, Yu et al. (2022) propose simple estimator and an efficient randomized design for total treatment effect estimation under heterogeneous linear interference model (Eckles et al., 2017). Building on this, Cortez et al. (2022) generalize their findings to polynomial models. this paper, we propose a CauGramer algorithm to estimate the main effect, peer effect, and total effect of treatments in observation networked data under unknown network.
30


Reviews:
[{"review_id": "aL9gjPeuHF", "reviewer": "Reviewer_EAHD", "summary": "The paper addresses causal inference in situations involving interference, where both the interference graph and the summary function are unknown (note that the interference graph is distinct from a social network). The authors assume that interference occurs within an \nL-order neighbor network, where peer information can be aggregated using a graph transformer. They utilize a cross-attention mechanism to capture complex sequential interference representations. Additionally, they apply techniques such as regularization to balance the distribution of confounders across control and treatment groups, and they leverage moment conditions to account for confounding and mitigate model misspecification.", "questions": "Could you provide some additional justification or intuition for how the bridge function makes the residual independent of x?\n\tLine 377: You mention, “If the model is correctly specified and properly optimized…” Could you elaborate on how you ensure the model is both correctly specified and optimized?\n\nAlso, why is it necessary for the residual to be independent of t?\tSection 4.3 could benefit from a bit more clarity in its presentation.\n\nMany of the baselines, as far as I’m aware, assume a fixed and known exposure mapping. In your experiments where the interference graph is unknown, or the graph is known but the summary function is not, could you clarify how comparisons are made with these baselines that assume the exposure map is already known?\n\nline 414 and 415: In the presence of unmeasured confounders, we will collect negative control variables and embed them into the Eqs. (11) and (12)\n\nCould you elaborate on this aspect of your technique? line 481: We conduct 10 replications to report (mean±std) results. Given that this is a relatively small number of trials, could you provide some reasoning behind this choice?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper has a well-defined research focus on causal inference under interference with unknown interference structures.\n\nThe experiments conducted are engaging and effectively demonstrate the proposed methods.\n\n Overall, the paper is well-organized and presented.\n\nA few sections lack clarity, as noted in my specific questions.", "weaknesses": "The paper’s primary contribution—clearly distinguishing the interference graph from the social network—is highly valuable. However, if the approach is limited to neighboring nodes only, it risks undermining this motivation. On the other hand, if this limitation is not imposed, the computational complexity becomes quadratic in the number of nodes when using transformers, which poses feasibility issues for large graphs.\n\nAdditionally, a category of related work on non-iid data, particularly doubly robust estimators, IPW, and representation-based methods, appears to be missing from the review. Including this literature [1, 2, 3] could provide a more comprehensive view of the field.\n\nSince doubly robust estimators represent the state-of-the-art for causal parameter inference and have advantages over IPW-based methods, adding at least one of these approaches as a baseline would further strengthen the comparisons [1, 2, 3].\n\n\nrefs:\n[1] Ogburn, Elizabeth L., et al. “Causal inference for social network data.” Journal of the American Statistical Association 119.545 (2024): 597-611.\n[2] Khatami, Seyedeh Baharan, et al. “Graph Neural Network based Double Machine Learning Estimator of Network Causal Effects.” arXiv preprint arXiv:2403.11332 (2024)\n[3] Leung, Michael P., and Pantelis Loupos. “Unconfoundedness with network interference.” arXiv preprint arXiv:2211.07823 6 (2022).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rvb0qc2GAQ", "reviewer": "Reviewer_uH9y", "summary": "CauGramer leverages a Graph Transformer architecture with cross-attention to aggregate peer information within an L-order neighborhood.  This approach addresses the limited receptive field of traditional GCNs.  The model learns interference representations for both features and treatments, using cross-attention to capture complex interactions between units.  To mitigate confounding bias, the model incorporates confounder balancing and minimax moment constraints, further enhancing the robustness of the treatment effect estimation.  The authors extend the model to handle unmeasured confounders by incorporating negative control variables.", "questions": "Can you elaborate on the computational complexity of CauGramer, especially compared to other graph neural network and transformer-based methods? Are there any specific strategies employed to mitigate the computational burden, such as approximation algorithms or distributed training?\n\nHow sensitive are the results to the choice of the L-order neighborhood parameter in CauGramer? What is the impact of choosing too small or too large an L value? Is there an optimal way to select this parameter?\n\nThe paper employs the Wasserstein distance (IPM) to measure the discrepancy between treatment and control group representations. Have you considered other divergence measures (e.g., Jensen-Shannon divergence, Kullback-Leibler divergence)? How would the results change with different choices of divergence measures?\n\nYou address the issue of unmeasured confounders by using negative controls. What is the impact on the performance of CauGramer if the negative control variables are not perfectly valid or are missing? How robust is the model to misspecification in the negative control variables?\n\nWhat are the current study's limitations, and what are your plans for future work? Specifically, are there any plans to explore alternative architectures, different attention mechanisms, or more sophisticated methods for handling unmeasured confounders?\n\nIn some sense, I think this paper's theoretical is somehow a bit thin. I am curious whether we could give your method some theoretical guarantee.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Addresses a significant limitation: The paper directly tackles the crucial challenge of estimating treatment effects in networked data with unknown interference structures and aggregation functions, a problem neglected by many existing methods.\n\nNovel architecture: CauGramer's architecture is innovative, combining the strengths of graph neural networks and transformers to capture complex interference patterns. The use of cross-attention is particularly insightful for broadening the receptive field.\n\nRobustness: The incorporation of confounder balancing and minimax moment constraints significantly improves the robustness of the model. The extension to handle unmeasured confounders further enhances its practical applicability.\n\nComprehensive evaluation: The empirical evaluation is thorough, covering various scenarios and comparing against multiple strong baselines.", "weaknesses": "NA, see the questions", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DJXhn3GiMI", "reviewer": "Reviewer_qEMS", "summary": "The paper \"Causal Graph Transformer for Treatment Effect Estimation under Unknown Interference\" introduces an innovative Interference-Agnostic Causal Graph Transformer (CauGramer) framework for causal effect estimation under unknown network interference. This approach captures complex interference information across nodes through an L-order graph attention mechanism combined with cross-attention, without relying on a known interference graph. The model incorporates confounder balancing and minimax moment constraints to achieve robust causal effect estimation.", "questions": "1.In the COVID case study, where transmission primarily occurs through contact, how can interference occur in the absence of a direct link? Could you clarify potential mechanisms of transmission under such conditions?\n2.Could you clarify why the final output \\( r_x^{(L)} \\) is used as the input to the first layer of the peer-treatment representation network, specifically \\( r_t^{(1)} = r_x^{(L)} \\)?\n3. Please explain the statement: \"we simulate unmeasured confounding by removing features.\"\n4. The paper states that the relationship between peer-treatment \\( t_{Pi} \\) and outcome \\( y_i \\) remains confounded by the common causes \\( r_i \\), leading to biased estimates of peer and total effects. Moreover, the peer-treatment \\( t_{Pi} \\) is unknown. In the main text, it is mentioned that \\( r = r_x \\oplus r_t \\). Could you clarify why \\( r_i \\) acts as a confounder for both \\( t_{Pi} \\) and \\( y_i \\)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1.The study introduces a novel approach by addressing causal inference from the perspective of an unknown interference graph. The combination of L-order graph attention with cross-attention for modeling interference information.\n2.The paper is well-structured.\n3.Causal inference under unknown interference graphs is a significant challenge in network data analysis, and the outcomes of this study provide a reference for future research.", "weaknesses": "1.In Table 1, which includes i.i.d. data, the title \"Causal Networked Data\" may not be fully accurate. Perhaps \"Observational Data\" would be more accurate.\n2.On line 129, \"Representation\" could be more accurate formatted in lowercase as \"representation.\"\n3.The method appears to rely on an unconfoundedness assumption, yet latent confounders are mentioned later, violating this assumption.\n4.In Section 4.1, Ma & Tresp’s work is not necessarily limited to first-order interference.\n5.There are multiple instances of quotation mark formatting errors in lines 375, 449, 487, and 516, among others.\n6.How did you address latent confounders in their approach?\n7.When network interference is unknown, how does the proposed method differ from existing work (e.g., Rf1-Rf3)?\n8.The work lacks a description of the number of layers (L) and attention heads (M) used.\n9.While the paper compares CauGramer with several mainstream methods, it lacks a detailed discussion of specific methods for unknown interference graphs (e.g., Rf1-Rf3).\nRf1: Sävje, Fredrik, Peter Aronow, and Michael Hudgens. \"Average treatment effects in the presence of unknown interference.\" Annals of Statistics 49.2 (2021): 673.\nRf2: Hoshino, Tadao, and Takahide Yanagi. \"Causal inference with noncompliance and unknown interference.\" Journal of the American Statistical Association (2023): 1-12.\nRf3: Lin, Xiaofeng, et al. \"Treatment Effect Estimation Under Unknown Interference.\" Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer Nature Singapore, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vfnSiXtw4E", "reviewer": "Reviewer_ii3V", "summary": "This study aims to estimate treatment effects in the presence of unknown interference (including unknown network and summary mechanism). To achieve this goal, it proposes a method named CauGramer, which uses the transformer to address unknown interference.\nHowever, some important studies and baseline methods for unknown interference are ignored.", "questions": "# Questions:\n1. Why do you not introduce the existing works [R1-R4], which have excellent contributions for unknown interference? Can you introduce the difference between your works and these works?  What limitations of prior methods does CauGramer address? What new capabilities does your method enable?\n\n2. [R1-R4] can work when the network information is totally unknown. Can the proposed method work when the network information is totally unknown? How does the performance change in this case?\n\n3. In definitions 1 and 3, as well as equations (1) and (2), should the $t$ in the right of '=' be 1 instead of $t$?\n\n4. For more details, please see Weakness.\n\n\n\n# Reference\n[R1] Bhattacharya, et al.  Causal inference under interference and network uncertainty.  UAI 2019\n\n[R2] Sävje, et al. Average treatment effects in the presence of unknown interference. The Annals of Statistics 2021\n\n[R3] Mayleen, et al. Staggered rollout designs enable causal inference under interference without network knowledge. NeurIPS 2022.\n\n[R4] Lin, et al. Treatment effect estimation under unknown interference. PAKDD 2024.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The presentation is clear.\n2. A new method is proposed.\n3. Identifiability of total effect is given, which provides insight for studying unknown interference.\n4. Experiments are conducted to verify the proposed method.", "weaknesses": "# Weakness\n1. Some important studies are ignored. Unknown interference has been addressed in many existing works [R1-R4]. These works have excellent contributions for unknown interference, but they are ignored by authors. Authors should introduce the difference between their work and these works. Specifically, the authors add a paragraph in the related work section comparing their approach to [R1-R4], highlighting key differences in methodology, assumptions, and capabilities.\n\n2. As introduced in Weaknesses 1, unknown interference is not a new challenge in causal inference, because there are some existing methods [R1~R4] that focus on this issue. \nThe novelty of this study might be limited. It might be helpful if the authors could introduce the limitations of these existing methods and how this study addresses them. Specifically, what limitations of prior methods does CauGramer address? What new capabilities does it enable?\n\n3. In addition, existing methods [R1-R4] have a more relaxed assumption for unknown interference. They do not require any network information, whereas the proposed method requires network information, i.e., $A$.\n\n4. For experiments, authors need to compare with the existing methods for unknown interference. In the current version, authors did not compare with [R1-R4]. The authors can add [R1], [R3] and [R4] as baselines in their experiments, following the implementation details.  [R1] can create a network (you can only take the edges among different nodes) when the input graph is a complete graph or an initial graph, i.e., $A$ in the authors' setting. Then, you can apply G-HSIC on the network created by [R1] as a baseline. [R3] and [R4] can directly work when you give or not give an input graph. [R2] can be ignored, as it is not a learning method. \n\n5. In definitions 1 and 3, as well as equations (1) and (2), should the $t$ in the right of '=' be 1, as the treatment is a binary value, i.e., 1 or 0.\n\n\n\n\n\n# Reference\n[R1] Bhattacharya, et al.  Causal inference under interference and network uncertainty.  UAI 2019.\n\n[R2] Sävje, et al. Average treatment effects in the presence of unknown interference. The Annals of Statistics 2021.\n\n[R3] Mayleen, et al. Staggered rollout designs enable causal inference under interference without network knowledge. NeurIPS 2022.\n\n[R4] Lin, et al. Treatment effect estimation under unknown interference. PAKDD 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)