Higher-Order Causal Message Passing for Experimentation with Complex Interference
Authors: Mohsen Bayati
Yuwei Luo
William Overman
Sadegh Shirani
Ruoxuan Xiong
Venue: NeurIPS 2024 poster (2024)
Keywords: Causal Inference
Network Interference
Dynamic Treatment Effect
Randomized Experiment
Approximate Message Passing
Primary area: causal_inference

Abstract: Accurate estimation of treatment effects is essential for decision-making across various scientific fields. This task, however, becomes challenging in areas like social sciences and online marketplaces, where treating one experimental unit can influence outcomes for others through direct or indirect interactions. Such interference can lead to biased treatment effect estimates, particularly when the structure of these interactions is unknown. We address this challenge by introducing a new class of estimators based on causal message-passing, specifically designed for settings with pervasive, unknown interference. Our estimator draws on information from the sample mean and variance of unit outcomes and treatments over time, enabling efficient use of observed data to estimate the evolution of the system state. Concretely, we construct non-linear features from the moments of unit outcomes and treatments and then learn a function that maps these features to future mean and variance of unit outcomes. This allows for the estimation of the treatment effect over time. Extensive simulations across multiple domains, using synthetic and real network data, demonstrate the efficacy of our approach in estimating total treatment effect dynamics, even in cases where interference exhibits non-monotonic behavior in the probability of treatment.

Higher-Order Causal Message Passing for Experimentation with Complex Interference
Mohsen Bayati!
Yuwei Luo
William Overman'
Sadegh Shirani!
Ruoxuan Xiong
Stanford Graduate School of Business {bayati, yuweiluo, upo, sshirami Satrrannnnssttafoooooooo edu, xionggemory.edd
Emory University
Abstract
Accurate estimation of treatment effects is essential for Decision-making across various scientific fields. This task, however, becomes challenging in areas like social sciences and online marketplaces, where treating one experimental unit can influence outcomes for others through direct or indirect interactions. Such interference can lead biased treatment effect estimates, particularly when the structure of these interactions is unknown. We address this challenge by introducing new class of estimators based on causal preessae--passnngg specifically designed for settings with pervasive, unknown interference. Our estimator draws on information from the sample mean and variance of unit outcomes and treatments over time, enabling efficient use of observed data to estimate the evolution of the system state. Concretely, we construct non-linear features from the moments of unit outcomes and treatments and then learn function that maps these features to future mean and variance of unit outcomes. This allows for the estimation of treatment effect over time. Extensive simulations across multiple domains, using synthetic and real network data, demonstrate the efficacy of our approach in estimating total treatment effect dynamics, even in cases where interference exhibits non con-oonot behavior in the probability of treatment.
Introduction
Randomized experiments are widely recognized as reliable method in data-driven srciision-making for determining the causal effects of new interventions, such as medical treatments or upgrades of market products. The conventional approach involves administering the new treatment to a randomly selected subset of the observation units (e.g., patients, products, geographical areas), referred to as the irealment group, and comparing their outcomes with those units who received no treatment, the control group. However, the validity of these methods substantially relies the assumption that treating group units does not interfere with the outcomes of the control units, known as the Stable Unit Treatment Value Assumption (SUTVA) [Cox, 1958, Rubin, 1978, Manski, 1990, Imbens and Rubin, 2015, Sussman and Airoldi, 2017].
In many social science and online marketplace scenarios, treating one unit impacts not only its outcome but also the outcomes of units that directly or indirectly interact with the treated unit [Bond et al., 2012, Blake and Coey, 2014, Holtz et al., 2020, Johari et al., 2022, Bright et al., 2022]. This interference of treatments and outcomes makes estimating causal effect of treatment particularly challenging. Considering the network of interactions, when a unit is treated, its interactions with neighboring units lead to subsequent changes in their outcomes. These interactions continue over the experimental time horizon and may display complex behaviors. For example, as the treatment expanded to a larger population, the interference effect may intensify or diminish. This necessitates efficient data usage and robust estimators to capture and adapt to such intricacies.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Given the complexity of analyzing interference phenomena. research on network interference often relies on a series of simplifying assumptions. One common assumption is to ignore variations over time and assume outcomes are observed at equilibrium, which discards valuable information before the system reaches equilibrium. To reduce the complexity of the analysis, further assumptions are imposed on the nature and level of interference [Choi, 2017, Cortez. et al., 2022, Li and Wager, 2022a], such as the neighborhood interference assumption or assumptions on the maximum degree of the network. Additionally, a frequently made assumption to help estimate treatment effects is that the interference network is observed [Chen et al., 2024, Agarwal et al., 2022, Jia et al., 2024], which is impractical in some settings, such under pervasive interference. For example, in large-scale online platforms, units may interact through competing platforms, making it difficult to account for all sources of interference. Our aim in this paper is to relax these assumptions.
The impact of network interference can be intricate, particularly when considering interactions among units over time. For example, applying the treatment to one unit can have spillover effects on some control units, or one unit's outcome can directly exert peer effects on other units' outcomes. Simultaneously, treatments with long-lasting effects can have carryover effects to future time periods, and units' outcomes can be serially correlated or have autocorrelation over time. Consequently, whenever SUTVA fails to hold, the number of potential outcomes grows exponentially with the population size and the time horizon of the experiment. This renders the estimation of causal effects under general interference structures impossible due to non non-identifiabiliyy challenges [Manski, 2013, Aronow and Samii, 2017, Basse and Airoldi, 2018, Karwa and Airoldi, 2018, Forastiere et al., 2022].
Recently, Shirani and Bayati [2024] introduced a new framework called Causal Passang (CMP) to address the challenge of causal effect estimation under unobserved pervasive interference. Their methodology relies on observing outcomes over time and is rooted in statistical physics [Mezard et al., 1986, Mezard and Montanari, 2009] and approximate message passing (AMP) [Donoho et al., 2009, Bayati and Montanari, 2011] from high dimensional statistics. Instead of investigating the complex relationships among units, which requires knowledge of the network, CMP focuses on the dynamics of one-dimensional quantities, such as the sample mean and sample variance of units' outcomes over time. These one-dimensional equations, also known as state evolution equations, can help track how the administered intervention propagates through the network of units over time, which enables the estimation of counterfactual scenarios. However, it remains underexplored how to use state evolution estimate causal effects.
In this work, we propose to utilize machine learning to learn a mapping that updates key parameters of the distribution of outcomes over time for causal effect estimation. This is achieved by introducing a set of non-linear feature functions that act on the observed outcomes, creating a "basis" for the learning task. By training properly designed machine learning model on this extracted basis, we estimate the Total Treatment Effect (TTE), also known as the Global Treatment Effect (GTE) or Global Average Treatment Effect (GATE), which measures the causal effect of altering the treatment scenario from treating no one to treating everyone. The result is a family of estimators that allow one to extract more information from the experimental data, thereby ensuring efficient use of the data.
To be more specific, this work builds on the foundation established by Shirani and Bayati [2024], extending their method in two directions by introducing Higher-Order Causal Message Passing (HO-CMP) algorithms. First, HO-CMP incorporates higher-order moments of unit outcomes, unlike Shirani and Bayati [2024]'s approach, which only employs the first moments for estimation. Second, while Shirani and Bayati [2024] focus solely on two-stage experiments with two different probabilities of treatment, our work leverages the additional data provided by having more than two experimental stages with multiple probabilities of treatment. Thus, our work aligns with the common practice in the tech industry of rolling out treatments through sequence of experiments [Kohavi et al., 2020].
We then validate the performance of HO-CMP by simulating multiple experimental settings, en- compassing both linear and non-linear outcome specifications and various types of interference, such as synthetic random geometric networks and real-world networks. Specifically, we introduce a Mon-LinearinMenn
outcome specification, where the spillover effect is mon-oonot in the fraction of treated neighbors; as an example of complex treatment effect structure, we demonstrate how HO-CMP successfully estimates the total treatment effect by effectively utilizing higher-order moments of unit outcomes.
Simulating the experiments also allows us to calculate the ground truth value of the TTE, which remains unknown in real experiments, enabling us compare the performance of HO-CMP to

---

the ground truth TTE. Additionally, we benchmark HO-CMP against standard approaches such as difference-in-mee mn-means and Horvitz- -hrvisz-Thompson estimators, a recent technique of Cortez et al. [2022]. and first-order CMP estimation, like the one by Shirani and Bayati [2024]. We emphasize that large body of recent estimators, e.g., Jia et al. [2024], requires knowledge of the interference network and is not applicable in our setting. The results showcase HO-CMP outperforming the benchmarks in estimating the TTE over time and its flexibility to cover different outcome specifications and interference structures.
Related causal inference literature. The primary objective of research on causal inference in the context of network interference is estimate causal effects while relaxing SUTVA. For this purpose, various assumptions and methods have been proposed. We briefly discuss the predominant ones.
A common approach to relax SUTVA is partial interference. Under this assumption, units are divided into disjoint clusters and interference assumed only within the same cluster [Sobel, 2006, Rosenbaum, 2007, Hudgens and Halloran, 2012, Tchetgen and VanderWeele, 2012, Liu and Hudgens, 2014, Kang and Imbens, 2016, Viviano, 2020b, Bhattacharya et al., 2020, Qu et al., 2021, Auerbach and
2021, Candogan al., 2023, Ugander and Yin, 2023]. When interference extends across clusters, standard estimators become biased. To address this, Eckles et al. [2016] propose a cluster-randomized approach that randomizes treatment assignment across clusters, reducing bias.
However, it requires knowledge of the clusters.
The other assumption to replace SUTVA is the Neighborhood Interference Assumption (NIA). NIA states that outcomes are only influenced by the treatments of neighboring units in the network. This assumption is commonly imposed in the literature that relaxes the SUTVA [Sussman and Airoldi, 2017]. Some recent studies combine the NIA with the availability of either fully or partially observed interference structure [Leung, 2020, Viviano, 2020a, Agarwal et al., 2022, Belloni et al., 2022. Li and Wager, 2022b]. Without prior knowledge of the interference structure. Cortez et al. [2022] consider low-degree polynomial interactions among units in the network. Leung [2022] also introduces a weaker version of the NIA, where the interference between two units located far away from each other is allowed to be nonzero, but negligible.
Another approach is to facilitate the estimation of causal effects by setting restrictions on the network structure [Chin, 2018, Jagadeesan et al., 2020, Wang et al., 2020, Li and Wager, 2022a, Agarwal et al., 2022, Jagadeesan et al., 2020, Leung, 2022]. These restrictions include bounding the largest node degree of the interference graph, limiting the degree of the dependency graph, observing specific patterns in the network, locally constrained interference structures, and restricting the topology of the interference network.
Driven by applications in marketplace platforms and two-sided marketplaces, several recent works have examined specific interference patterns [Holtz et al., 2020, Wager and Xu, 2021, Munro et al., 2021, Johari et al., 2022, Harshaw et al., 2022, Farias al., 2022, Bright et al., 2022, Farias et al., 2023]. For example, Farias et al. [2022] study experiments in Markovian systems where interference effects propagate through constraints like limited inventory.
From another perspective, most of the existing literature on network interference focuses on the case of single-time point observation [Hudgens and Halloran, 2012, Aronow and Samii, 2017, Basse et al.. 2019, Jackson et al.. 2020, Sävje et al., 2021]. These studies have provided insightful results on spatial interference effects, but they often overlook temporal variations of the treatment effect. Recently. there has been shift to consider settings with multiple-time observations [Li and Wager, 2022a, Boyarsky et al., 2023]. However, the problem of considering the dynamics of units' outcomes remains understudied (Arkhangelsky and Imbens, 2023].
2 Setup and Foundation
Consider system of N units indexed by i E [N] := {1,... .N) subject to a randomized experiment. The units are observed over a time horizon of T + periods and for each t {0, 1,.., T), we let Wi denote the treatment status of unit i during time period t. For simplicity, we consider a Bernoulli randomized design such that Wi Bernoulli(m,.) That is, at time unit i receives the trealment with a probability of Tr, corresponding to W: = Otherwise, unit i belongs to the control group and W O. In this context, we collectively define л = (To; Ti, ...,...77 .T) as the experimental design. Then, following the potential outcome framework [Imbens and Rubin, 2015], let (W) represent
3

---

the potential outcome of unit at time t, where W denotes the entire treatment allocation matrix, with W the entry in row and column
Administering the treatment of unit i at time i according to w (as one realization of the random variable W all units throughout the experiment; accordingly, we let v = (? = w) be the observed outcome of unit at time under the treatment assignment w:
(as realization of W) show the matrix that captures the treatments of
w
wi
V0
y
y0
Wi
wi
V
Vi
y
WT
UI
VT
Observing (w, y), we are interested in estimating the TTE of the intervention, defined as below:
N
TTE, lim N N
N (1(1))
t= 0,
where 1 and O are matrices of all and all of appropriate dimensions (in this case, + by N). Intuitively, the TTE measures the average effect of changing the treatment for the entire population. This is common estimand in the network interference literature and provides important insights into the efficacy of the treatment for decision-makers [Jia al., 2024, Chen al., 2024, Viviano al., 2023, Yu et al., 2022, Cortez et al., 2022].
Deriving practical and efficient estimator for the TTE is challenging due the fact that we observe the population only under one treatment scenario [Holland, 1986]. Indeed, in Eq. (1), we can observe most one of y; (1) Y, (0), and often, neither." In the following sections, we address this challenge by proposing new class of estimators grounded in the CMP framework. These estimators rely on the efficient use of experimental data, y and w, yielding accurate causal estimation under unknown network interference.
2.1 Potential outcome specification and state evolution of the experiment
In this section we provide summary of the outcome specification and results of Shirani and Bayati [2024] that we utilize in the remaining. For 0, .., T 1, we let 9. an unknown measurable function. We also use W = (Wa...., wz) assignment of unit i during the experiment. Accordingly, the treatment allocation matrix Wis a T by N matrix with columns equal to W Given potential outcomes Y? (W) at time and j E [N], their outcomes in time period & 1 specified by
IR T + E be
to denote the treatment
(v((W), +G, 9, = j=1
GD
t= 0,
1,
(2)
where G quantifies the impact of unit j on unit at time i and é is zero-mean Gaussian noise with
variance of o² accounting for measurement errors. In addition, we let G = [G [.JE[N] and refer to as the interference matrix. Then, according to Eq. (2), the function captures the impact of past outcomes and treatment assignments of other units on the current outcome of unit i.
Now, fixing f, we define
v (W) = lim +00
Σ (W),,
(W)2 = lim Σ ( u No0 N
N
1=1
Then, as shown by Shirani and Bayati [2024], whenever the elements of the interference matrix GÖ are i.id. Gaussian random variables with mean μ/N and variance /2/N, under mild moment conditions on initial values Yú, we have
V (W) a.s. μE (w) + + wT),
(4)
Pet1 (wy2 us BFB (u, (W),, (W),, wT)]
We can only observe y for one of exponentially many realizations of

---

where Ze ~ ~O, 1) is independent from W Bernoulli(T)) (that is, W. Bernoulli(m;) and W = WW W1,- 1y..... w)) T) and the equalities hold almost surely. We note that the theory behind this result is rooted in the AMP literature, going back to Bolthausen [2014], Bayati and Montanari [2011]. However, Shirani and Bayati [2024] note, there is major distinction between the AMP literature and the above setting: in the AMP literature, the matrix G is observed, and the aim is to construct proper functions 9t for completely different objective, which is studying the high-dimensional asymptotics of first-order algorithms. However, in the current context, the matrix G and functions 9t unknown and the goal is estimate them.
Considering Eq. (3), the equations in (4) determine the dynamics of the sample mean and sample variance of unit outcomes over time in large sample asymptotics, and are denoted by the State Evolution (SE) equations of the experiment [Shirani and Bayati, 2024]. In the next section, we present an efficient algorithm to learn the state evolution dynamics outlined in Eq. (4). This method enables us to accurately estimate the TTE defined in Eq. (1) and its corresponding confidence interval.
3 Algorithm
In this section, we introduce Higher-order Causal Message-passing (HO-CMP) for estimating the TTE over the entire time horizon of the experiment. Briefly speaking, HO-CMP directly estimates the update function in the state evolution equations (4). thereby estimating counterfactual quantities while accounting for the impact of unknown network interference. To this end, by Eqs. (1) and (3), we rewrite the TTE as the difference of the sample means the large limits:
TTE; (1) (0).
That means the problem of estimating the TTE is equivalent to estimating (1) and (0) using the observed data, denoted by (w, y). On the other hand, considering the state evolution equations in (4), the system state at time I. + 1, denoted by (w)),, (tv) (), is (nonlinear) function of the system state distribution at time i, characterized by (Me(w), Pe (tw)) ?) and W, encompassing the sample mean and variance of observed outcomes as well as the design of the experiment. However, because the exact functional form and parameters of equations in (4) are unknown, one cannot directly apply the SE to track the evolution of states. Therefore, we propose to estimate the unknown update functions in SE equations, utilizing the observed data (w, z). For this purpose, fix the treatment assignment matrix w and define
N
N Σ 1=1
(www) =
VE
p=(w)2 = N
(vt (z))))
N
W = N
-= (wo, wT)
In addition, let T -= [σk))][K] be a prespecified vector of measurable feature functions of current estimates of the sample mean 2i (w), sample variance Pr (w)", and the design w. We define æ; to represent the feature vector as follows:
o (x(()), (w)). w²) = [00 (Dx())), Pr(w),
Φx (e,(w),,:(w)) [w)
Then, we formally propose learning the mapping f-)-) defined by,
(D++iww), (++1 = fo(x.)
(5)
We summarize the method Algorithm Note in our experiment design we begin with all units under control by setting Ra O, meaning no units receive treatment in period O. Additionally, to avoid non-identifiability issues, the experiment requires at least two stages, which corresponds to having at least two distinct values in the set {m....., .T).
The proposed HO-CMP method encompasses a rich family of estimators, offering flexibility through the selection of feature functions {Ø& ]ee[k] and model fo(.). Specifically, incorporating proper feature (basis) functions, with examples shown in Table facilitates the extraction of informative patterns for learning the unknown nonlinear dynamics of the system throughout the experiment. In
5

---

Table 1: Two examples of feature functions
Algorithms
Feature functions
w) ke(k]
fe(.)
FO-CMP {Ur(wn); 11 (w) wz}
linear regression
HO-CMP
{Pr(w)), W (c(w) we (rw)) w
linear regression
practice, one could choose these basis functions based on heuristics, domain knowledge, and prior information about the dynamics.
Specifically, in this paper, we consider the following estimators, as summarized in Table 1.
FO-CMP (First-Order Causal Message-Passingg This corresponds to the simple setting where (t-((w) assumed to be a function of the previous sample mean (2ww), the sample mean of the current treatment Wx+11 and an additional term to model the interaction of the dynamics and previous t) treatments vi(w) (w) W..- Consequently, this model irrelevant of the variance Pt+1 (w+). This is true
when Sr takes a simple nonlinear form (ye, xz) = agg + :w0++1 + 2 We remark that FO-CMP essentially uses the first state evolution equation in (4) and fails extract informative signals from the second evolution equation.
HO-CMP (Higher-Order Causal Message-Passing):: HO-CMP further introduces the second-order terms (w ) and (ww) to model the nonlinear treatment effects. It improves data efficiency by utilizing both state evolution equations. It also allows estimation of higher order terms Taylor series of gt.
While FO-CMP extends the estimation algorithm in Shirani and Bayati [2024] to accommodate experiments with more than two stages, HO-CMP introduces new dimension to the estimation problem by incorporating second-order terms. This inclusion enhances data utilization, resulting in higher estimation efficiency in HO-CMP compared to FO-CMP.
Algorithm 1: Higer-Oroe Order Causal Message Passing (HO-CMP)
Data: Observed data (w, y), feature functions Ø (Φ*))ε(K)) machine learning model fa(-) Step 1: Data processing for L - 0 to T do
(v())) 1/2 Σ
i= V4-
NN
(u(w) t 1 D
(vi
N
x, - (Dr(w), p. (w)) w)
end
Step 2: Model Estimation
Estimate fo from data { (x,, p++ fre|r-1]" guided by (5).
Pr+
Step 3: Counterfactual Estimation
Po(0) < Po ww), (1) (u(w), Pu (0) (= (n) pe(w) TTE, <- 0
for t < to T do
Compute the features and predict the counterfactuals
(0) +- ã (O), (0)2
(00))))). 0), x1(1) + (P.(1),
(pr+1 1 (0)
fo(x;(O)), (1,) + fo(
Estimate the TTE
TTE <- Pe+ (1) (0-
end
Result:
(TEE
(E(T]

---

Experiments
In this section, we use synthetic experiments under simulated and real-world network interference patterns, compare the performance of FO-CMP and HO-CMP estimators, outlined in Table 1 and Algorithm 1, with several benchmarks. First, we introduce the experimental design, benchmark estimators, interference patterns, and outcome specifications.
Experimental design.
We primarily focus the staggered rollout design with L distinct treated
probabilities, denoted by (1(1,, ,,,,,(,) where x!() increases monotonically with E (1..... L). In the first 7(1) periods, x x 100% of units are in the treatment group. From T(1) to ( periods, (2) x 100% of units are in the treatment group, and so forth. In the staggered rollout design, once unit is allocated to treatment, remains in the treatment group until the experiment concludes [Xiong et al., 2024]. In the appendix, we also consider the Bernoulli randomized design, where the treatment re-randomized at every time period, allowing units to switch between the treatment and control groups throughout the experiment. We use two values of T = 40, 200 and set J. = 4, with (T(1), T (2),, T(3) x T = (0.1, 0.2, 0.4, 0.5). In the appendix, we show the impact of increasing L or the maximum treatment probability (24)
,((4,
Benchmark estimators. We first present two benchmark estimators commonly used for treatment effect estimation, both in settings with and without network interference. The final estimator is designed specifically for settings with unknown network interference [Cortez et al., 2022].
The first benchmark estimator is the standard difference-in in means (DM) estimator given by
2j=1 Vi W Σド= w/ Σ w!)
Σn ve (1-11) w/)
dm
TTE
which is the difference in average outcomes between treated and control units at each time period f.
The second benchmark is the standard Horvitz and Thompson [1952] (HTT) estimator given by
1 N
N
y w
y (1 wt
TTE,
Tt.
which weights observed outcomes by the inverse propensity score (i.e., 1/x, or 1/(1 - mT)).
The third benchmark estimator is the polynomial interpolation estimator (PolyFit) introduced by Cortez et al. [2022]. PolyFit operates by obtaining estimates for the average of outcomes at equi- librium for treated probabilities =T(),, ... (L),, denoted by Vequil (ar (T())... Mequa (T))), then utilizes Lagrange interpolation method and obtains degree-L polynomial approximation for the function Vequil [0, > R which can be used to estimate the equilibrium values under global control
and treatment, (0) and Pequi(1) (1). Finally, TTE is estimated by
polyfit TTE,
Dequit (1) Dequit (0).
On the one hand, PolyFit does not need any knowledge of the interference network; however, it comes at the expense of having to grapple with two challenges. First, it may incur a high variance as increases due to fitting high-degree polynomial. The second challenge is that needs accurate estimates for each Vequil (?()), which requires treating (() fraction of units for long enough number of periods so that the outcomes reach an equilibrium. This can be achieved the staggered
roll-out design is performed over a long enough horizon with each (()) sufficiently large, and then estimating each Veguil (7) by sample average of outcomes time T() However, when such lengthy experiment not feasible, the estimates for (7) will be less accurate.
Interference networks. consider two networks (graphs). The first graph simulated random geometric graph model, studied by Leung [2022]. The second graph is social network of Twitch users [Rozemberczki and Sarkar, 2021]. either scenario, we denote the adjacency matrix of the graph by E e {0, N NxN, For any and Eg equals 1 if j is neighbor of and 0 otherwise.
7

---

PolyFr
HO-CMP
Data point
Eats
Puyfii
Polyfi molimete
I O-GMI ediimate
HO-CMP
041
0.3
0.1
11.5 105
10
0.5
0.2
0.4
2.5
0.5
10
0.0
0.2
0.1
0.6
0.5
(a) Mequil (T) vs
(b) Curve fitting of polyfit
(c) Curve fitting of HO-CMP
Figure 1: (a) Pequil (T) with PolyFit and HO-CMP estimates across runs (Non-LincarinMeans)) (b) and (c) show one sample estimates with observed data points.
Outcome generating processes. We consider two outcome specifications to model monotone and non-monotone interference patterns. Specifically, for both settings, generate outcomes using the following specification:
En Eigy Σラ=1 Eig
ΣS Eig BigWit+1 Σラ=1 Eij
Yi = =a+
+
+ +W + +
where in the first setting, g(-) taken to be the identity function, i.e., g(x) = x for any x. Therefore, Y depends linearly on the fraction of treated neighbors, and we refer to this setting as the LinearlnMeans outcome setting. This setting is widely studied in the causal inference literature [Cai et al., 2015, Eckles et al., 2016, Leung, 2022].
In the second setting, g(-) is specified by periodic function, i.e., g(x) sin()er) for any Therefore, Yi+1 on average, first increases and then decreases with the fraction of treated neighbors, as visualized by the Ground Truth curve panel (a) of Figure We refer to this setting as the Non- LineLeeaarnmannss outcome setting.
Results. We compare FO-CMP and HO-CMP with the three benchmarks for estimating the TTE across the aforementioned outcome specifications and interference networks for long (T 200) and short (T = 40) horizons. In each scenario, we perform 100 simulations of the synthetic experiment. The resulting distributions of ground truth and estimated TTES are shown in Figures 2-5. All experiments were conducted on MacBook Air with an Apple MI chip and 16 GB of memory, with each setting taking about 15 minutes for 100 iterations. The key takeaways are as follows.
First, the DM and HT estimators exhibit significant bias across all cases. This is intuitive, they estimate the TTE without accounting for the network interference.
Second, in the LinearlnMeans outcome setting, FO-CMP and HO-CMP achieve low estimation error and minimal bias. This holds for both long experiment durations (T 200), where outcomes reach equilibrium, and short experiment durations (T = 40), where outcomes have not yet reached equilibrium, as shown in Figures and 3, respectively.
Third, expected, PolyFit's dependence accurate estimates for each (erui((((O)) requires large to reduce estimation bias. This is evident when comparing Figures and 3: with a smaller T, PolyFit shows bias. This is also demonstrated in panel (b) of Figure 1, where the red points-which represent sample averages of outcomes (1)) ..... (4) _ not yet converged and are slightly lower than their ground truth (equilibrium) values. This causes PolyFit's estimation of Dequit (1) to be inaccurate, leading large bias. In contrast, HO-CMP, as shown in panel (c) of Figure 1, is immune to this problem as it is designed to work with off-equilibrium data. Even with a larger T, the degree-L L polynomial estimation costs PolyFit with higher variance than both FO-CMP and HO-CMP, as shown Figure 2. Overall, this underscores the more efficient data utilization FO-CMP and HO-CMP through their ability leverage off-equilibrium data.
Fourth, the Mon--iinearnMeens outcome setting, HO-CMP achieves substantially lower estimation error compared to FO-CMP, as shown in Figures 4 and 5. This makes intuitive sense, as the higher-

---

Nm40100 1-200 Graph-SSmubtted
Num nuns
NH1000 1-200 Grophe Twithh Gulcomen inllsnns, Num nunse 103
Grondd Trun
12
Mayer
10
CM4P
cup
25
125
200
100
175
point
Boxplet
Ground Trrhh
Edsiodon Motic
DM
Extmeeoo
F0-0MP
FO:CMP
HG-DUP
HD-MM
TTE
(a) Random graph
(b) Twitch graph
Figure 2: LinearlnMeans with T = 200. == 4, with (T)11, n (2) (3) n (1) (0.1,0.2,0.4,
and ('() 506 for all E 21, 3, 4}. Shaded regions show 95% percentile intervals of mean.
N-4020,
Graph--tmabiled, Culcame-LnnartMMMann Num
N-4000
Graph-Twikhh, Outcomo-Lnnerr runs-100
TM
DV
154
15.0
70-00"
rocM HO-UE
40 134
7.2
5.
5.0
Boookot lee time point
Bosplot fim
Ground Truit
Ground Tush
Edeintiion Kothod
DN
balmatoon Maatted
PojyF
Pol/Ft
FO-CMP
10U
12.5
17.
11.0
15.2
17.5
2010
(a) Random graph
(b) Twitch graph
Figure 3: LinearlnMeans with = 40. L = 4, with (I)(), (2)), π (3) ((1) = (0.1, 0.2, 0.4, 0.5) and
T() = 10k for all é {1,2, 3, 43. Shaded regions show 95% percentile intervals of mean.
order terms in HO-CMP better capture the nonlinearity of (oquil(T) in while leveraging the additional data sample variance dynamics time, thereby enhancing the estimation accuracy.
Finally, the proposed estimation method demonstrates robustness across different experimental setups, including both LinearlnMeans and Non-LinearIn outcome specifications. Additionally, robustness to graph structure-random versus Twitch graph-is evident from comparing the left and right plots in Figures 2-5. In Figure 6 of Appendix A, also demonstrate the robustness of the proposed methods to various parameters: the number of treatment probabilities L, the maximum treatment probability क (L), and the choice of experimental design (staggered rollout versus Bernoulli randomization).

---

Ne4000, 1-200, GraphnSmibbted, mearinleeass Num nunem100
N-4000. T-200, Gesphe Twitsh Soltommenon---neeeee Lnsern Num rumse
Grund Trun
Ground
Mayer CM4P
TTE
cup
25
1.5
200
75
100
175
point
point
Ground
Ground Trrch
Edsiodon Motcc
DM
F0-0MP
FO:CMP
HG-DUP
HD-MM
TTE
TTE
(a) Random graph
(b) Twitch graph
Figure 4: Non-LinearlnMeuns with T = 200. L = 4, with (n (1) (1) x n
(4)
(0.1, 0.4, 0.5), and T()) 50e for all t. Shaded regions show 95% percentile intervals of mean.
N-40000 1-60, Geaph-Simutttdd, Outome-kkn--reermteen Num rump-1000
N-+000 T-ill GmptrTTttch Aattomention-Lneematee Num rune-
2M
14
12
10
FO-CUP HD C0-OUp
HD 2M47
Boxplot time point
Ground Truit
Ground Tush
Edinado Mcched
Peypa
MMyr FO.CMP
F0-0MP
TTE
(a) Random graph
(b) Twitch graph
Figure 5: MoneLinearssMeans with T
40. L
= 4, with (- (L) (2)) (3) (3),,((1) x(4) =
(0.1,0.2 0.210.2,00 0.4, 0.5), and T()) = 10E for all f. Shaded regions show 95% percentile intervals of mean.
Conclusion
Estimating causal effects under pervasive interference presents significant challenges [Sussman and Airoldi, 2017]. Building on the causal message-passiinn passing framework of Shirani and Bayati [2024], we incorporate higher-order moments observed outcomes and treatment probabilities to estimate the total treatment effect, without requiring knowledge of the interference network. Our approach leverages machine learning techniques to extract informative patterns from these higher moments, enabling estimator to capture complex counterfactual behaviors, including non-monoooiic trends in outcome means relative to treatment proportions. While we demonstrate strong performance across various outcome specifications and network structures, the framework's applicability may be limited when multiple outcome observations are unavailable.
10

---

References
Anish Agarwal, Sarah Cen, Devavrat Shah, and Christina Lee Yu. Network synthetic interventions: A framework for panel data with network interference. arXiv preprint arXiv:2210.1 1355, 2022.
Dmitry Arkhangelsky and Guido Imbens. Causal models for longitudinal and panel data: A survey. Technical report, National Bureau of Economic Research, 2023.
Peter M Aronow and Cyrus Samii. Estimating average causal effeets under general interference, with application to social network experiment. 2017.
Eric Auerbach and Max Tabord-Mechan. The local approach to causal inference under network interference. arXiv preprint arXiv:2105.03810 2021.
Guillaume Basse and Edoardo M Airoldi. Limitations of design-based causal inference and a/b testing under arbitrary and network interference. Sociological Methodology, 48(1): 151, 2018.
Guillaume W Basse, Avi Feller, and Panos Toulis. Randomization tests of causal effects under interference. Biometrika, 106(2):487-494, 2019.
Mohsen Bayati and Andrea Montanari. The dynamics of message passing on dense graphs, with applications to compressed sensing. IEEE Transactions on Information Theory, 57(2):764-785, 2011.
Alexandre Belloni, Fei Fang, and Alexander Volfovsky. Neighborhood adaptive estimators for causal inference under network interference. arXiv preprint arXiv:2212.03668. 2022.
Rohit Bhattacharya, Daniel Malinsky, and Ilya Shpitser. Causal inference under interference and network uncertainty. Uncertainty in Artificial Intelligence, pages 1028-1038. PMLR, 2020.
Thomas Blake and Dominic Coey. Why marketplace experimentation is harder than it seems: the role of test-control interference. In Proceedings of the Fifteenth ACM Conference on Economics and Computation, EC 14. page 567-582, New York. NY, USA, 2014. Association for Computing Machinery. ISBN 978 450335663. doi: 10.1 (0.1145/2600057.0 26008837 URL httpt://d//o.. org/
10. 10..1//5/00007...1 2602837.
Erwin Bolthausen. An iterative construction of solutions of the tap equations for the sherrington- kirkpatrick model. Communications in Mathematical Physics, (25(():333-366)) 2014.
Robert M Bond, Christopher Fariss, Jason J Jones, Adam DI Kramer, Cameron Marlow, Jaime Settle, and James H Fowler. A 61-million-person experiment in social influence and political mobilization. Nature, 489(7415):295-298 2012.
Ariel Boyarsky, Hongseok Namkoong, and Jean Pouget-Abadie. Modeling interference using experiment roll-out. arXiv preprint a7Xiv:2305.10778, 2023.
Ido Bright, Arthur Delarue, and Iln Lobel. Reducing marketplace interference bias via shadow prices. arXiv preprint arXiv:2205. 02274, 2022.
Jing Cai, Alain De Janvry, and Elisabeth Sadoulet. Social networks and the decision to insure. American Economic Journal: Applied Economies, 7(2):81-108, 2015.
Ozan Candogan, Chen Chen, and Rad Niazadeh. Correlated cluster-based randomized experiments: Robust variance minimization. Management Science, 2023.
Qianyi Chen, Bo Li, Lu Deng, and Yong Wang. Optimized covariance design for ab test on social network under interference. Advances in Neural Information Processing Systems, 36, 2024.
Alex Chin. Central limit theorems via stein's method for randomized experiments under interference. arXiv preprint arXiv:1804.03111 2018.
David Choi. Estimation of monotone treatment effects network experiments. Journal of the American Statistical Association. 142 2017.
11

---

Mayleen Cortez, Matthew Eichhorn, and Christina Yu. Staggered rollout designs enable causal inference under interference without network knowledge. In Advances in Neural Information Processing Systems, 2022.
David Roxbee Cox. Planning of experiments. 1958.
David L. Donoho, Arian Maleki, and Andrea Montanari. Message-passinng algorithms for compressed sensing. Proceedings of the National Academy of Sciences, 106(45): 1891 199491899, 2009.
Dean Eckles, Brian Karrer, and Johan Ugander. Design and analysis of experiments in networks: Reducing bias from interference. Journal of Causal Inference, 5(1):20150021, 2016.
Vivek Farias, Andrew Li, Tianyi Peng, and Andrew Zheng. Markovian interference in experiments. Advances in Neural Information Processing Systems. 35:535-544 549,5499222 2022.
Vivek Farias, Hao Li, Tianyi Peng, Xinyuyang Ren, Huawei Zhang, and Andrew Zheng. Correcting for interference in experiments: A case study at douyin. arXiv preprint 2xXi:23305.02542, 2023.
Laura Forastiere, Fabrizia Mealli, Albert Wu, and Edoardo M Airoldi. Estimating causal effects under network interference with bayesian generalized propensity scores. Journal of Machine Learning Research, 23(289):1 2022.
Christopher Harshaw, Fredrik Sävje, David Eisenstat, Vahab Mirrokni, and Jean Pouget-Abadie. Design and analysis of bipartite experiments under a linear exposure-response model. In Pro- ceedings of the 23rd ACM Conference on Economics and Computation, EC '22, page 606, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 978 7781450391504. URL https://doi. http://do..orr//////// 10. 11445/449486..38 3538269.
Paul Holland. Statistics and causal inference. Journal of the American statistical Association, 81 (396):945-960, 1986.
David Holtz, Ruben Lobel, Inessa Liskovich, and Sinan Aral. Reducing interference bias in online marketplace pricing experiments. arXiv preprint arXiv:2004. 12489, 2020.
Daniel G Horvitz and Donovan J Thompson. A generalization of sampling without replacement from finite universe. Journal of the American statistical Association, pages 663-685, 1952.
Michael G Hudgens and M Elizabeth Halloran. Toward causal inference with interference. Journal of the American Statistical Association, 103(482):832-843 2012.
Guido W Imbens and Donald B Rubin. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press, 2015.
Matthew O Jackson, Zhongjian Lin, and Ning Neil Yu. Adjusting for peer-influence in propensity scoring when estimating treatment effects. Available at SSRN 3522256, 2020.
Ravi Jagadeesan, Natesh S Pillai, and Alexander Volfovsky. Designs for estimating the treatment effect in networks with interference. 2020.
Su Jia, Nathan Kallus, and Christina Lee Yu. Clustered switchback experiments: Near-optimal rates under spatiotemporal interference, 2024.
Ramesh Johari, Hannah Li, Inessa Liskovich, and Gabriel Y Weintraub. Experimental design in two-sided platforms: An analysis of bias. Management Science, 68(10):7069 -089, 2022.
Hyunseung Kang and Guido Imbens. Peer encouragement designs in causal inference with partial interference and identification of local average network effects. arXiv preprint arXiv: 1609, 1609,04464, 2016.
Vishesh Karwa and Edoardo M Airoldi. systematic investigation of classical causal inference strategies under mis- sissspecifiiction due to network interference. arXiv preprint 1810v:8110.080 2018.
Ron Kohavi, Diane Tang, and Ya Xu. Trustworthy online controlled experiments: A practical guide to a/b testing. Cambridge University Press, 2020.
12

---

Michael P Leung.
Treatment and spillover effects under network interference. Review of Economics and Statistics, 102(2):368-380, 2020.
Michael Leung. Causal inference under approximate neighborhood interference. Econometricu, 90 (1):267-293, 2022.
Shuangning Li and Stefan Wager. Network interference in micro-randomized trials. arXiv preprint arXiv:2202.05356, 2022a.
Shuangning Li and Stefan Wager. Random graph asymptotics for treatment effect estimation under network interference. The Annals of Statistics, 50(4):2334 2022b.
Lan Liu and Michael G Hudgens. Large sample randomization inference of causal effects in the presence of interference. Journal of the american statistical association, 109(505):288-300 2014.
Charles F Manski. Nonparametric bounds on treatment effects. The American Economic Review, 80 (2):319-333, 1990.
Charles F Manski. Identification of treatment response with social interactions. The Econometrics Journal, 161)), 2013.
M Mezard, G Parisi, and M Virasoro. Spin Glass Theory and Beyond, An Introduction to the Replica Method and Its Applications. World Scientific, Paris, Roma, November 1986. doi: 10.1142/0271.
Marc Mezard and Andrea Montanari. Information, physics, and computation. Oxford University Press, 2009.
Evan Munro, Stefan Wager, and Kuang Xu. Treatment effects market equilibrium. arXiv preprint arXiv:2109.11647, 2021.
Zhaonan Qu, Ruoxuan Xiong, Jizhou Liu, and Guido Imbens. Efficient treatment effect estimation in observational studies under heterogeneous partial interference. arXiv preprint arXiv:2107. 12420,, 2021.
Paul R Rosenbaum. Interference between units in randomized experiments. Journal of the american statistical association, 102(477):191-200, 2007.
Benedek Rozemberezki and Rik Sarkar. Twitch gamers: a dataset for evaluating proximity preserving and structural role-based node embeddings, 2021.
Donald B Rubin. Bayesian inference for causal effects: The role of randomization. The Annals of statistics, pages 34-58, 1978.
Fredrik Sävje, Peter Aronow, and Michael Hudgens. Average treatment effects in the presence of unknown interference. Annals of statistics, 49(2):673, 2021.
Sadegh Shirani and Mohsen Bayati. Causal message-passing for experiments with unknown and gen- eral network interference. Proceedings of the National Academy of Sciences, 121(400):2322232121, 2024.
Michael Sobel. What do randomized studies of housing mobility demonstrate? causal inference in the face of interference. Journal of the American Statistical Association, 101(476):13981144 2006.
Daniel L Sussman and Edoardo M Airoldi. Elements of estimation theory for causal effects in the presence of network interference. arXiv preprint /rXiv: 2017.
Eric J Tchetgen Tchetgen and Tyler J VanderWeele. On causal inference in the presence of interference. Statistical methods in medical research, 21(1):55-75, 2012.
Johan Ugander and Hao Yin. Randomized graph cluster randomization. Journal of Causal Inference, 11(1):0222014, 2023.
Davide Viviano. Experimental design under network interference, arXiv preprint arXiv:200..0 08421,, 2020a.
13

---

Davide Viviano. Policy design in experiments with unknown interference. arXiv preprint arXiv:2011.08174, 2020b.
Davide Viviano, Lihua Lei, Guido Imbens, Brian Karrer, Okke Schrijvers, and Liang Shi. Causal clus- tering: design of cluster experiments under network interference. arXiv preprint arXiv:2310.14983 2023.
Stefan Wager and Kuang Xu. Experimenting in equilibrium. Management Science, (((11):694-6715, 2021.
Ye Wang, Cyrus Samii, Haoge Chang, and PM Aronow. Design-based inference for spatial experi- ments with interference. arXiv preprint arXiv:2010.13599 2020.
Ruoxuan Xiong, Susan Athey, Mohsen Bayati, and Guido Imbens. Optimal experimental design for staggered rollouts. Management Science, 70()):31175536, 2024.
Christina Lee Yu, Edoardo M Airoldi, Christian Borgs, and Jennifer T Chayes. Estimating the total treatment effect in randomized experiments with unknown network structure. Proceedings the National Academy of Sciences, 119(44)::2208895111 19, 2022.
14

---

A Supplementary Experiments
Nm40100 [+40, Graph- Twithh Outtommeehimn-eeeeeeee www.mmneee.rrmeeers....mm Num 1ump
NH4000 THID GmphhTTitih Oktramentoon- Lnnee-LLeernnn milenns Num nuns- 100
Grund Trunh
Mayer
Pubyi
10
CUP
point
point
Ground Trrhh
Ediiddn Monic
DM
F0-CMP
FO:CMP
HG-DVP
TTE
(a) The same setting as Figure 5b
(b) Increasing number of '
K-40100 T-40,
OutcomenNen--Le cecemmmmkcc-rrrrtteeeerr IneatiiMearss 10ne
N-+000 Twiil Gipph-Twtchh Oktromention- Swrrion-LLneeeeeee intesns,, Num nune
Ground Trath
Courd Truch
Paylt
Puy-:
10
20
25
30
35
10
12
30
20
Boxplct of ast time point
Boxglitt last time point
Essnadonn Method
PM
IIT
ITT
F0-CUP
FO.CMP
HC-CVP
HD-MM
(c) Increasing the largest
(d) Bernoulii randomized design
Figure Robustness check, under the Menssss with Twitch graph and = 40, i.e., setting
of Figure 5b. (a): Original Figure 5b. (b): Increasing L: i.e., =(0 0.11 and T()) 8 for
{1, {l..... 5). (c): Increasing treatment probabilities, i.c., (I (2)) x(3) (()) (0.1,0.2,0.4.006))
(d): Using Bernoulli randomized design.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Guidelines: Yes, the theoretical results in Sections and 3 and the empirical results in Section 4 and Appendix A support the main claims made the abstract and introduction.
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the work in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
16

---

Justification:
All the assumptions are clearly stated in Sections 2 and 3.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: All necessary information for reproducing the main experimental results are stated in Section 4.
Guidelines:
The answer NA means that the paper does include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer: [Yes]
Justification:
We will provide open access to the data and code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: /niiss wwtppcccpppppaeeeccccecccccceeee / ssissssiiciiyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
All the details of the experiments are provided in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
All of our experiments are replicated for 100 times and the error bars reported.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
18

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide the details of the computer resources in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics tttpp:://eerrps... http://urrsscccppulllpeesese
Answer: [Yes]
Justification:
Guidelines:
The research conducted in the paper adheres to the NeurIPS Code of Ethics.
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss broader impacts in Section 1.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer:[NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite the real network graph in Section 4.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We will release the code with documentation.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
21


Reviews:
[{"review_id": "w69WTywBoq", "reviewer": "Reviewer_p2A8", "summary": "The authors motivate a regression framework using Causal Message Passing to estimate global average treatment effects under some unknown forms of interference. The simplest way to think about their proposed algorithm is that CMP is used to motivate particular sufficient statistics for the interference dynamics which are fed into a regression model to perform corrections and enable estimation.", "questions": "see above", "limitations": "see above", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The CMP framework is, I think, a very interesting direction for research. In particular, it's a very important direction to weaken the kinds of assumptions about interference necessary to make when working with experiments.\nI also think there's something convenient in working with high-level statistics from experiments (for example, many large companies' experimental systems makes these particularly easy to retrieve, so reflecting that makes it easier to integrate proposed methods).", "weaknesses": "The biggest problem with this paper is that it doesn't actually set out theoretical scope conditions for when it works. I'll be a bit more specific on two points:\n1. The combination of the definition of TTE_t as the large sample limit combined with HO-CMP-I under Bernoulli randomization. Suppose W_t is determined by Bernoulli randomization (with a fixed probability, \\pi across all time periods). Then as N \\to \\infty, then \\bar{w}_t \\to \\pi. This means that the relationship between \\bar{w}_t and \\hat{\\nu}_t will always be exactly zero. Even under the useful linear form discussed on line 228, it appears that this method could not possibly learn anything effectively. I suspect, in fact, that this would also be problematic in finite sample sizes (see below notes on evaluation). Critically, however, you haven't spelled out any of these scope conditions, so the paper does not make clear what is necessary to achieve success. And I will reiterate that this is using the very asymptotics gestured to in the paper and an experimental design which is explicitly discussed.\n2. More specifically, I will return to line 228, which gestures at a generative model under which the proposed method (HO-CMP-I) should work well. But what does this mean? Should we expect the estimator to be unbiased? Should we expected it to be BLUE? Should we expect it to be semi-parametrically efficient? It would be ideal to have theoretical results around this, but barring that, these questions should be addressed directly and explicitly in the experiments.\n\nOn the subject of the experiments, I want to see more specifics than the charts you've provided give. Are the estimators unbiased? What are their MSEs? How do these evolve as a function of sample size? What are their convergence rates? In the absense of theoretical results, it is critical to have clear answers to these kinds of questions. \n\nSome minor points that I don't think matter much at all, but will share regardless:\n- I think \"higher-order\" is an overly confusing way to define this approach. When I think higher-order, I think about something like a Taylor series: using more granular information about the setting to define a better estimator. In contrast, your approach is doing something more akin to higher-_aggregation_. I would consider using a term like \"higher-level\" or something which connotes this kind of aggregation.\n- On line 211 you refer to \"multi-label\" ML models, but I think you mean \"multi-task\" models or \"multivariate\" models.\n- What you've done here doesn't seem obviously like \"machine learning.\" While I have no problem with linear regression, what it appears that you've done here\n- line 264: you don't need to tell the reader what \\pi is :)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rve9fl5Ho5", "reviewer": "Reviewer_YfYx", "summary": "This paper studies experimental interference, in which the treatment assignments of one unit affects the outcome of another. The majority of work in this area assumes interference acts through a network, and requires knowledge of that network to reduce bias in the resulting estimator. This paper takes a general approach in which interference occurs due to an unknown graph, and a unit's outcome can be affected by its neighbors' treatment status as well as their past outcomes. With such a flexible potential outcomes model, it is very difficult to estimate the total treatment effect in an unbiased way. The paper suggests an extension of the causal message passing algorithm that estimates the state evolution equations of the potential outcomes using higher-order functions of the past treatment status and outcomes, as opposed to the original CMP paper which used linear models of these quantities.", "questions": "1. Is the HO-CMP-I algorithm the same as that of Shirani and Bayati? If not, I would suggest including the original CMP algorithm as a baseline.\n2. [Authors do not need to provide new experiments; discussion would suffice.] The HO-CMP algorithm outperforms polyfit as shown in Figure 1. It's unclear to me if this is due to increased regularization (maybe a 2-degree polynomial provides a better approximation to a half-sine than a 4-degree polynomial on the data in Figure 1) or an inherent property of the HO-CMP algorithm (maybe the claim is that not all types of interference are captured by the polyfit algorithm?). I would be curious if the authors had some intuition here, perhaps an example where polyfit wasn't obviously overfitting, but HO-CMP still outperformed polyfit.", "limitations": "Yes", "rating": 3, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 1, "strengths": "The paper provides thorough experimental validation of the proposed method. In particular, the method is evaluated on simulated and real graphs, and is compared against three baselines including Cortez et al's polynomial fit method. The experiments are simulated under the staggered rollout design, which is of interest in the academic literature as well as in practical industry settings.\n\nThe ideas in the paper are presented well, with a high quality of writing throughout.", "weaknesses": "If I understand correctly, this work extends the CMP algorithm of Shirani and Bayati by using nonlinear estimators of the state dynamics. It is unclear to me that this delta is a significant enough improvement over the original CMP method to merit publication at Neurips.\n\nI reviewed the Shirani and Bayati paper, and am concerned at how closely the writing, structure, and in some cases individual sentences of this paper parallel that work. I will leave it to the AC to make any recommendations about copyright / academic integrity, but here are the passages of most concern:\n1. The phrase \"Inspired by the literature on AMP algorithms, we refer to (4) as state evolution equations of the experiment\" appears verbatim in the original CMP paper.\n2. The related works of this paper appears to be a reworded and edited version of a subset of the related works of the CMP paper. In particular, the paragraphs beinning on lines 124 and 134 almost exactly mirror their counterparts in the CMP paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a1rhOUYbjZ", "reviewer": "Reviewer_V4GS", "summary": "This paper discusses the causal effect estimation under spatiotemporal interference characterized by a dynamic system. The authors are devoted to improving the causal message-passing framework by adding interaction terms of inputs in the summary function $f_\\theta$ (a linear regressor). Synthetic and semi-synthetic experiments are conducted to show the performance of the proposed method.", "questions": "I have no questions.", "limitations": "No, the limitations of this paper are discussed inadequately, as discussed in weaknesses.", "rating": 3, "confidence": 5, "soundness": 1, "presentation": 2, "contribution": 1, "strengths": "1. The authors explicitly present their methodology with clear definitions and notations.\n2. The experiment results are summarized clearly.", "weaknesses": "1. The comparison with the most related work [1] is very important but scarce, and I think the marginal contribution is little.  \nThe basic framework is all the same with [1], and the scanty innovation of this paper is the so-called \"higher order\", which is formulated as two more interaction terms and a square term. I don't think these additional input terms are necessary when given a non-linear model $f_\\theta$, especially when the authors don't develop any theory that necessitates the simplicity of linear regression.  \n\n2. The literature review is disordered, and there are apparent misunderstandings for certain important works. I list some examples in the following. \n- The taxonomy \"Restrictions on interference structure\" and \"Treatment effect dynamics and temporal interference\" overlap severely, wherein [2] studies the general temporal interference characterized by a MDP, while the authors list this paper with other two-sided marketplace paper in \"Restrictions on interference structure\".  \n- I don't understand why [3] and [4] are discussed in the category of \"Partial interference\", since these two papers are important and don't belong to this category. \n\n3. More discussion on the basic setting is needed. I think neither the authors nor [1] discuss the concrete and applicable scenario of the dynamic system to show it's indeed meaningful, especially when they are agnostic to the network. I take network interference as an instance for the cross-sectional part. As the number of units increases, how the network topology change indeed impact the scenario, e.g. whether the degree of units increases with $n$ (Erdos-Renyi) or just keeps constant? The details behind the asymptotics indeed matter.\n\n\n\nRef. \n[1] Sadegh Shirani and Mohsen Bayati. Causal message passing: A method for experiments with unknown and general network interference. arXiv preprint arXiv:2311.08340, 2023.  \n[2] Vivek Farias, Andrew Li, Tianyi Peng, and Andrew Zheng. Markovian interference in experiments. Advances in Neural Information Processing Systems, 35:535–549, 2022.  \n[3] Christina Lee Yu, Edoardo M Airoldi, Christian Borgs, and Jennifer T Chayes. Estimating the total treatment effect in randomized experiments with unknown network structure. Proceedings of the National Academy of Sciences, 119(44):e2208975119, 2022.  \n[4] Johan Ugander, Brian Karrer, Lars Backstrom, and Jon Kleinberg. Graph cluster randomization: Network exposure to multiple universes. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 329–337, 2013.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LRnPJjazj5", "reviewer": "Reviewer_YRHE", "summary": "The paper considers the problem of total treatment effect estimation under interference, when the interference structure is unknown.  It proposes an estimation approach based on fitting a data-driven model for low-dimensional dynamics of the mean and variance of observed experimental outcomes over time, motivated by state evolution dynamics.  These state evolution equations come from approximate message-passing (AMP) applied to the potential outcome dynamics chosen in Equation 2.  To produce total treatment effect estimates, initial means and variances are observed from the experiment and propagated forward via the learned state evolution model.", "questions": "1.  Why only consider the mean of W_t as a feature in Table 1? \n2.  Equation 3: sums should be over i not n?\n3.  What are the initial conditions of the counterfactual estimation part of Algorithm 1?\n4.  In Equation 2, are the diagonal entries (i=j) treated the same as the off-diagonal entries?  What is the motivation for this?", "limitations": "Limitations are adequately addressed.", "rating": 3, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper investigates an important unsolved problem in causal inference.\n- The data-driven model of the state evolution equations does not rely upon exact functional forms of the dynamics and instead attempts to learn the appropriate dynamics from the observed data.  This increases methodological flexibility by allowing a portion of the dynamics, in addition to the interference, to remain unknown.\n- The empirical analysis includes experiments beyond the setting under which the estimator is derived ( e.g. interference occurs across binary edges, the outcome generating process used, and staggered rollout design), increasing the relevance for practice.  Improvements are seen over the few available baselines for the unknown interference setting.", "weaknesses": "- The authors should clarify the scope of their algorithm.  Equation 2 defines specific dynamics and does not meet the generality emphasized in the introduction and abstract around equilibrium and interference.  These dynamics do not obviously cover what may be occurring in real-world systems (e.g. lacking memory and being additive).  The authors motivate their paper as relaxing assumptions, but introduce their own alternative assumptions in Section 2.1.  The necessity of those assumptions is also undiscussed, leaving the range of applicability unclear.  For example, the interference matrix is assumed iid Gaussian but the experiments contain binary edges.  \n- While past research Causal Message Passing is cited prominently in the paper, the paper could use more discussion over what is novel and different here.  To my understanding, the novel contribution is primarily the HO-CMP algorithms introduced in Section 3.  While the experiments demonstrate improvements by using HO-CMP, the idea of data-driven modeling of the state evolution equations appears a relatively straightforward extension.  \n- The HO-CMP algorithms in Table 1 only leverage simple feature specifications that are low-dimensional. \n- Algorithm 1's computational complexity and scalability is unaddressed.\n- Overall the paper could use increased clarity.  Many complex ideas are mentioned and occasionally described in insufficient detail.  For example, the authors do not justifying the necessity of considering state evolution instead of modeling Equation 2 directly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work mirrors somewhat closely the work of Sadegh Shirani and Mohsen Bayati (https://arxiv.org/abs/2311.08340) and in light of that (beside concerns of plagiarism or self-plagiarism, since I don't see the authors' identities), the contribution of this submission remains a bit incremental.
Decision: Accept (poster)