Pessimistic Data Integration for Policy Evaluation
Authors: Xiangkun Wu
Ting Li
Gholamali Aminian
Armin Behnamnia
Hamid R. Rabiee
Chengchun Shi
Venue: NeurIPS 2025 poster ()
Keywords: Data Integration
Policy  Evaluation
Causal Inference
A/B testing
Primary area: general_machine_learning

Abstract: This paper studies how to integrate historical control data with experimental data to enhance A/B testing, while addressing the distributional shift between historical and experimental datasets. We propose a pessimistic data integration method that combines two causal effect estimators constructed based on experimental and historical datasets. Our main idea is to conceptualize the weight function for this combination as a policy so that existing pessimistic policy learning algorithms are applicable to learn the optimal weight that minimizes the resulting weighted  estimator's mean squared error. Additionally, we conduct comprehensive theoretical and empirical analyses to compare our method against various baseline estimators across five scenarios. Both our theoretical and numerical findings demonstrate that the proposed estimator achieves near-optimal performance across all scenarios.

Pessimistic Data Integration for Policy Evaluation
Xiangkun Wu" School of Mathematical Sciences Zhejiang University Hangzhou, China 122350310zju. edu.
Ting Li"
School Statistics and Data Science Shanghai University of Finance and Economics
Shanghai, China tingli@mail shufe. thuue.edu.ccm cn
Gholamali Aminian
Armin Behnamnia
The Alan Turing Institute London, UK gaminianSturing ac. uk
Sharif University of Technology
Tehran, Iran arminbehnamni a@gmail............mm.m com
Hamid R. Rabiee
Chengchun Shi
Department of Computer Engineering
Department Statistics
Sharif University of Technology
London School of Economics and Political Science London, UK
Tehran, Iran rabiee@sharif.. edu
C. Shi701se.... ac. uk
Abstract
This paper studies how to integrate historical control data with experimental data to enhance A/B testing, while addressing the distributional shift between histori- cal and experimental datasets. We propose pessimistic data integration method that combines two causal effect estimators constructed based on experimental and historical datasets. Our main idea is to conceptualize the weight function for this combination as a policy so that existing pessimistic policy learning algorithms are applicable to learn the optimal weight that minimizes the resulting weighted esti- mator's ' mean squared error. Additionally, we conduct comprehensive theoretical and empirical analyses compare method against various baseline estimators across five scenarios. Both our theoretical and numerical findings demonstrate that the proposed estimator achieves near-optimal performance across all scenarios.
Introduction
A/B testing is widely used by various technology companies such as Amazon, Google, Netflix, Uber, and Didi evaluate the performance of new products, policies treatments compared to existing controls. However, the effectiveness of such evaluations is often limited by short duration of online experiments. For instance, in ridesharing, most experiments last no more than two weeks [1]. Before conducting these experiments, companies usually have access to a substantial amount of historical data collected under the control policy. Recent work has demonstrated that integrating these historical control data with experimental data can largely improve the efficiency of A/B testing [2].
The primary challenge in data integration stems from the distributional shift between historical and experimental data, which can generally be categorized into three types: (i) covariate shift the changes in the distribution of contextual covariates; (ii) policy shift the changes in the behavior
"Equal contribution. *Coresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Reward (Estimated
(Context)
Actor (Treatment)
Weight
ATE
Figure 1: Workflow of the proposed estimator.
policy or propensity score; and (iii) posterior shift the changes in the outcome distribution given covariates and treatment. Such distributional shifts can substantially bias the resulting treatment effect estimator and hinder the effective use of historical data for A/B testing.
This paper studies the integration of historical data to improve A/B testing, while addressing all three types of distributional shifts simultancously. Our contributions are as follows.
Methodologically, we propose a weighted average treatment effect (ATE) estimator that optimally combines information from both experimental and historical datasets. Our main idea is the devel- opment a pessimistic data integration approach that conceptualizes the weight function for data combination policy, which enables existing pessimistic policy learning algorithms to learn this optimal weight function; see Figure for an illustration of our methodology.
Theoretically, we conduct comprehensive comparative analysis to compare our proposed weighted ATE estimator against five baseline estimators across five different scenarios, reflecting differing degrees of posterior shift between experimental and historical datasets, and different levels of heavy- tailedness in reward residuals. Our analysis reveals that while each baseline estimator may perform optimally under certain scenarios, they often fail in others. In contrast, the proposed estimator is adaptive: satisfies the oracle property across all scenarios, meaning that it achieves comparable performance to the optimal scenario- specific estimator, working effectively if it knew the optimal weight function. Our theories are further supported by synthetic and real-world data analyses. Table 1 summarizes our theoretical and numerical findings, with detailed descriptions of the baseline estimators and scenarios provided in Sections 3.2 and 4.
2 Related work
Our paper is closely related data integration, the pessimistic principle in offline policy learning and off-pooiccy evaluation (OPE), which we claborate below.
Data integration. Data integration is related to various fields in statistics and machine learning, ranging from the classical meta-analysis [3], to more recent advancements in transfer learning [4],
and their applications to numerous downstream tasks such multi-task learning [5], multimodal
learning [6. 7], fusion learning [8], individualized treatment regime estimation [9, 10], and RL [11].
Our proposal is particularly related those methods tailored for causal inference; see [12] and [13] for recent reviews. Based on their approach to handling distributional shifts, these methods can roughly be classified into two categories:
The first category addresses the covariate and policy shifts under the assumption of no posterior shift [14-19]. A primary example is given by Li et al. [20], who integrated experimental data with historical controls and developed an estimator that achieves the asymptotically smallest MSE.
The second category handles all three types of distributional shifts simultancously [21-24]. Among these methods, a notable subset applied an lytyye penalty function for selecting external data [25, 26). Recent advancements include federated causal inference approaches [27, 28), which employed such penalization to estimate ATE in manner that preserves privacy.
Our approach differs from both categories the following ways. Methodologically, we overcome several critical limitations of the aforementioned methods, resulting in more flexible and practical

---

Table 1: MSEs of different ATE estimators across five scenarios. Green indicates that the estimator achieves the oracle property (its MSE is asymptotically equivalent to that of the optimal estimator). Yellow indicates that the estimator may generally have high MSE but can attain the oracle property in some special cases. Red indicates that the estimator exhibits a generally large MSE.
Scenarios
EDO
HDB
MVE
CWE
NonPessi
Proposed
(i) Heavy-tailed historical rewards
(ii) Heavy-tailed experimental rewards
(ii) (ii)Small posterior shifts
(iv) Moderate posterior shifts
(v) Large posterior shifts
solution. First, unlike the first category of methods which assumes no posterior shift, our proposal accounts for this shift. Second, whereas the second category of methods requires the posterior shift to be either zero or sufficiently large for consistent data selection failing in intermediate regimes [2] our method remains robust even with moderate posterior shifts (see Corollary 4).
In terms of applications, our work focuses on combining experimental and historical data for A/B testing, whereas most works either integrate data from multiple treatment centers or trials for meta analysis [29-32], or combine RCT and observational data handle unmeasured confounding [33-35].
Pessimistic policy learning. The pessimistic principle is fundamental to most cxisting offline policy learning algorithms, which aim to learn optimal policy from a pre-collected historical dataset. This principle originates from the seminal works of Swaminathan and Joachims [36, 37], who proposed a counterfactual risk minimizing approach that incorporates the uncertainty of policy's 's value estimator as a penalty term to learn policies with lower-variance value estimates. It has been widely employed in contextual bandits [38-41], dynamic treatment regimes [42], RL [43-53], and more recently in the training of large language models [see e.g., 54-56] to prevent value function overestimation and encourage the learning policy stay close the behavior policy.
A recent proposal by Li et al. [2] applied the pessimistic principle to data integration for policy evaluation. In particular, they proposed to linearly combine policy value estimators computed from experimental and historical datasets using a weighted average. While their approach is closely related to ours, key difference lies in their use of fixed weight function for data integration. In contrast, our approach employs a covariate-dee -eeeedetedeeeeeent weighting, leading to a more accurate estimator than theirs, demonstrated analytically in Sections 4 and empirically in Section 5.
huge literature on OPE bandits and RL; see [57] and [58] for reviews. The goal of OPE is to estimate the expected outcome of a target policy using offline data collected under potentially different behavior policy. Existing approaches can be classified into four main categories: (i) Model-based methods that estimate dynamic model (c.g.. a Markov decision process) from offline data and compute the target policy's 's expected outcome via dynamic programming Monte Carlo [59-62]; (ii) Direct methods that estimate the expected outcome by learning either a reward or value function from the offline data [63-73]; (iii) Importance sampling (IS) methods that reweight observed rewards using the IS ratio (the density ratio between target and behavior policies) 844: (iv) Doubly or multiply robust methods that combine the estimated reward or value function from direct methods with the IS ratio from IS methods, and require only the reward/value function or the IS ratio to be consistent [85-99].
Off-policy evaluation. There
Many of these approaches have been recently adopted for A/B testing [see e.g. 100-111J. However, these works rely solely on experimental datasets, without leveraging historical datasets to improve policy evaluation.
More recently, [112] proposed doubly robust (DR) estimator by combining data from multiple experimental studies. Their approach requires the target outcome and covariate distribution to match those in at least one experimental dataset. We also notice that there an emerging line of research that studied how to improve OPE estimation by strategically combining multiple base OPE estimators to leverage their strengths [see e.g., 38, 113-115). While similar spirit to our combination of OPE estimators from experimental and historical datasets, the base estimators in these papers were derived from single dataset (thus avoiding distributional shifts). On the contrary, we need to address the challenges posed by the distributional shift between experimental and historical datasets.
3

---

3 Pessimistic data integration
We first introduce two baseline OPE estimators (formally defined in Equations (1) and (2)). We next introduce our proposed estimator, which builds upon these baseline estimators.
3.1 Two baseline estimators
Suppose are given an experimental dataset D(e) and historical dataset D(h), During the experiment, at each time f, the decision maker observes certain contextual covariates (c.g.. market features), and assigns action between new treatment strategy (denoted by 1) and a baseline control (0), resulting in a reward that measures the company's 's profit at that time. Thus, the experimental data can summarized as a set of context-action-reward Ole) = (S(e), A R(()) triplets, which are assumed to be i.i.d. over time. Similarly, the historical dataset consists of another set of i.i.d. triplets ( = (s(h), AA) R()),, but differs in distribution from ((e) in the following three aspects:
1. Covariate shift: the probability mass function of ( (denoted by Pc) might differ from that of Sp. (denoted by Ph), leading the IS ratio ((s) = P. (s) generally deviating from
2. Policy shift: actions in the historical dataset are exclusively generated under the control policy such that A (h) = 0 almost surely, whereas actions in the experimental dataset are generated under both the control and the treatment for A/B testing.
Posterior shift: the reward function (a, s)
E(R a, (())()) s) in the experimental dataset might differ from that in the historical dataset (denoted by r((h).
Our objective lies in estimating the ATE the difference between the expected reward under the treatment and that under the control, i.e..
ATE = (1, (1,) (0,
using both experimental and historical datasets.
The first baseline estimator for ATE introduce is the experimenntll-dtt-onnyy (EDO) estimator, which uses exclusively the experimental dataset D(e) to learn the ATE. This estimator is simple to describe: we construct two OPE estimators using D(e) to estimate the expected outcomes under treatment 1 and 0, respectively, and compute their difference to obtain the ATE estimator. Notice that any OPE method discussed in Section 2 can be applied for estimation.
As concrete example, consider the IS estimator with the estimating function v/ (O(*)) = I(A a) RR(e) (m S(e)) where TTAA == ())x(alS) denotes the IS ratio of the target policy the hehavior policy (i.e., propensity score) π the experimental data. Using the change of measure theorem, can be shown that (O(e)) is unbiased to the expected outcome under treatment This motivates the use E,,, [v5h (vs( (O(=)) to estimate this expected outcome, leading to the following EDO estimator for the ATE,
EDO E, (v ( (O(e)) - E, (o)))(((())))
where IE,, denotes the empirical average over the offline dataset.
The second baseline estimator is the historical-dan data-based (HDB) estimator. Similar to EDO, it uses E, (e) (O(e))] to estimate the expected outcome under the new treatment. For the control policy, the R(M) is constructed using solely the historical data. Here, the IS ratio fa(:) denotes the density ratio of the probability mass/density
corresponding estimator E,, (h) (O))) with vig (O(M) = ((S())))()
function of g(e) over that of g(e), It depends only on the contextual variable, since the historical data is exclusively generated under the control policy, leading to a propensity score T (A(h) s(h) of almost surely. Similarly, it can be shown that this estimator is unbiased to (E[((") (0, summary, we have
5(h)). (0,,5)) In
HDB E,
(o (o(e)) E,, (0((0)))))))
To conclude this section, we remark that there a bias-variance trade-off between the two estimators. Specifically, the HDB estimator is generally biased due the incorporation of historical data.

---

Table 2: A numerical example demonstrating the bias-variane trade-off (see Appendix for details). As shown, EDO achieves the lowest bias, while HDB attains the lowest variance. The proposed estimator strikes a balance between bias and variance, resulting in the lowest overall MSE.
Method
MSE( 95% CT)
Bias( % CT) )
Variance( 95% CT)
EDO
1.701 (1.598-1.844
0.007 (-0.064- 0,051)
1,701 (1.598- -1804)
HDB
2.372 (2.289-2.455)
1.400 (1.372-1.42))
0.413 (0.388 0.438)
Proposed
1.394 (1.312-1.476)
0.221 (0. 0.22222)))
1.345 (1.262-1.428)
Although it addresses the covariate shift through the use of the IS ratio μ., the posterior shift from the experimental data is extremely challenging to correct, resulting non-negligible bias. In contrast, the EDO estimator, derived exclusively from the experimental data, remains asymptotically unbiased. On the other hand, HDB typically achieves lower variance by leveraging the historical data, which usually has much larger sample size than the experimental data. Finally, our proposed estimator, which we introduce in the following section, effectively strikes balance between bias and variance and outperforms both baseline estimators; see Table 2 for illustration.
3.2 A pessimistic estimator for data integration
We begin with summary of our proposal. Our approach is linearly combine the two baseline estimators presented in Section 3.1 for data integration, while taking into account the posterior shift between the experimental and historical data. The key here determine the optimal weight (see e.g., Equation (3) below for the definition) for data combination. Our main idea is to transform this weight selection problem into offline policy learning. Specifically, we conceptualize the choice of weight as an 'action', which could vary as function of the contextual information. This conceptualization effectively frames the weight selection as a policy learning problem where the goal is to identify an optimal policy that maximizes reward or minimizes cost, the latter of which corresponds to the MSE of the weighted ATE estimator. Figure gives an overview of the proposed estimator pipeline. Adopting this perspective enables us to employ state-of-the-arr pessimistic policy learning algorithms such as counterfactual risk minimization to effectively determine the weight.
We next detail our methodology. Similar to EDO and HDB, our estimator employs E, [ ( (Ofe)) to estimate the mean outcome under the treatment policy. As for the control, it uses weight function wx)s) to linearly combine the estimating functions used in EDO and HDB. Specifically, we define the following estimating function,
di (O(()) (0()) w((()))) (c) (O)))) + [1 w (() (O(A).
It is immediate to that setting w 1 recovers EDO's 's estimating function v6()) whereas setting 0 recovers the HDB's 's estimating function ()))), This leads to the following weighted ATE
estimator,
ATE(w) (w) E, [v) (1 (O(e))) E,, (O(s), O(k)))
where the second empirical average E, is taken over all pairs of (O()),0(0) E D(c), D(N)
It remains to identify the optimal weight function w* that optimally balances the bias and variance of the ATE estimator to minimize its MSE. As mentioned earlier, we adopt an offline policy learning framework and view each value of w bounded between and 1- as an arm in a contextual bandit model. Given that w is function of the covariates, it defines policy on this contextual space. The identification of wn is thus equivalent to the identification of the optimal policy that minimizes the cost, which in our case equals the MSE.
Although the oracle MSE is unknown, it can be estimated from the offline data. Specifically, since EDO is derived solely from D.. it is expected to be asymptotically unbiased. Thus, its deviation from ATE(w) (i.e., bias(w) ATE(w) EDO) can be used to measure ATE(w)'s bias (denoted by bi). Additionally, its variance (denoted by Var(w)) can be estimated using the sampling variance formula (see Appendix B.3 for details). Denote the resulting variance estimator by Var(w). Given a parametric function class W, the optimal w* can be estimated by minimizing
2 MSE(w) w) = bias (w) + Var(w),
(5)
5

---

over E W. Following the pessimistic principle, we instead minimize an upper bound of the
estimated MSE given by
MSEg(1) (w) = biasy (w) + (w)),
(6)
where biasy and Vary are required to upper bound the oracle bias and variance so that the following assumption is satisfied.
Assumption 1 (Coverage probability). Assume WW (DDTwwwbbassr( (biase(ww) 2 [bias(w)))) 2 1 and P(Dwew (W)) 2 Var(w))) 2 a for some
< 1.
In practice, biasy: and Vary can be constructed using concentration inequalities [116] or Wald-type confidence intervals [117]. We detail our implementation in Appendix B.4. We also remark that for clarity of presentation, we focus on IS estimators for ATE estimation in this section. However, our actual implementation employs DR estimators, which are known to be more efficient than IS with well-specified reward models [1 [118]. The detailed formulas are relegated to Appendix B.2
Let i denote the minimizer of (6), which yields proposed estimator ATE( To conclude, remark that our framework unifies several baseline estimators through specific choices of
EDO: Setting @ recovers the experimentalldatt--on estimator;
HDB: Setting @ yields the historical-data-basee estimator;
MVE: Omitting the bias term in (5) and minimizing (5) leads to the minimal-variance estimator
in Li et al. [20];
4. NonPessi: Minimizing (5) opposed to (6) produces the non-pessimistic estimator;
5. CWE (short for constant weight estimator): Restricting W constant functions of the context and setting Vary Var result the pessimistic estimator Li al. [2].
We will analytically compare these estimators in the following section.
Statistical properties and analy tical comparisons
We first analyze the MSE of our proposed estimator. We next analytically compare it against other baseline estimators. Our analysis covers five different scenarios:
(i) Heavy-tailed historical rewards, where the reward residual R(h) - r()()), s(h)) exhibits sub- stantial variability;
Haavy-tailed experimental rewards, where the reward residual from the control group exhibits substantial variability;
iiii Small posterior shifts, where the bias due to posterior shift ((((()((()))) (((e) (O(e)) is much smaller than the standard deviation of its estimator E,, (()()(((M)))))))))))) - w/ (O(e))
(iv) Moderate posterior shifts, with the bias being much larger than the estimator's standard deviation, yet falling within its high-confidence bound, making it undetectable from the data;
(v) Large posterior shifts, where the bias is larger than the upper confidence bound of Ey (k)) (O(h)) - (((()) (Ote))), allowing to be detected from the data.
See the formal definitions of these scenarios in Corollaries 1 4. While each of the aforementioned baseline estimators might be optimal in certain scenarios, they can perform poorly in others. In contrast, the proposed estimator is adaptive and robust: performs comparably to the scenario-specific optimal estimators in most cases. See Table for a summary.
We begin by introducing boundedness assumption and presenting upper bound for the MSE of the proposed estimator. To simplify the theoretical analysis, we follow [2] and study a sample-split version of the ATE estimator where half of the data triplets in D(e) and (hh) are used to estimate by solving (6), while the remaining half are used to construct the ATE estimator in (4).

---

Assumption 2 (ATE boundedness). There exists some constant B >> O such that both the absolute values of ATE and our estimator ATE(2) ()) are upper bounded by B.
Theorem 1 (MSE of the proposed estimator). Under Assumptions 1 and 2, we have for any w E W, MSE(ATE ()) MSE(ATE(())) can be bounded by:
E ( (w) - bias (w)) + E V/Varo (w) - Var(w)] + O(aB²).
Theorem 1 is generic in that it applies to any OPE estimator direct, IS or DR - used to learn the ATE, provided that Assumptions 1 and are satisfied. We also remark that Assumption is mild. In practice, the size of the ATE is typically very small in A/B testing [102. 1 119, 108, 107]. Equation (7) upper bounds the difference in MSE between the proposed ATE estimator and any weighted estimator with a fixed weight function w. Under the realizability assumption [see e.g.. 120] where w" E W. setting w = =u in Theorem leads to an upper bound on the difference between the MSE of our estimator and that of the optimal weighted estimator. According to (7), this upper bound can be decomposed into three parts: the first two terms quantify the estimation errors of the squared bias and the two variances respectively, and the last term, being proportional to a, represents the probability of under-coverage- - the probability that biasy or Vary fails to upper bound the oracle bias or variance.
Notice that through the use of concentration inequalities, the last term can be made arbitrarily small without largely inflating the estimation errors of the bias and variance. As for the first two terms, a key observation is that the bias and variance upper bounds in these terms depend on the weight function only through fixed w, rather than the estimated weight T. This arises from the pessimistic principle, which, in policy learning, ensures that the regret of the estimated policy depends only on the reward estimation error under the optimal action, rather than under the estimated optimal action [121, 122]. In our setting, this principle is crucial for enabling the proposed pessimistic estimator to achieve adaptivity. To elaborate, we impose the following conditions.
Assumption 3 (Coverage). The probability mass functions of both (A(e), S(el) and ((h) are bounded from below by some constant c >> O.
Assumption 4 ( (Additive noise). Assume R(h) = =,(1) (0,, (t)) + (1)) for some mean-zero random error (b) independent of S(h), Similarly, assume R((e) = (A(()),,)))) + Alc) for mean-zero random errors e and c(²) independent of S(c) and A(e).
Assumption 5 (Reward function boundedness). The reward functions (b)) and p(e) are uniformly bounded in absolute value by some constant Tmax 0.
The coverage and boundedness assumptions are commonly imposed in RL and OPE [see e.g.. 123, 124]. Note that the boundedness condition applies only to the reward function, not to the reward itself. The reward being the sum of the reward function and the residual - can be unbounded due to the potential heavy-tailedness of the residual. The additive noise assumption is widely imposed in machine learning and statistics [see e.g., 125, 126]. Under this assumption, we use o and (()) to (e) respectively. These standard deviations are used to denote the standard deviations of e (h) and ED
characterize the tails of these error distributions. Specifically, in the first two scenarios with heavy- tailed reward residuals, ((h) and g(e) can be substantially large. These two cases naturally favor EDO and HDB as optimal estimators, respectively, since they avoid incorporating heavy-tailed rewards for ATE estimation. In the last three scenarios, we measure the posterior shift by the reward difference b(x) = (h) (0, s) - p((e) (0, 8). When [b(x)] is small so that variance dominates the squared bias, MVE asymptotically optimal since is designed for variance minimization. With moderate-to-larg values of ]b(s)], EDO becomes again the optimal estimator as it avoids bias by excluding the historical dataset from the ATE estimation. The following corollaries demonstrate that our proposed estimator performs comparably to these optimal estimators across all scenarios, maintaining robustness with either heavy-tailes reward residuals or posterior shift.
Corollary 1 (Scenario (i)). Assume Assumptions 1- hold. Let 8 I/D(t)) denote the ratio between the sample sizes of the two datasets. Then with heavy-tailed historical rewards where ((h) > j (B(((()) + mmax)), w (s) + 1 for any s so that EDO becomes the asymptotically optimal estimator. By setting w in Theorem to 1, the difference in MSE between the proposed estimator and EDO is
E E|Var (EDO) Var(EDO)) + O(aB²), +(abB)
(8)
7

---

which is much smaller than MSE(EDO) itself under mild conditions specified in Appendix C.4. Corollary (Scenario (ii)). Assume Assumptions 1 1-5 hold. Then with heavy- experimental rewards where (()) > [F-1/2
+ V\D(e)) w (8) - 0 for any so that HDB becomes the asymptotically optimal estimator. Additionally, the difference in MSE between the proposed estimator and HDB is much smaller than MSE(HDB) itself under mild conditions specified in Appendix C.5.
Corollary 3 (Scenario (iii). Assume Assumptions 1 5 hold. Then with small posterior shifts such that [b(s] < min(a(s) /DD(e ath)) // //D(")] MVE achieves the smallest MSE. Additionally, the difference in MSE between the proposed estimator and MVE is much smaller than MSE(MVE) itself under certain conditions specified in Appendix C.6.
Corollary 4 (Scenarios (iv) and (v)). Assume Assumptions - hold and that either b(s) > 0 ffr all or b(s) < 0 for all Then with either moderate posterior shifts such that
(()) + Fmax
(1h) + Tmax <
<1b(b)]
-
(()))
+ Tuax
log [D())
((h) Tmax
V log (D(h))
D
VElDINT
Ve(Die]
VelDinij
for any 8, or large posterior shifts such that
[b[s)] >> (o(e) + Tmax
(h) Tmax
log [Die)]+ +
log (D(N)))
Ve|Die)]
Ve(DiM)
for any (s) + 1 for any so that EDO becomes the asymptoticallly optimal estimator. Addition- ally, the difference in MSE between the proposed estimator and EDO is upper bounded by (8), which is much smaller than MSE(EDO) itself under mild conditions specified in Appendix C.7.
Corollaries - 4 upper bound the excess MSE of the proposed estimator over the scenario-specioo optimal estimators across Scenarios (i)-(v) Importantly, the excess MSEs in (i), (iv) and (v) are independent of (Ah) b(s). which demonstrates become (moderately) large in their respective scenarios. Furthermore, these corollaries establish the oracle property of our estimator: it asymptotically achieves the same MSE as the optimal estimator for each scenario, working efficiently as if knew the underlying scenario.
estimator's robustness when these parameters
We next compare against the baseline estimators mentioned in Section 3.2 analytically.
EDO: According to Corollaries and 4, EDO is asymptotically optimal in Scenarios (i), (iv) and (v). However, it underperforms our estimator in Scenarios (ii) and (iii), where incorporating historical data yields more accurate ATE estimation.
HDB: As demonstrated in Corollary 2, HDB is asymptotically optimal in Scenario (ii). However, unlike the proposed estimator, it generally fails in Scenarios (i), (iii), (iv) and (V).
MVE: Corollary 3 shows that MVE is asymptotically optimal in Scenario (iii.. However, it suffers from a large bias in Scenarios (iv) and (v), due to the posterior shifts.
NonPessi: Similar to Corollaries 3 and 4, we can show that NonPessi is asymptotically optimal in scenarios (iii) and (v) when the posterior shift is either small or large. However, it not optimal for moderate shifts in Scenario (iv). This is because without adopting the pessimistic principle,
its excess MSE depends on the estimation errors bias ()) and Var(w) the estimated weight @. Although the optimal population-level weight w² + with moderate posterior shifts, the estimated to may not, since the bias is not large enough be detectable [2]. Similarly, in the first two scenarios with heavy-tailed rewards, NonPessi unlike the pessimistic estimator - can suffer from a large MSE when (e) and (h)) are large, yet not sufficiently so to be detected from the data.
CWE: While [2] showed that CWE optimal in Scenarios (iv) and (v), it differs from our method in two ways: (a) it restricts W to constant weight functions, and (b) it applies the pessimistic principle only partially - to upper bound the squared bias term but not the variance. (a) leads to its sub-optimality in Scenario (iii), where the optimal weight for MVE is typically context-dependent rather than being constant. Similar to NonPessi, (b) makes CWE sub-optimal in the first two scenarios.
We have so far focused exclusively on the estimation of the ATE, To conclude this section, we remark that our proposal also accommodates valid inference for A/B testing. By employing sample-splittngg
8

---

and doubly robust ATE estimation, valid p-values can be readily obtained when combined with. Specifically, we use one half of the data to estimate the weight function w and nuisance functions (the reward and density ratio), and the other half to construct the ATE estimator. Following [127], one can show that the resulting ATE estimator is asymptotically normal under suitable regularity conditions. As a result, standard z-tests based on normal approximation can be used to obtain valid p-values. Our numerical studies in Section 5 confirm that the resulting p-values remain valid across all experimental settings.
5 Numerical experiments
In this section, we evaluate the finite-sample performance of the proposed estimator, comparing against EDO, MVE, CWE, and NonPessi (introduced in Section 3.2). We exclude HDB as performs similarly or worse than MVE in our experiments. Instead, we include LASSO, proposed by Cheng and Cai [25], which selects weights by minimizing the estimated variance of the ATE estimator with Lasso penalty. MSEs are computed over 100 simulation replications. Details of the data generating process are provided in Appendix A.
Example 5.1 (Synthetic-data simulation). We design settings to cover all five scenarios mentioned in Section 4 and Table 1. Specifically, we model the difference between reward functions b(s) as jair x d(s) for some nonzero function d and a scalar parameter jai E [0, 5] controlling the degree of posterior shift. When /aam = O, the reward functions are identical, indicating no posterior shift. Increasing /Aar leads to larger shifts. This covers Scenarios (iii) - (v), which range from small to moderate large posterior shifts. We also consider two forms for the function d: (i) piecewise function of the context variable, and (ii) a linear function of the context variable, resulting in piecewise and linear shifts, respectively. Finally, we allow the reward residuals follow either normal distribution (light-tailed) or a Student's 's t-distribution with 6 degrees of freedom (heavy-tailed). This covers the first two scenarios.
The top panels of Figure 2 report the MSEs of all ATE estimators under piecewise shifts, while the middle panels exclude MVE, which exhibits large MSEs even under moderate to large posterior shifts, to allow for a clearer comparison of the remaining estimators. It can be seen that when Jair is small, MVE achieves the lowest MSEs, and the proposed method performs comparably. As Jai increases, the proposed estimator outperforms all baseline alternatives in most cases. Notably, even when Mair is large where EDO is expected to perform best our proposal still achieves lower MSES under normally distributed experimental reward residuals. This benefits from its use of a context-adaptive weight function. When the reward difference b(s) is negative for some contexts and positive for others. a properly chosen context-adaptive weight can still incorporate historical data to reduce variance while effectively cancelling out bias. In contrast, CWE and LASSO, which rely on constant weights, tend to converge toward EDO's 's performance.
Bottom panels of Figure 2 show similar trends under linear shifts (excluding MVF). We also consider a nonlinear form of d(x)) and conduct additional experiments in Appendix A, which confirm similar patterns under nonlinear posterior shifts. Finally, we remark that the LASSO estimator is implemented using a carefully chosen tuning parameter to ensure competitive performance. Additional results in Appendix A (Figure 8) reveal LASSO's s sensitivity this hyperparameter.
Example 5.2 (Ridesharing-data -dat-b---e- simulation). In this example, we construct a simulation environment based on real-world A/A dataset collected from ridesharing platform. The contextual information includes two variables: the total online time of drivers and the number of order requests across one day. The reward is defined as the total daily income earned by each driver. We first learn the outcome model using these variables from this A/A dataset, and then generate synthetic experimental and historical data based on this model, following scenarios similar to those in Example 5. Results reported in Figures 3 and additional results in Appendix A (Figures 9-12) align with the findings in Example 5.1, where the proposed estimator achieves the lowest MSEs in most cases.
Additionally, we conduct clinical data based simulation in Example A.1 (Appendix). The results exhibit patterns consistent with those in Examples 5. 1.22, where the proposed estimator outperforms competing methods most cases. Furthermore, we assess the inference procedure by testing the nullity of the ATE and comparing it with those based on EDO and CWE. As shown in Table 3, all three methods adequately control the Type error under the null (ATE 0), while the proposed test demonstrates higher power under the alternative.

---

EDO
Proposed
Piecewise normal
Piecewise normal4
Piecewise t-normal
Piecewise t-t
2.0
M56
MSE
MSE
0.5
0.5
Halff
Halft
Hoatt
0.43
0.42
0.42
0.00
0.000
0.40
0.40
0.30
0.29
Halff
Halft
Holft
Linear normal-normal
Linear normal-t
Linear t-normal
Linear
0.350
8.59
MBE
0.003
1.30
1.50
688
0.873
Hdiff
Haiff
Adif
Maiff
Figure 2: MSEs in Example 5.1. Top panels show all estimators under piecewise shifts; middle panels zoom without MVE; bottom panels present results for linear shifts excluding MVE.
EDO
CWE
NonPessi
Proposed
LASSO
MVE
normal-normal
normal-t
t-normal
1-t
20
10
15
0.
Haitt
Hditt
Hattt
Hott
0.475
0.450
0.50
0.500
0.42
6.625
0.60
0.400
0.28
0.75
Haiff
Hdif
Hdlff
Heiff
Figure 3: MSEs of ATE estimators in Example 5.2. Top: all estimators; Bottom: exclude MVE.
Discussion
In this work, we study how to integrate historical control data with experimental data to enhance A/B testing. We proposed a covariate-dependent weighting scheme that treats the weight as policy and learns it by minimizing pessimistic upper bound on the estimator's MSE. We establish MSE bounds for the resulting estimator. We evaluate it against competitive basclines across five representative settings. Both theoretical analysis and empirical results demonstrate greater robustness to heavy-tailed rewards and near-optimal handling of diverse posterior shifts.
Several directions merit future exploration. The proposed method focuses on non-dynamic setting, whereas in many practical applications, treatments are sequential and may influence future outcomes. A natural extension is to accommodate dynamic settings with carryover effects by explicitly modeling underlying dynamics. Our theoretical analysis also relies on bounded-reward assumptions, which could be relaxed to handle unbounded outcomes. Beyond IS and DR estimators for the ATE, the proposed framework can be extended more general off- off-policy evaluation and reinforcement learning objectives, including least-squares temporal-different methods and fitted Qevvaluation.
10

---

Acknowledgments
Xiangkun Wu's research is supported by the National Key Research and Development Program of China (Grant No. 2024YFC251 1003). Ting Li's research is partially supported by the National Natu- ral Science Foundation of China (No. 12571304), the Shanghai Pujiang Program (No. 24PIC030), CCF-DiDi GAIA Collaborative Research Funds and the Program for Innovative Research Team of Shanghai University of Finance and Economics.
References
[1] Zhe Xu, Zhixin Li, Qingwen Guan, Dingshui Zhang, Qiang Li, Junxiao Nan, Chunyang Liu, Wei Bian, and Jieping Ye. Large-scale order dispatch on-demand ride-hailing platforms: A learning and planning approach. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 905-913, 2018.
[2] Ting Li, Chengchun Shi, Qianglin Wen, Yang Sui, Yongli Qin, Chunbo Lai, and Hongtu Zhu. Combining experimental and historical data for policy evaluation. In Proceedings of the 4 st International Conference on Machine Learning, pages 28630-28653 2024.
[3] Rebecca DerSimonian and Nan Laird. Meta-analysis in clinical trials. Controlled clinical trials, 177):177-188 1986.
[4] Sai Li, T Tony Cai, and Hongzhe Li. Transfer learning for high-dimensional linear regression: Prediction, estimation and minimax optimality. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(1):149-173, 2022.
[5] Lu Tang and Peter XK Song. Fused lasso approach in regression coefficients clustering- learning parameter heterogeneity data integration. Journal of Machine Learning Research, 17(113):1-23, 2016.
[6] Fei Xue and Annie Qu. Integrating multisource block-wise missing data in model selection. Journal of the American Statistical Association, 1166536):911--922;; 2021.
[7] Qi Xu and Annie Qu Representation retrieval learning for heterogeneous data integration. arXiv preprint arXiv:2503.0944 2025.
[8] Jieli Shen, Regina Y Liu, and Min-ge Xie. fusion: Individualized fusion learning. Journal of the American Statistical Association, 115(531) 2020.
[9] Chengchun Shi, Rui Song, Wenbin Lu, and Bo Fu. Maximin projection learning for optimal treatment decision with heterogeneous individualized treatment effects. Journal of the Royal Statistical Society Series B: Statistical Methodology, 90(4):681-702 2018.
[10] Xinlei Chen, Victor B Talisa, Xiaoqing Tan, Zhengling Qi, Jason N Kennedy, Chung-Chou H Chang, Christopher W Seymour, and Lu Tang. Federated learning of robust individualized decision rules with application heterogeneous multihospital sepsis population. The Annals of Applied Statistics, 19(2):1270-1291 2025.
[11] Zhuangdi Zhu, Kaixiang Lin, Anil K Jain, and Jiayu Zhou, Transfer learning in deep reinforee- ment learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45 (11):13344-1333) 13362, 2023.
[12] Irina Degtiar and Sherri Rose. A review of generalizability and transportability. Annual Review of Statistics and Its Application, 10(1):501-524, 2023.
[13] Xu Shi, Ziyang Pan, and Wang Miao. Data integration in causal inference. Wiley Interdisci- plinary Reviews: Computational Statistics, 15(1)=1181, 2023.
[14] Nathan Kallus, Yuta Saito, and Masatoshi Uehara. Optimal off-policy evaluation from multiple logging policies. International Conference on Machine Learning, pages 5247-5256. PMLR, 2021.
11

---

[15] Sijia Li and Alex Luedtke. Efficient estimation under data fusion. Biometrika, 1 110(4): 1041-1054, 2023.
[16] Shu Yang, Chenyin Gao, Donglin Zeng, and Xiaofei Wang. Elastic integrative analysis of randomised trial and real-world data for treatment heterogeneity estimation. Journal of the Royal Statistical Society Series B: Stafistical Methodology, 85(3):575-596 2023.
[17] Haotian Wang, Haoxuan Li, Wenjing Yang, Hao Zou, Wanrong Huang, and Kun Kuang. Estimating treatment effect across heterogeneous data sources: An instrumental variable approach. In NeurIPS 2024 Causal Representation Learning Workshop, 2024.
[18] Chenyin Gao, Shu Yang, Mingyang Shan, Wenyu Ye, Ilya Lipkovich, and Douglas Faries. Improving randomized controlled trial analysis via data-adaptive borrowing. Biometrika, 112 (2)asac0699, 2025.
[19] Xueqing Liu, Qinwei Yang, Zhaoqing Tian, Ruocheng Guo, and Peng Wu. Optimal policy adaptation under covariate shift. arXiv preprint 2at.:222100000... 2025.
[20] Xinyu Li, Wang Miao, Fang Lu, and Xiao-Hua Zhou, Improving efficiency of inference in clinical trials with external control data. Biometrics, 79(1):394-40) 2023.
[21] Ruoxuan Xiong, Allison Koenecke, Michael Powell, Zhu Shen, Joshua T Vogelstein, and Susan Athey. Federated causal inference in heterogeneous observational data. Statistics in Medicine, 42(24):4418 443444994144499 2023.
[22] Ying Sheng. Jing Qin, and Chiung-Ya Yu Huang. Sequential data integration under dataset shift. Technometrics, 66(4):662-677, 2024.
[23] Peng Wu, Shanshan Luo, and Zhi Geng. On the comparative analysis of average treatment effects estimation via data combination. Journal of the American Statistical Association, pages 1-12, 2025.
[24] Yi Liu, Alexander W Levis, Ke Zhu, Shu Yang, Peter Gilbert, and Larry Han. Targeted data fusion for causal survival analysis under distribution shift. arXiv preprint 1rXiv:2501.18798. 2025.
[25] David Cheng and Tianxi Cai. Adaptive combination of randomized and observational data. arXiv preprint arXiv:2111 11.1112,11 2021.
[26] Issa J Dahabreh, Sarah E Robertson, Lucia C Petito, Miguel A Hernán, and Jon A Stein- grimsson. Efficient and robust methods for causally interpretable meta-analysis: Transporting inferences from multiple randomized trials target population. Biometrics, 79(2):1057-1072. 2023.
[27] Larry Han, Zhu Shen, and Jose Zubizarreta. Multiply robust federated estimation of targeted average treatment effects. Advances in Neural Information Processing Systems, 36:70453- 70482. 2023.
[28] Larry Han, Jue Hou, Kelly Cho, Rui Duan, and Tianxi Cai. Federated adaptive causal estimation (face) of target treatment effects. Journal of the American Statistical Association, (Accepted), 2025.
[29] Rebecca DerSimonian and Nan Laird. Meta-analysis in clinical trials revisited. Contemporary clinical trials, 45:139-145, 2015.
[30] Takahiro Hasegawa, Brian Claggett, Lu Tian, Scott Solomon, Mare Pfeffer, and Lee-Jen Wei. The myth of making inferences for an overall treatment efficacy with data from multiple comparative studies via meta-analysis. Statistics Biosciences, 9:284-297, 2017.
[31] Qinshu Lian, Jing Zhang, James S Hodges, Yong Chen, and Haitao Chu. Accounting for post-randomizztion variables in meta-analysis: A joint meta-regression approach. Biometrics, 79(1):358-36 2023.
12

---

[32] Kollin W Rott, Gert Bronfort, Haitao Chu, Jared D Huling, Brent Leininger, Mohammad Has- san Murad, Zhen Wang, and James S Hodges. Causally interpretable meta-analysis: Clearly defined causal effects and two case studies. Research Synthesis Methods, 15(1 15(1):61-72, 2024.
[33] Bénédicte Colnet, Imke Mayer, Guanhua Chen, Awa Dieng, Ruohong Li, Gaël Varoquaux, hean-Philippp Vert, Julie Josse, and Shu Yang. Causal inference methods for combining randomized trials and observational studies: review. Statisticul science, 39(1):165-191, 2024.
[34] Piersilvio De Bartolomeis, Javier Abad Martinez, Konstantin Donhauser, and Fanny Yang. Hidden yet quantifiable: A lower bound for confounding strength using randomized trials. In International Conference on Artificial Intelligence and Statistics, pages 1045-1053, PMLR, 2024.
[35] Guido Imbens, Nathan Kallus, Xiaojie Mao, and Yuhao Wang. Long-term causal inference under persistent confounding via data combination. Journal of the Royal Statistical Society Series B: Statistical Methodology, 87(2):62-3388 2025.
[36] Adith Swaminathan and Thorsten Joachims. Batch learning from logged bandit feedback through counterfactual risk minimization. The Journal of Machine Learning Research, 16(1): 1731-1755, 2015.
[37] Adith Swaminathan and Thorsten Joachims. The self-normalized estimator for counterfactual learning. Advances in Neural Information Processing Systems, 28, 2015.
[38] Yi Su, Maria Dimakopoulou, Akshay Krishnamurthy, and Miroslav Dudík. Doubly robust off-policy evaluation with shrinkage. In International Conference on Machine Learning, pages 9167-9176. PMLR, 2020.
[39] Chenjun Xiao, Yifan Wu, Jincheng Mei, Bo Dai, Tor Lattimore, Lihong Li, Csaba Szepes- vari, and Dale Schuurmans. On the optimality of batch policy optimization algorithms. In International Conference on Machine Learning, pages 1 11362-11371. PMLR, 2021.
[40] Gene Li, Cong Ma, and Nati Srebro. Pessimism for offline linear contextual bandits using Lp confidence sets. Advances in Neural Information Processing Systems, 35:20974-20987, 2022.
[41] Olivier Jeunen and Bart Goethals. Pessimistic decision-making for recommender systems. ACM Transactions on Recommender Systems, 1(1):1-27, 2023.
[42] Yunzhe Zhou, Zhengling Qi, Chengchun Shi, and Lexin Li. Optimizing pessimism dynamic treatment regimes: A bayesian learning approach. In International Conference on Artificial Intelligence and Statistics, pages 6704 67211 PMLR, 2023.
[43] Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine Conservative q-learning for offline reinforcement learning. Advances in Neural Information Processing Systems, 33: 1 1179-1191, 2020.
[44] Tianhe Yu, Garrett Thomas, Lantao Yu, Stefano Ermon, James Y Zou, Sergey Levine, Chelsea Finn, and Tengyu Ma. Mopo: Model-based offline policy optimization. Advances in Neural Information Processing Systems, 13:14129-141427 2020.
[45] Masatoshi Uehara and Wen Sun. Pessimistic model-based offline reinforcement learning under partial coverage. In International Conference on Learning Representations, 2022.
[46] Tengyang Xie, Ching-An Cheng, Nan Jiang, Paul Mineiro, and Alekh Agarwal. Bellman- consistent pessimism for offline reinforcement learning. Advances in Neural Information Processing Systems, 34:6683-6694, 2021.
[47] Laixi Shi, Gen Li, Yuting Wei, Yuxin Chen, and Yuejie Chi. Pessimistic q-learning for offline reinforcement learning: Towards optimal sample complexity. In International Conference on Machine Learning, pages 19967-20025. PMLR, 2022.
[48] Xiaohong Chen, Zhengling Qi, and Runzhe Wan. Steel: Singularity-aware reinforcement learning. arXiv preprint 13152, 2023.
13

---

[49] Wenzhuo Zhou. Bi-level offline policy optimization with limited exploration. Advances in Neural Information Processing Systems, 36:55022-55033 2023.
[50] Yue Wang, Zhongchang Sun, and Shaofeng Zou, A unified principle of pessimism for offline reinforcement learning under model mismatch. Advances in Neural Information Processing Systems, 37:9281 -9288, 2024.
[51] Danyang Wang, Chengchun Shi, Shikai Luo, and Will Wei Sun. Pessimistic causal reinforce- ment learning with mediators for confounded offline data, arXiv preprint arXiv:2403. 2024.
[52] Jin Zhu, Runzhe Wan, Zhengling Qi, Shikai Luo, and Chengehun Shi. Robust offline reinforce- ment learning with heavy-tailed rewards. In International Conference on Artificial Intelligence and Statistics, pages 541-549 PMLR, 2024.
[53] Jin Zhu, Xin Zhou, Jiaang Yao, Gholamali Aminian, Omar Rivasplata, Simon Little, Lexin Li, and Chengchun Shi. Semi-pessimistic reinforcement learning. arXiv preprint S.Kiv22505.19002 2025.
[54] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744, 2022.
[55] Banghua Zhu, Michael Jordan, and Jiantao Jiao. Principled reinforcement learning with human feedback from pairwise or k-wise comparisons. In International Conference on Machine Learning, pages 43037-43067. PMLR, 2023.
[56] Pangpang Liu, Chengehun Shi, and Will Wei Sun. Dual active learning for reinforcement learning from human feedback. arXiv preprint arXiv:2410.02504, 2024.
[57] Miroslav Dudik, Dumitru Erhan, John Langford, and Lihong Li. Doubly robust policy evaluation and optimization. Statistical Science, pages 485-511, 2014.
[58] Masatoshi Uehara, Chengehun Shi, and Nathan Kallus, A review of off-policy evaluation in reinforcement learning. arXiv preprint arXiv:2212.06355, 2022.
[59] Josiah Hanna, Peter Stone, and Scott Niekum. Bootstrapping with models: Confidence intervals for off-policy evaluation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.
[60] Omer Gottesman, Yao Liu, Scott Sussex, Emma Brunskill, and Finale Doshi-Velez. Combining parametric and nonparametric models for off-policy evaluation. In International Conference Machine Learning, pages
PMLR, 2019.
[61] Ming Yin and Yu-Xiang Wang. Asymptotically efficient off-policy evaluation for tabular reinforcement learning. In International Conference on Artificial Intelligence and Statistics, pages 3948-3958. PMLR, 2020.
[62] Shuguang Yu, Shuxing Fang. Ruixin Peng, Zhengling Qi, Fan Zhou, and Chengchun Shi. Two-way deconfounder for off-policy evaluation in causal reinforcement learning. Advances Neural Information Processing Systems, 37:78169-78200, 2024.
[63] Steven Bradtke and Andrew G Barto. Linear least-squares algorithms for temporal difference learning. Machine learning, 22(1):33-51) 1996.
[64] Lihong Li, Rémi Munos, and Csaba Szepesvári. Toward minimax off- off-policy value estimation. In Artificial Intelligence and Statistics, pages 608-616. PMLR, 2015.
[65] Wei Luo, Yeying Zhu, and Debashis Ghosh. On estimating regression-based causal effects using sufficient dimension reduction. Biometrika, 104(1):51-63 2017.
[66] Hoang Le, Cameron Volohin,, and Yisong Yue. Batch policy learning under constraints. In International Conference on Machine Learning, pages 3703-3712, PMLR, 2019.
14

---

[67] Yihao Feng, Tongzheng Ren, Ziyang Tang, and Qiang Liu. Accountable off-policy evaluation with kernel bellman statistics. In International Conference on Machine Learning, pages 3102-3111. PMLR, 2020.
[68] Botao Hao, Xiang Ji, Yaqi Duan, Hao Lu, Csaba Szepesvari, and Mengdi Wang. Bootstrapping fitted q-evaluation for off-policy inference. In International Conference on Machine Learning, pages 4074-4084. PMLR, 2021.
[69] Peng Liao, Predrag Klasnja, and Susan Murphy. Off-policy estimation of long-term average outcomes with applications to mobile health. Journal of the American Statistical Association, 1 (16(533):82-3311 2021.
[70] Chengchun Shi, Sheng Zhang, Wenbin Lu, and Rui Song. Statistical inference of the value function for reinforcement learning in infinite-horizon settings. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(3):765-793, 2022.
[71] Andrew Bennett, Nathan Kallus, Miruna Oprescu, Wen Sun, and Kaiwen Wang. Efficient and sharp offl-y evaluation in robust markov decision processes. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[72] Masatoshi Uehara, Haruka Kiyohara, Andrew Bennett, Victor Chernozhukov, Nan Jiang. Nathan Kallus, Chengchun Shi, and Wen Sun. Future-dependent value-based off-policy evaluation in pomdps. Advances in Neural Information Processing Systems, 36, 2024.
[73] Zeyu Bian, Chengchun Shi, Zhengling Qi, and Lan Wang. Off-policy evaluation in doubly inhomogeneous environments. Journal of the American Statistical Association, 120(550): 1102-1114, 2025.
[74] James J Heckman, Hidehiko Ichimura, and Petra Todd. Matching as an econometric evaluation estimator. The Review of Economic Studies, 65(2):261-294 1998.
[75] Keisuke Hirano, Guido W Imbens, and Geert Ridder. Efficient estimation of average treatment effects using the estimated propensity score. Econometrica, 71(4): 161-1189, 2003.
[76] Philip Thomas, Georgios Theocharous, and Mohammad Ghavamzadeh. High-confidence off- policy evaluation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 29, 2015.
[77] Qiang Liu, Lihong Li, Ziyang Tang, and Dengyong Zhou Breaking the curse of horizon: Infinite-horizon off-policy estimation. Advances neural information processing systems, 31, 2018.
[78] Bo Dai, Ofir Nachum, Yinlam Chow, Lihong Li, Csaba Szepesvári, and Dale Schuurmans. Coindice: Off-policy confidence interval estimation. Advances Neural Information Process- ing Systems, 33:9398-944 2020.
[79] Mark Rowland, Anna Harutyunyan Hado Hasselt, Diana Borsa, Tom Schaul, Rémi Munos, and Will Dabney. Conditional importance sampling for off off-policy learning. In International Conference on Artificial Intelligence and Staristics, pages 45 55. PMLR, 2020.
[80] Muhammad Faaiz Taufiq, Araad Doucet, Rob Cornish, and Tean-Francois Ton. Marginal density ratio for off-policy evaluation in contextual bandits. Advances in Neural Information Processing Systems, (06655648-5269 2023.
[81] Yuchen Hu and Stefan Wager. Off Off-policy evaluation in partially observed markov decision processes under sequential ignorability. The Annals of Statistics, 51(4):1561-1585 2023.
[82] David M Bossens and Philip S Thomas. Low variance off- off-policy evaluation with based importance sampling. 2024 IEEE Conference on Artificial Intelligence (CAI), pages 871 883. IEEF, 2024.
[83] Shuze Liu and Shangtong Zhang. Efficient policy evaluation with offline data informed behavior policy design. In International Conference on Machine Learning, pages 32345- 32368. PMLR, 2024.
15

---

[84] Hongyi Zhou, Josiah P Hanna, Jin Zhu, Ying Yang. and Chengchun Shi. Demystifying the paradox importance sampling with an estimated Distory-dependen behavior policy in off-policy evaluation. In Forty-second International Conference on Machine Learning. 2025.
[85] Zhiqiang Tan. Bounded, efficient and doubly robust estimation with inverse weighting. Biometrika, 97(3):661-68) 2010.
[86] Pedro HC Sant' Anna and Jun Zhao. Doubly robust iiirrenceiin-dfffeennees estimators. Journal of econometrics, 219(1): 2020.
[87] Miroslav Dudik, John Langford, and Lihong Li. Doubly robust policy evaluation and learning. In Proceedings of the 28th International Conference on International Conference on Machine Learning, pages 1097-1104. 104,, 2011.
[88] Baqun Zhang, Anastasios A Tsiatis, Eric Laber, and Marie Davidian. robust method for estimating optimal treatment regimes. Biometrics, 68(4): 2012.
[89] Nan Jiang and Lihong Li. Doubly robust off ooflpo value evaluation for reinforcement learning.
In International Conference on Machine Learning, pages 652-661. PMLR, 2016.
[90] Philip Thomas and Emma Brunskill. Data-efficient off-policy policy evaluation for reinforce- ment learning. In International Conference on Machine Learning, pages 2139-2148. PMLR, 2016.
[91] Mehrdad Farajtabar, Yinlam Chow, and Mohammad Ghavamzadeh. More robust doubly robust off -fo-colicy evaluation. International Conference on Machine Learning, pages 1447-1456. PMLR, 2018.
[92] Chengehun Shi, Wenbin Lu, and Rui Song. Breaking the curse of nonregularity with inference of the mean outcome under optimal treatment regimes. Journal of Machine Learning Research, 21(176):1-67, 2020.
[93] Masatoshi Uehara, Jiawei Huang, and Nan Jiang. Minimax weight and q-function learning for off-policy evaluation. In International Conference on Machine Learning, pages 9659-9668. PMLR, 2020.
[94] Chengchun Shi, Runzhe Wan, Victor Chernozhukov, and Rui Song. Deeply-debiased off- policy interval estimation. In International conference on machine learning, pages 9590-9591. PMLR, 2021.
[95] Nathan Kallus and Masatoshi Uchara. Efficiently breaking the curse of horizon in off-policy evaluation with double reinforcement learning. Operations Research, 7066):3282-3302, 2022.
[96] Peng Liao, Zhengling Qi, Runzhe Wan, Predrag Klasnja, and Susan A Murphy. Batch policy learning in average reward markov decision processes. Annals of statistics, 50(6):3364, 2022.
[97] Yang Xu, Chengehun Shi, Shikai Luo, Lan Wang, and Rui Song. Quantile off ool-cy evaluation via deep conditional generative learning. arXiv preprint arXiv:2212.14466 2022.
[98] Jeonghwan Lee and Cong Ma. Off-policy estimation with adaptively collected data: the power of online learning, Advances in Neural Information Processing Systems, 37:133908-133947, 2024.
[99] Chengchun Shi, Jin Zhu, Ye Shen, Shikai Luo, Hongtu Zhu, and Rui Song. Off-policy confidence interval estimation with confounded markov decision process. Journal of the American Statistical Association, 119(545):73322888 2024.
[100] lavor Bojinov and Neil Shephard. Time series experiments and causal estimands: exact randomization tests and trading. Journal of the American Statistical Association, 114(528): 1665-1682, 2019.
[101] Peter W Glynn, Ramesh Johari, and Mohammad Rasouli. Adaptive experimental design with temporal interference: maximum likelihood approach. Advances Neural Information Processing Systems, 33:15054-15064 2020.
16

---

[102] Vivek Farias, Andrew Li, Tianyi Peng, and Andrew Zheng. Markovian interference in experiments. Advances in Neural Information Processing Systems, 35:535-549, 2022.
[103] Chengchun Shi, Runzhe Wan, Ge Song, Shikai Luo, Hongtu Zhu, and Rui Song. A multiagent reinforcement learning framework for off-policy evaluation in two-sided markets. The Annals of Applied Statistics, 17(4):2701-222) 2023.
[104] Chengchun Shi, Xiaoyu Wang, Shikai Luo, Hongtu Zhu, Jieping Ye, and Rui Song. Dynamic causal effects evaluation in a/b testing with reinforcement learning framework. Journal of the American Statistical Association, 118(543):20992007) 2023.
[105] Ting Li, Chengchun Shi, Jianing Wang, Fan Zhou, et al. Optimal treatment allocation for efficient policy evaluation in sequential decision making. Advances Neural Information Processing Systems, 36:48890 48905, 2023.
[106] Ting Li, Chengchun Shi, Zhaohua Lu, Yi Li, and Hongtu Zhu. Evaluating dynamic conditional quantile treatment effects with applications in ridesharing. Journal of the American Statistical Association, 1196547) 2024.
[107] Ke Sun, Linglong Kong, Hongtu Zhu, and Chengchun Shi. Arma-design: Optimal treatment allocation strategies for a/b testing in partially observable time series experiments. arXiv preprint 2rXiv:2408.05344 2024.
[108] Ruoxuan Xiong, Alex Chin, and Sean Taylor. Data-driven switchback experiments: Theoret- ical tradeoff's and empirical bayes designs. arXiv preprint arXiv:2406.06768, 2024.
[109] Shikai Luo, Ying Yang. Chengchun Shi, Fang Yao, Jieping Ye. and Hongtu Zhu. Policy evaluation for temporal and/or spatial dependent experiments. Journal of the Royal Statistical Society Series B: Statistical Methodology, 86(3):623-649, 2024.
[110] Jinjuan Wang, Qianglin Wen, Yu Zhang, Xiaodong Yan, and Chengchun Shi. A two-armed bandit framework for a/b testing. arXiv preprint arXiv:2507.181/ 2025.
[111] Qianglin Wen, Chengchun Shi, Niansheng Tang, Hongtu Zhu, et al. Unraveling the interplay between carryover effects and reward autocorrelations in switchback experiments. In Forty- second International Conference on Machine Learning, 2025.
[112] Yonghan Jung and Alexis Bellot. Efficient policy evaluation across multiple different exper- imental datasets. Advances Neural Information Processing Systems, 37:136361 -33392, 2024.
[113] Mengjiao Yang. Ofir Nachum, Bo Dai, Lihong Li, and Dale Schuurmans. Off-policy evaluation via the regularized lagrangian. Advances in Neural Information Processing Systems, 33:6551- 6561, 2020.
[114] Christina Yuan, Yash Chandak, Stephen Giguere, Philip Thomas, and Scott Niekum. Sope: Spectrum of off-policy estimators. Advances in Neural Information Processing Systems, 34: 18958-18969, 2021.
[115] Allen Nie, Yash Chandak, Christina Yuan, Anirudhan Badrinath, Yannis Flet-Berliac, and Emma Brunskill. Opera: Automatic offline policy evaluation with we-weighted aggregates of multiple estimators. Advances Neural Information Processing Systems, 37:103652-103661 103680 2024.
[116] Stéphane Boucheron, Gábor Lugosi, and Olivier Bousquet. Concentration inequalities. In Summer school on machine learning, pages
Springer, 2003.
[117] George Casella and Roger Berger. Statistical inference. CRC press, 2024.
[118] Anastasios A Tsiatis. Semiparametric theory and missing data, volume 4. Springer, 2006.
[119] Susan Athey, Peter J Bickel, Aiyou Chen, Guido W Imbens, and Michael Pollmann. Semi- parametric estimation of treatment effects in randomised experiments. Journal of the Royal Statistical Society Series B: Statistical Methodology, 1515):1615-16638 2023.
17

---

[120] Shai Shalev-Shwartz and Shai Ben- Ben-David. algorithms. Cambridge university press, 2014.
Understanding machine learning: From theory to
[121] Ying Jin, Zhuoran Yang, and Zhaoran Wang. Is pessimism provably efficient for offline ri? In International Conference on Machine Leurning, pages 5084-5096. PMLR, 2021.
[122] Paria Rashidinejad, Banghua Zhu, Cong Ma, Jiantao Jiao, and Stuart Russell. Bridging offline reinforcement learning and imitation learning: A tale of pessimism. JEEE Transactions on Information Theory, 68(12):8156 8196, 2022.
[123] Jinglin Chen and Nan Jiang. Information - considerations in batch reinforcement learning. In International Conference on Machine Learning, pages 1042-1051. PMLR, 2019.
[124] Jianqing Fan, Zhaoran Wang, Yuchen Xie, and Zhuoran Yang. A theoretical analysis of deep q-learning. Learning for dynamics and control, pages 486-489. PMIR, 2020.
[125] Jonas Peters, Joris Mooij, Dominik Janzing, and Bernhard Schölkopf. Causal discovery with continuous additive noise models. The Journal of Machine Learning Research, 15(1): 2009-2053, 2014.
[126] Chengchun Shi, Yunzhe Zhou, and Lexin Li. Testing directed acyclic graph via structural, supervised and generative adversarial learning. Journal the American Statistical Association, 19(547):1833-1846 2024.
[127] Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters, 2018.
[128] Aad W Van Der Vaart and Jon A Wellner. Weak convergence. In Weak convergence and empirical processes: with applications to statistics, pages 16-28. Springer, 1996.
[129] Victor Chernozhukov, Denis Chetverikov, and Kengo Kato. Gaussian approximation of suprema of empirical processes. The Annals of Statistics, 42(4):1564, 2014.
[130] Victor Chernozhukov, Denis Chetverikov, and Kengo Kato. Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors. The Annals of Statistics, pages 2786-2819, 2013.
[131] lavor Bojinov, David Simehi-Levis and Jinglong Zhao. Design and analysis of switchback experiments. Management Science, 0711377777-7777777
18

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The abstract and introduction clearly describe the problem setting, the pro- posed method, and the scope the contributions. They outline the motivation (addressing distributional shift in data integration, particularly in the context of A/B testing), introduce our method pessimistic data integration approach), and summarize the main findings (near-optimal performance across five scenarios). These claims are substantiated by both theoretical analysis and empirical evaluations presented the main body of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? [Yes]
Answer:
Justification: We provide a detailed discussion in Appendix D.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
19

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: In Section 4, we present the full set of assumptions required by our analysis and discuss their practicality and commonality. We first derive general expression for the MSE of our proposed method and then provide performance results for five specific scenarios in the form corollaries. All assumptions and theorems are properly numbered and referenced. Complete and rigorous proofs are provided in Appendix C, where we also elaborate on how each assumption is used within the corresponding theorems.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yss]
We propose algorithm, which is described in detail in Section 3.2. Addi- tional implementation details are provided in Appendix B. To further enhance reproducibility, we include the full source simulation code in the supplementary material, covering both the algorithm implementation and the experimental setup.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
20

---

(c) If the contribution is a new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide full access to the simulation code and experimental scripts in the supplementary material, along with detailed instructions for reproducing the main experimental results. "README" file is included to guide users through the process. For the semi- synthiict experiment based on private data, we are unfortunately unable to release the dataset due to privacy constraints. However, we provide detailed descriptions of the experimental setup, processing procedures, and key results in Section 5,
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cc)
saaiii//uiiiess/ddeSubbiiii wwmmiiissssioccyooiccooiio for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / /nips. cc/public/gui hoooli/c/giddes/odeeuubiiiii www.bbmmmssioooioccy... for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer:[Yes]
Justification: We provide comprehensive experimental details, including hyperparameter settings, and training configurations. These are described in Section 5, with additional implementation and tuning details provided in Appendix A and the supplementary code. We also specify the choice of optimizer, the number of repetitions, and how parameters were selected ensure fairness across baselines. This information allows readers to fully interpret and evaluate the experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
21

---

The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:[Yes]
Justification: In the toy example2, we report 95% confidence intervals derived using the Central Limit Theorem, based on repeated simulations and the estimated standard errors. For the simulation and real-data-base over 100 independent runs. This motivated by the Law of Large Numbers, aiming to ensure statistical reliability and reduce the impact of randomness, rather than relying on small number of trials. Details of the error estimation process are provided in Appendix 5.
simulation experiments, we report the mean MSE
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: All experiments were conducted on higghppeffrmance computing node equipped with dual AMD EPYC 7742 64-Core Processors (128 logical cores). No GPUs were used. Each simulation experiment with 100 replications are typically completed within 90 minutes. We have provided the above details in Appendix A.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make into the paper).
22

---

9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: This research complies fully with the NeurIPS Code of Ethics. No human subjects personally identifiable information involved. The real-data-base simulation is based on private structured data that cannot be released due to legal and privacy constraints, but all usage complies with institutional and data protection regulations.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: This work contributes to methodological advancements data integration and policy evaluation, particularly in the context of A/B testing under distributional shift. Potential positive societal impacts include enabling more efficient and statistically sound experimentation in fields such as healthcare, education, and online platforms. One potential risk is the misuse of historical data integration sensitive domains, which could amplify existing biases if the data distribution is not properly accounted for.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
23

---

This paper does not involve the release of models datasets that earry a high risk of misuse. All simulations are based on synthetic or private structured data, and no pretrained generative models or scraped datasets are used. Guidelines:
Justification:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer:[Yes)
Justification: While not explicitly mentioned in the main text, we use standard Python packages such as NumPy.SciPy ,and scikit-learn in implementation. All usage complies with the respective license terms.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi wwwoee. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's ss creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper introduces a new method and provides implementation code for reproducibility. However, it does not release any new dataset, pretrained model, or software asset intended for reuse beyond the experimental context. Therefore, we consider that no new asset has been introduced.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/mmodel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.

---

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA].
Justification: This paper does not involve crowdsoureing or research with human subjects. We simply propose a new algorithm and evaluate it using simulation data.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve research with human subjects crowdsoureing. We propose new algorithm and evaluate using simulated data, which do not require IRB equivalent approval.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer:[NA]
The core methodology and experiments in this paper do not involve any large language models (LLMs) as important, original, or non-standard components.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM policy pettps:/nneuraai / for what should or should not be described.
25

---

Appendix
Appendix Contents
A Experimental Settings and Additional Results
27
Implementation Details
32
B.1 Importance Sampling Method
32
B.2 Doubly Robust Method
................................
32
B.3 Explicit Form of Var(w) and its Estimator Var(w)
................................
33
B.4 Explicit Form of MSE (w)
34
................................
B.5 Estimation of Nuisance Function
................................
36
B.6 Estimation of ((S)
37
B.7 Estimation of the weight function wSS)
37
Proofs of the Theorems and Corollaries
37
C.I Proof of Theorem
37
C.2 Supporting Lemmas
38
C.3 Preliminaries for the Proofs of Corollaries
40
C.4 Proof of Corollary
41
C.5 Proof of Corollary 2
43
C.6 Proof of Corollary
47
C.7 Proof of Corollary
49
D Limitation
50
26

---

A Experimental Settings and Additional Results
In this section, we present details of the data generating process for the toy example in Table and Section 5, and additional experimental results. All experiments were conducted on a high- performance computing node equipped with dual AMD EPYC 7742 64-Core Processors (128 logical cores). Each experiment with 100 replications can be typically completed within 90 minutes.
Toy Example in Table 2. We define the reward functions for the experimental and historical data as follows
10 + 3S. + GEe:
if Ae =0 10 + + Degt if Ag
R₂
R4 11.4 Sh. + 66h,
where the state variables Sn, S. and the errors Cb ce are independently drawn from the standard normal distribution (0,, 1). In the experimental data, actions are evenly split between the two treatment groups, with half assigned to and the other half
Example 5.1 (Continued). In this example, we consider five posterior shift scenarios as follows.
(1) Piecewise Shifts: The reward functions for the historical and experimental data are:
Re 10 As + Se 26e,
RA 10 + dμ (Sn)Nan Sh. (2 + ds (Sx)))
where
1, Sc-1
0, S<-1
ds(s) =
-1, -155 <0, dn(s))
1, -15$<0,
2, S>0
1.
S>0
and S. and S₁ are sampled from (N0 1). The noise terms 6, and 6, follow four distribution combinations: normal-normal, normal-t, t-normal, and t-t, where normal is N(,, 1) and t is the i-distribution with 6 degrees of freedom.
The parameter Aair captures distributional differences between datasets, discretized into 25 values from 0 to 5. In the experimental dataset, the action A, alternates deterministically between 0 and 1-0 for the first, third, and fifth samples, and 1 for the second, fourth, and sixth samples. The shift magnitude depends on the state S through (S), and the noise variance varies with the state via (S).
(2) Linear Shifts: The reward functions for the two datasets are
Re 10 A + Se 3ce, R 10 pai + Sa + 3Gh:
where Se, Sh, te, c.. and A₂ follow the same configurations to Setting (1).
(3) Cosine Shifts: The reward functions are given by
Re = 10 + A + cos(S.) + 3Ee, R4 = 10 + Man + Adift cos( + Sh ЗEA-
All other configurations remain the same Setting (1).
(4) Quadratic Shifts: The reward functions are given by
Re = 10 + A. + S² 3E Ra 10 + Maiff + S S2 + Sh 3Ea
All other configurations remain the same.
(5) Absolute Shifts: The reward functions are given by
R_ 10 + A. + SSal + 3 E2z
RA 10 + paa + Paa) Skl + |Sa| 3
All other configurations remain the same.
Figure 4 shows the empirical MSEs of all estimators, including MVE, under linear shifts. When paur small, MVE performs well due to minimal bias the historical data. However, as Aai grows, this bias increases sharply, causing MVE's MSE to rise substantially. Figures 5-7 present the empirical MSES under cosine, quadratic, and absolute shift settings. The results remain consistent across varying levels of posterior shift. The MVE method shows marginal gains only when pain is very small; however, performance quickly deteriorates as Maar increases, because it ignores the bias from the historical data, resulting in substantially large MSE.
27

---

EDO
CWE
NonPessi
Proposed
LASSO
MVE
Linear
t-normal
Linear
MSE.
MBE
MBE
1.30
Hoif
Hoift
Haif
Main
Figure 4: Empirical means of MSE for various estimators, including MVE, under the piecewise shifts and linear shifts.
When the experimental data follow normal distribution, method consistenlly outperforms all baselines over the entire range of jain E [0, 5]. regardless the data generating process of the historical dataset. It achieves much smaller MSE compared to EDO and Pessi when Maur is small. Although its MSE increases as jain grows, our method still maintains clear advantage over other baselines. This gain arises from the use of non-constant, learned weight function that adapts to distributional shifts, in contrast to EDO's fixed weight.
When the experimental data follow l-distribution, our method continues to perform well, particularly when Puiff 2. As Udiff increases, the performance of all methods converges to that of EDO.
EDO Cosine normal-normal
CWE NonPessi
Proposed
LASSO
MVE
Cosine norrmalt
Cosine (-noraal
Cosine t-t
MSE
MBE
Mail
HgifT
Hail
Maiff
Cosine normal-norral
Cosine - normal-4
Cosine t-normal
Cosine
0.353
1.35
1.35
0.303
1.30
1.30
1.25
Halff
Hditf
Hdit
Hait
Figure 5: Empirical means of MSEs Example 5.1 under cosine shifts. The top panel shows all estimators; the bottom panel zooms in by excluding the MVE method.
EDO
OWE Quadratic
NonPassi normal-t
Proposed
LASSO
MVE
normal-normal
Quadratic
Quadratic
MBE
MSE
MSEE
Haiff
Mdiff
Haif
Hdiff
Quadratic normal-normal
Quadratic normal-t
Quadratic t-normal
Quadratic
0.350
0:325
Hait
Hair
Hart
Ham
Figure 6: Empirical means of MSEs in Example 5.1 under quadratic shifts. The top panel shows all estimators; the bottom panel zooms in by excluding the MVE method.
Sensitivity Analysis of the Lasso Tuning Parameter. We investigate the performance of the Lasso estimator across range of tuning parameters À E {0.1, 0.2, ... 1.0), using the same data generating process as the piecewise shifts setting. Figure reports the performance Lasso and EDO across varying À values. For small values of Maiff, Lasso with small À outperforms EDO. However, its
28

---

EDO
OWE
NonPessi
Progosed
LASSO
MVE
Absolute
Absolute
MSE
MBE
MSE
MBE
2.5
Adit
Hait
Hail
Haiff
Absolute normal-normal
Absolute bnormal
Absolute
1.00
0:375
1.45
MSE
1.35
0.903
0.879
Hdiff
Udiff
Fdit
Adif
Figure 7: Empirical means of MSEs in Example 5.1 under absolute shifts. The top panel shows all estimators; the bottom panel zooms in by excluding the MVE method.
performance deteriorates Aui increases. In contrast, Lasso with large À performs comparably to EDO.
LASSO-11
LASSO-6.3
LASSD-O5 LASSD-0.
LASSD-0.7
LASSD-B
EDD
LASSO-42
LASS0-04
LASSD-3
LASSD-13
normal-normal
normal
E-normal
0.485
0.50
0.54
0.425
0.400
0.40
0.58
0.48
0.33
0.410
0.56
0.38
Malf
Uddt
Hain
Mott
Figure 8: Hyperparameter Sensitivity Analysis
Example 5.2 (Continued). This example uses real-world A/A experiment dataset from leading ridesharing platform. The reward R represents daily driver income, and the contextual variables Si and S22 denote the number of ride requests and total online time during the first hour of the day. respectively.
For privacy concerns, company and city identifiers omitted, and all variables scaled. The contextual features S₁ and S2 are normalized to have unit standard deviation. We fit the following linear model:
R= 3o + B1S1 + 32S2 +
and obtain the estimates Bo, 31 and 32. Based the estimated coefficients, we generate the experimental and historical datasets as follows:
R2 = 3. + A + S.Sa + B2Se2 + dec.
R, 3o 3, 3ySn2 Man + Pann (8.1 SI2) Sen.
where Se1, SKi are sampled from N (H1, 1) and Se2, S12 from N (H2, 1). with μ1 and μ2 being the empirical means of S and Sn from the real dataset. To ensure privacy, we do not report μ4 and μ2 individually, but their sum lies between 10 and 20. The action A, is binary, assigned deterministically: even-indexed samples receive A.
odd-indexed samples Ag 0. The noise terms and éj follow four combinations: normal-normal normal-t, t-normal, and t-t, where the t-distribution has degrees of freedom. The experimental dataset contains |D_| 48 samples, and the historical dataset has |D,l = m [Dal with m E 2,3 A noise scaling constant {1, 2,3) controls the noise magnitude.
Figure reports the empirical MSEs across methods, showing consistent patterns with the m case in the main text. Across all settings, method consistently achieves strong performance.
29

---

When the experimental data is heavy-tailed, it significantly outperforms non ponssimiiiiiti baselines with lower MSEs across all Mair values. With small posterior shifts, it clearly outperforms both EDO and Pessi; in the moderate shift regime, it outperforms Lasso; and under large posterior shifts, it remains stable and performs slightly better than EDO.
E20
GIE normal-t
NerPaol
Preposed
LADEO
m=2 normal-normal
m=2
m=2 t-normal
m=2
0.42
MSC
0.553
MSE
0.550
0.480
0.42
0.523
0.525
6.500
0.375
0.593
Haiff
Hdiff
Mdif
jddif
ED0
CRE m=3 normal-t
NanPaal
Propoaed
LASSO
m=3 normal-normal
m=3 t-normal
m=3 t-t
0.50
0.40
0.44
MSE
0.55
MSE
0.55
0.42
0.48
Hait
Hart
Hairt
Haitt
Figure 9: Empirical means of MSEs of various methods with = 1 in Example 5.2 for m = 2 (top)
and m 3 (bottom).
Figure 10 shows the results under varying noise magnitudes. As expected. the MSE of all methods increases with higher residual variance (characterized by 8), reflecting the impact of noise on estimation accuracy. Nevertheless, our method consistently outperforms all baselines across the full range of paa values.
NonPessi
Proposed
Delta=1.0
Delta=2.0
Delta=3.0
2.3
5.2
0,600
MSE
5.0
0.525
2.1
4.6
0.508
4,4
Haitl
Hdift
Hditf
Figure 10: Empirical means of MSEs of various methods with m in Example 5.2 across different ds.
Figure 11 examines the impact of treatment assignment under varying probabilities P(A, = 1)
prob, with prob E (0.3, 0.5, 0.7], in the normal-normal setting with m 1. The results show consistent performance across all methods, with our approach remaining robust and effective under different assignment probabilities.
EDO
CWE
NonPessi
Proposed
LAS8O
prob=0.3
prob=0.5
prob=0.7
0.54
0.46
0.44
0.525
MSR
0.475
0.48
0.38
Hdiff
Hdiff
Maiff
Figure 11: Empirical means of MSEs of various methods with m 1,5 in Example 5.2 in different prob scales.
30

---

Figure 12 examines the performance of various methods as sample size increases, focusing on the t-normal noise setting, As expected, MSEs decrease with larger sample sizes due to reduced variance. Notably, our method consistently performs well across all subplots and levels of jaift-
EDO
CWE
NorsPesai
Proposed
LASSO
Sample size 48
Sam ple size 72
Sample size
0.600
0.46
0.29
MSE
0.575
0.550
0.44
MSE
0.20
0.42
0.27
0.525
0.26
0.500
0.38
Hdiff
Hdiff
Hdiff
Figure 12: Empirical means of MSEs of various methods with m 1,6 in Example 5.2 across [Del = 48, 72, 96.
Example A.1 (Clinical-data-basee simulation). In this section, construct simulation envi- ronment based on the public real-world dataset ACTG175, which consists of 2,139 HIV-positive individuals randomized to four treatments. We focus comparing ZDV+ddI (n 522) and ZzV+zal (n = 524), treating them as actions and 0, respectively. The outcome of interest is the rescaled CD4 count, and consider three covariates: age (Si). ) homosexual activity (S2), and hemophilia (Sş). We construct a simulator similar to Example 5.2, and generate both experimental and historical data for evaluation.
Specifically, the outcome model specified as:
f(5) 3o + B1S + 32S1 8352 + 8453 +
We use the data to fit the model.Using fitted parameters, we generate synthetic outcomes based on real data:
f + 0.88ce: R₁ = f(A)) + Ma + 0.05 0.88ch,
where e, is drawn from N (0, 1) and ε₁ from either N N(0 1) or the heavy-tailed ts distribution; fg. This setting yields four possible
alternatively, <a is drawn from and 6₁ from either NO0, scenarios, corresponding to all combinations of c and 6 being sampled from standard normal or from heavy-tailed distribution. Here, S) represents covariates generated from normal distribution fitted using the empirical mean and variance of all observed Sj. Here, S% and Sy, are sampled from Bernoulli distributions with parameters set the empirical means of S2 and S3, respectively. We estimate the variance parameter as the average of the squared residuals from the fitted model. In experimental data, treatment assignments are randomized (A Bernoull((O.5). whereas in historical data they are fixed at A = 0. Both datasets contain 48 samples. We vary [0, 5] (25 points) and compare the empirical average MSE over 100 simulations. Results in Figure 13 reveal that the proposed estimator achieves the lowest MSEs in most cases.
EDO
CWE
NonPessi
Proposed
LASSO
MVE
normal-norral
normal-t
t-normal
12.5
12.5
11.0
10.9
MBEE
Hattt
Mattr
Matt
Mattt
0.33
0.20
0.47
3.40
3.45
0.43
3.44
0.30
6.02
1.43
Hallt
Hattf
Man
Mai
Figure 13: MSEs of ATE estimators in Clinical example. Top: all estimators; Bottom: exclude MVE. We also consider the following hypothesis testing problem:
Ho ATE 0 vs. H1 ATE +f 0.
31

---

To conduct valid inference, we estimate the ATE using doubly robust procedures and compute p-values using a Wald test. To compare the proposed method against EDO and Pessi, we first set the true ATE to O and conduct 1 1,000 Monte Carlo simulations to estimate the Type error rate of each method the 5% significance level. To assess empirical power, we gradually increase the signal strength by setting the ATE to {0.5, 1.0, 1.5, 2.0, 2.5, 3.0). We further examine three levels of posterior shifts: small, moderate, and large. Table reports the empirical rejection rates. Under the null where ATE equals zero, all three methods control Type error. Under the alternative, the proposed test more powerful than the two competitors.
Table Empirical Type-1 error rates and power under different bias levels. The best results in each row are highlighted in bold.
ATE
Metric
Small Posterior Shifts
Moderate Posterior Shifts
Large Posterior Shifts
EDO
CWE
Proposed
EDO
CWE
Proposed
EDO
CWE
Proposed
0.0
Type-T Error
0.030
0.032
0.024
0.030
0.035
0.029
0.030
0.030
0.027
0.5
Power
0.081
0.097
0.093
0.081
0.103
0.114
0.081
0.084
0.088
1.0
Power
0.267
0.310
0.320
0.267
0.311
0.335
0.267
0.276
0.286
1,5
Power
0.546
0,592
0.611
0,546
0.586
0.613
0,546
0,558
0.572
2.0
Power
0.810
0.847
0.867
0.810
0.835
0.863
0.810
0.815
0.825
2.5
Power
0.955
0.967
0.974
0,955
0.965
0.970
0.955
0.960
0.962
3.0
Power
0.994
0.996
0.997
0.994
0.996
0.997
0.994
0.994
0.994
B
Implementation Details
In this section, we provide detailed procedures for implementing our proposal based importance sampling estimator (Section B. and doubly robust estimator (Section B.2).
B.1
Importance Sampling Method
Recall that ( (d (O(e)) = I[(A = a) R(le) /m[a S(e),, and (O(A)) - ((S(i)))(()) weighted ATE estimator given by (4), and can be obtained follows.
The proposed
Step I: estimate 4! (O(o)) and (hh)) (O(h) using the methodology detailed in B.5 and B.6
Step 2: Estimate w(S) using Section B.7 by minimizing the upper bound of the estimated MSE derived in B.4.
Step 3: Obtain the weighted ATE estimator by plugging in the unknown terms with their estimates in (4).
To simplify notation, we define
z(o) (Ofr)) w(((e) (O(e)),
(w) = (1 w (( (9)
The weighted ATE estimator for given weight function thus be written
ATÉ(w) E,, (w)) E., ( (w)).
B.2 Doubly Robust Method
We present two baseline DR estimators before introducing the proposed weighted DR estimator. The first baseline estimator constructed using only experimental data,
(c)
Σ [blc]] (O(s)) (0()))
(e() - dr
Drl
O.ED.
where the estimating function (e for {0, 1} is given by
((A(c) a)
dr (O(e))
R(e) - (e) (A()), s(0)] + (a,
7(a S(e))
32

---

()(c It can be shown that dr (O(e)) is unbiased to the mean outcome under treatment as long as either the propensity score model æ or the outcome model r()) is correctly specified. This yields the doubly robustness property.
The second estimator incorporates historical data into the ATE estimation and defined as
(1) dr
dr, (O(+))
Σ
(0,
1
Σ
(h) Vär (O(N),
Die
(D(e]
OUNEDON
where the estimating function d (k) (O())) is given by
(K) (h) (o = (((())) R - (0, sth)) (o,))))) dr
Next, given a weight function w, we define the weighted DR estimator as: w (O(o))
(w)
Σ
1 Die]]
Σ
w(s ( (O())
D
Ol ED(
Σ
(1 (1-w(S(()) (0)) Sta))
D
gD(()
Σ (1-w(s(h)) 1 (O
(10)
[DNj
((M)E
Similar to IS, we define [ (w) and (h) (w) as
(w) (w) w/cc (Ole)) w (S Jo -1- (1= ((e)))
(11)
(w) (1 w w(Shh)) 2
(12)
Using these notations, the final ATE estimator is given by
ATE (w) = E,, ( (w) - En, (2 ( (w)).
To summarize, our DR ATE estimator can be constructed through the following steps.
Step estimate (e(e) (O(e)) for a E [0, (0) s(t) vi (h)) (O(k) using the method in
B.5and B.6.
Step 2: Estimate (8) using Section B.7 by minimizing the upper bounded of the estimated MSE B.4.
Step 3: Obtain the weighted ATE estimator by plugging in the unknown terms with their estimates in (10).
B.3 Explicit Form of Var and its Estimator Var(w)
In this part, we present the detailed expression for the variance term Var(w) in (5), and its estimator Var(w). Since the experimental and historical dataset are mutually independent, and samples within each dataset are independently and identically distributed, Var(w) can be written as:
Var(w) = Var (aTEA)() (ar(Z()(w)) + Var Var(z())(w) [D(e)
(13)
[D(A)
Therefore, to estimate Var(w), it suffices estimate Var (u:)) and Var (z (ar(z
(w)).
Their estimators Var(z(e)(w)) and Var(z(h) (w)) can be obtained using the standard sample variance formula,
DD
Var (()())))(w)) = [D(0)
Σ
(w)
E,, E.,
==1
DD(A
Var(z(1)(w)))
1
(2 (w)
(w) - EE, (.((((())w)
D(b)
j=1
33

---

where E, (xx))(((w)) and E, (2((()()) are defined as:
( (w),
(D!N)
En (m(0
(w): and En [x(
1
Σ (w) (w).
[D(e]
Diaj
This yields the variance estimator
Var (Z(e
Var (ar(z()(()) +
Var(w):= =
(14)
[D(e]]
[D())
B.4 Explicit Form of MSEr(w)
In this section, derive MSE (w) (6). It consists of the two terms: Varg (w) and biasy (w). We seek these two terms that satisfy the coverage probability in Assumption 1. In what follows, we take the bias term as an example and present three approaches for its construction: one based on empirical process theory [128], another based on Markov inequality and Bonferroni's 's inequality, and third based on the multiplier bootstrap [129].
For the first one, define the following function classes:
F(() = 11) = (1 (x))) NEW e
F(M) = { (0) = (1 -1- (s)))))) (0) e w].
Then, bias(w) bias(w) is the difference between two empirical processes indexed by w. the function class W satisfies certain complexity properties (e.g., being a VC class), one can apply empirical process theory to construct uniform upper bound U such that
P sup [bias(w) bias biissw)] S <U)
2
(15)
weW
or
bias(w) (w) - bias(w)
P sup
< <U 2 21-0. -
(16)
x(w)
WEW
where ((w) is a consistent estimator of the asymptotic variance of bias(w). Accordingly, we can set:
biasy (w) (w) U [bias(w)] in the unnormalized case;
biast(w) (w) ā (w) bias(w)) in the normalized case.
More specifically, when the error terms c(C), (f) are sub- Sab-Gaussian, we may set:
CT max
v log Rmax + log(1/a)
U
Amin
where > is a constant, v denotes the VC dimension of the function class W, 7max m(e) [D())] and Rmiin min([D(e)], (D(M))) This also be extended to heavy-tailed errors [see e.g., 129, Section 5].
Alternatively, under the following mnite-hypothesis-eee classs assumption, biasy and Vary can be con- structed based on asymptotic normality and Bonferroni's 's inequality.
Assumption 6 (Finite hypothesis class). The number of elements in W is finite.
Assumption 6 is commonly employed in machine learning to simplify the theoretical analysis [see e.g. 123].
Assumption 7. The error terms e c (e) and Eg assumed to have finite eighth moments.
34

---

Assumption 7 is standard in high-order moment analysis and is commonly adopted in the literature on finite-sample concentration .The class of distributions with a finite eighth moment very broad.
We define Vary (Z (w:) Var (Z (w:)) + U (w;) as the upper bound for Var (z(h) (w;) and Varu (2th) (wx)) Var (2th (wx) (w:) as the upper bound for Var (2((2()) (w;)). We consider a set W with K elements. Since the samples are independent and identically distributed (i.i.d.) and the experimental dataset D(e) is independent of the historical dataset D(4), we have:
(ar)((w;)= = Varu (z(e [D(e]
+ Varu (wx))
[D(N)]
Var (ws)) + (ws))
Var( (16) (w:)) + (wos)
+
[D(e)
D(Nj
Var(w;) ( (w:) (w;)
[D(e]]
[D(M)]
Assume there exist positive constants 01, ag 03, 04 > 0, and define Ovvar := ai + Q2 and Obiis := 03 + a4. For each i E {I...., K} we construct a confidence interval via Markov's inequality applied to the fourth moment:
Var (w:))) Var Var(Zie) (2(e (101)) 4 E
Varl (2x)))) 2 Varu (ww)))
Q1 (17)
s
(w,)
K
E (Var (2(h)) - Var (ar(z()) (w:))
Var (wx)) 2 Varu (w:)
02
s
(18)
(1i))
K
Direct calculations lead to:
P Var(w;) 2 Vare (azr)
P Var
((1)(w))) 2 Var (w:))
+ (wx))
Var
((b)(
Var
(2
+ ()(h(w))
P
Var ((()(ws))
Var
(w:))
( (w:)
P
Var
(wx)
Var
(z
(rh) (w;)
a2
O'aar
(19)
K
K
where the first inequality follows from the relationship between the sets, and the second inequality follows from the probability of the union bound.
We can write biasy (w:) as:
biasy = [bias(w;)) + (wx)) + 0ghh (wx)
(20)
35

---

This bound can also be derived via the fourth moment version of Markov's inequality,
P [bias(w:)= bias( S U/ (c:) + U,h (w:)
IE (a w- 10 1B(1 w(S(e)) 16 (O(el)) (O()))))))(())))) 47
Ug (w,)²
(11 wot - B(1 - (0/
+
(w:)4
01
044
Olbias
(21)
K
K
K
According to Bonferroni's inequality, we have
K
P
biasy (wi) 2 [bias(w:)]]
Σ
P biast (wxx) (wrk)
[bias(w:)))
Olbias. (22)
p( Vare (wx) 2 Var(w:))
P
Σ
P (arr((us) Var(us) 2 OVar
(23)
Then, combining (22) and (23) and setting Ovvar + Obias. we obtain the claim. This is the procedure we adopt to construct biasy and Vary in Corollaries 1-4
Finally, one may use the high-dimensional multiplier bootstrap to construct biasy [129, 130J. Under mild regularity conditions, the distribution of the supremum in (15) or (16) converges to that of a Gaussian process [129, Theorem 2.1]. Furthermore, the supremum of this Gaussian process can be approximated via the multiplier bootstrap, which enables us to set to set U to the a-quantile of the supremum of the following bootstrapped process
D
(
sup wEW
[D(e)]
Σ
(1 - (1-w(5(e) )2 (0{"))
[D(A)]
Σ j=1
(1
in the unnormalized case, and
[2)(0]
120
sup wEW W ((w)
1 [D())]
Σ
(1 -
Σ (1 (1-w ((S(h)))
19
in the normalized case, where 9:s are i...d. standard Gaussian variables. Repeating the process over multiple bootstrap samples provides an empirical estimate of the quantile.
Similar to the first approach, the uniform bound can then be defined as:
biasy (w) = + [bias(w)) or biase (w) = U + +(w))bbas(w)))
As for the double robust estimator, to obtain Vary(w)) we only need to replace (w)( and (hh) (w)
with z(e) (we) and Z (wr) The bias upper bound can be similarly established.
In our implementation, conservative. Therefore, instead of enforcing a global bound, we adopt pointwise, non-uniform upper bounds for Vary(w) and biasy (w). computed individually at each w based on normal approximation.
find that the uniform upper bound over E W tends to be overly
B.5 Estimation Nuisance Function
Accurate estimation of the propensity score T(a ((+) is essential for both DR and IS. In implementation, we estimate this nuisance function via logistic regression. The outcome functions (0, S()),, (0) S(e)), and ((() S(e)) can flexibly estimated using a variety of regression models, including basis function expansions, random forests, and neural networks.
36

---

B.6 Estimation of (e(S)
To estimate the density ratio (u(S), we adopt moment- matching approach. Specifically, we seek function ((S) such that the following moment conditions are satisfied:
Elu(S!e) (s(()))]] ((()), for k I...., K,
where 4% (S) denotes the k-th test function.
Let ((S) (S), (S)..... x [S)] be the corresponding feature map. We approximate the density ratio w(S) by linear model of the form juS) ss ((S) T and estimate the coefficient vector IR by solving sample moment-man matching equation between the historical and experimental datasets.
Σ ED
(1S (O
Σ
DA
D
ED.
In practice, the feature function (()) can be set to polynomials, splines, or neural network features. Under mild negligible covariate shift, the density ratio can be simplified to 1.
B.7 Estimation of the weight function (S))
In our implementation, we parameterized w()) using a logistic model,
w S)
1+e
which ensures that w(S) E (0. for all S. Alternatively, neural network can be employed. Given a parameterized w, the pessimistic objective function derived in Section B.4 can optimized using gradient-based methods to learn the parameters.
C Proofs of the Theorems and Corollaries
In this section, we present the proofs of Theorem and Corollaries 1 Recall that Theorem applies any ATE estimator, while Corollaries 4 specific to the IS estimator.
C.1 Proof of Theorem
Proof of Theorem I. facilitate the analysis, we define the following events:
A n (wa) 2 [bias(w)]} B n Vary 2 Var(w)]. AnB
wEW
wEW
For given w, define MSE(w) the MSE of the ATE estimator ATE(w). Since the estimated weight @ itself is random, MSE(ū) is random variable. This MSE is well-defined due to the use of sample splitting which ensures that the ATE estimator is independent of w. The MSE of our proposed estimator is given by the expected value of MSE(().
MSE (STÉ(()()) EMSEE(
where the expectation on the right-hand-side (RHS) is taken with respect to the randomness of w
We decompose the difference in MSE into two parts:
MSF((AT(()) MSE (ATF(w)) [MSS(( (7)
MSE(w)) Tcl
M4 E (MSE((T) MSE(w)) 16
(24)
Mz
Bounding My: Under Assumption on event C. we have:
bias (w) 2
Var( biasy (w) Varo (ū).
37

---

Moreover, since v minimizes the pessimistic objective defined in (6), it follows that
biasy (6) + Varo - biasy (w) + Vary(w).
Thus, M1 can be bounded as follows:
M1 E (bias + Var(w) bias (w) Var(w)) lc]
E biasy (w) bias? (w) + Varu(w) Var(w)) 1c
(25)
2 E biasy (w) bias bias((w) (w)) + E. Vary(w) - Var(w))
Bounding M2: We next bound the term associated with the complement event C By the definition of the mean squared error (MSE), we have:
MSE(ATE ()) E (ATT(() ATE)
S (IATE())++ATT))
O(B²),
where the first inequality follows from the triangle inequality, and the second follows from Assump- tion 2. Similarly, we have (ISE(w)= S O(B2) ) for any W.
Using this and the union bound on probabilities:
P(C*) = PP4 U B9) s P(A)) PPBB) S 2a.
Hence, we bound term M2 as:
M2 E (M))((w) MSE(w)) |c.] S 2B² P(C") & O(aB²).
(26)
Plugging the bounds for terms M₁ in (25) and M2 in (26) into (24), the result follows.
C.2 Supporting Lemmas
Lemma (MSE decomposition). For a given weight function w, the MSE of the weighted estimator can be decomposed as:
MSE(w)= (ar(Z(<(w)) Var (ar(z(()ww)
(IE (1 w(s()))
(27)
[D(e]]
[Din]]
We interpret the three terms on the right-hand-sid of (27) as follows: (1) variance from the ex- perimental data, (2) variance from the historical data, and (3) the squared bias introduced by incorporating historical data.
Proof of Lemma 1. We decompose the MSE into the sum of variance and squared bias MSE(w) Var(w) bias)(w). We have already derived the closed-form expression of the variance term in (13). As for the bias, we note that:
[[(1 - w(S w(s()) (168)) (0())) E[(1 w(s(h) (0))))))))))) (1 w(({)))) (0, + (0)) |stos]
E
EE (1 w((())) Pa ((()))
(28)
Pn (S(h)) 11 (0, S(N) + (1)) [(Ais]
-E
(1 w(s(t))) (O. s(e)) - (M) (0, (0, )())
(w((()))) - 1) 11))-11)s)))) b(S(e))]
The second equality follows from Assumption 4 that c is independent of ((e) with E(c,")
(c)
(0) ((t)
and that (h) is independent of ((h) with (*) ((()] O. Combining (13) and (28) completes the proof.
Lemma (Variance from Experiment Data). For a given weight function w, the variance of (()
defined in (9) given by
p(0) (1, (1, + (0)
w wo((t)) (0, s(o)) + (o) (3)))
Var (w)) =E
+
x(1 ((e))
x(0 S(e))
(29)
-(B/r00(1, (()))) - E ((
2
38

---

Proof. Direct calculation leads to
Var (w))
Vare))((O(e))) + (0
2Cov( w(Ste) (
(Ole))).
We proceed to compute each term on the RHS, respectively.
1. Variance of (/ (O(e)): We use the law of total variance:
(ar(X)= V|Var(X X |S]] Var(E[X (S)-
In our case, X = 1b(+) (O(e)) According to Assumption 4, have:
(l | |(t) = (0) (1, (O())) s(0) (())))) + (0)
T(1 S(i)
follows from the total variance formula that
(arl)))(O(()) = E (r(0) (1)
+ (ar(-l))(,,(te)) (1. (30)
t[1 [(ee)
Variance un un((())) (e) (O(e)): Under Assumption direct calculation yields
E (( wub (e) (O) s(e) w/ ((( (0,
(( (0)² s(e} w ( (0, S(e)
(n) x x(0 S(a)
o
Here, we use af()) denote the standard deviation of e of 6
(e) and d (c)
denote the standard deviation
We next apply the total variance formula, which leads
(S(teije. (0, + (o(5))2)
x(0 S(e))
(31)
Var(w ((l))) ((e))) E 1B--BB((5s (w(s
3. Covariance term: Since w1 (Ole)) is nonzero only when A and w/ (O(e)) only when A = O, their supports are disjoint; hence B[y! (l (Ole)) (el) simplifies to
(f (O(e))) = 0. Therefore, their covariance
(Ol(), w!
Ellb( (l (O(e))] (Ofe)))) - (1, Elw (0) s())..
(32)
Combining (30)-(3)) completes the proof of the lemma.
Lemma 3 (Variance from Historical Data). For given weight function w, the variance of ((h)(w) defined in (9) given by
Var (w() = Var
s(,))) s())) + [n 22 (0)
Proof. By definition,
(1 (() = )(((()) (1,5 + em)
39

---

Under Assumption 4, the conditional mean of ((h) given S is zero. We obtain that
Var (L- w(((h)) 20th (s(h)) Var w (())))) (0.5
Var (1- (1())()
As for the second term, by the variance formula have
E (1 w(S
(
21 -
[B((L [a- w(S(h))) ((()))
E w
(since Ele (l and e ( dependent of 5th)
E (1- (
[c()))]]] E [1- (2 (a(h))2.
C.3 Preliminaries for the Proofs of Corollaries
We first derive the closed-form expression of the MSEs of EDO and HDB,
The MSE of EDO estimator: When w 1, the weighted ATE estimator becomes EDO, and its MSE simplifies to,
(r(e) (1, S())) + (
(0, s(e) + (a(-))²
MSE(EDO) [Din)]
+E
T(1 [(()
x S(r))
(33)
[B|r(e) (1, s(t))] - - (0, )):
The MSE HDB estimator: When 0, the weighted estimator becomes HDB, and its MSE can be expressed as,
(()))))) - (E) (1, (1, T(1 [(e)
(1, (1,
MSE(HDB)
E
(34)
(Var(u( (0, ((h))) + Elu(s()))) (0 + (B(()()n))²
+ |Dij
Next, we notice that although the MSE of the weighted estimator varies with the weight function w, certain components of the MSE remain constant with respect to w. To focus on the terms that vary with we define a new loss function L(w) as follows:
1 [Die)]
(((e)))
(r (0, s(e)) + (o)-))3
[(w)
E
π | S(e))
s(el)] Ei () s(c)) (EW(((()))01000001)) {b + (1,
(35)
[D(1] Var((1 - "," (0,
[D(A) B[(1 w(s((h)) (() ((( ( (1)2) + E ((())((())))(((0))
[ (Li + L2 Lz) + [Dhi]
(C1 + Ls) + Co.
where Li...., Eg denote the above six terms, respectively.
Remark. To simplify the proof of the corollaries, consider the case where the contextual variables g(t), ((h) are discrete. Our results can be extended to settings with continuous contextual variables.
40

---

C.4 Proof of Corollary 1
Optimality of EDO. We prove the optimality by contradiction. Suppose the minimal MSE achieved by weight function that does not converge to one. Then, there must exist non-empty set Si = (S: wSS) 2 A > 0}. Under this assumption, we derive a lower bound of L5 (35):
===
Σ (L w(S())))) p( (s
(mm (a
S(NESS
E (1- w(S (n-()))))) ES
(36)
(M((i)) (a(1))
where the third equality follows from the definition ((((h)= = Pe(S(e) PA(S0)))
As for other terms in the objective function ((w) in (35), under Assumption 5, it is easy to show that
L2 s (
((w(s(())r (0,,())) S 2r maz?
L3 ([ s()))
1 "aas
leading to L2 L3 2- 3r max. Since La and La are always non nonnnegaeve, we have
1
1
[(w) 2
(L1 L2 La) + [Din] La 2
(L1
1 3r max)
Ls
[D(e]]
[D(e]
[D(k)
w((tely2
(() (o))))e)
1
E [
Ls
<00 S(e)
[D(e]]
[DA)]
Consider the EDO estimator. By setting = its objective function L(1) becomes
(o)) (0. S())) + (
1
(=) (L. s(e) (t (0 (0, (E)r))))))())))
E
[
=(0 | Ste)
and it is smaller than
(0, sle)z2 (0,,tt)) (o))))
+ 2 (1, (1,, (0, + (E (t,33))))
E
[D(e)]
T(0) (())
Futhermore.it can be bounded by
(e) (0, s(e))2 (af (ol)g T(0 S(e))
+ 3r
D
max
where the first inequality follows from the triangle inequality, and the second holds by the condition (r())()))) < Tmax Assumption 5. Thus,
(r
L(w) - E(1) 2 G +
(37)
(D(e)]
where
G
1
Σ
(1 - (((h)))2 (p.(S(o)))
(A(2))2
D(N)
pM(((N))
g(M)
Σ (1 w(((el)2)) (S(e)) p(e) (() + (0)g3
1
Die)
T00 S(el)
g(n)ss,
(1 wo wwswss (((e)) ) (S(())
((o1)))_ (0. S(o))2 + (a) (e) pm (S(M))
D(h) pn (S(()))
T(0 ((r))
(S(r))
gtoles
41

---

where the first equality follows using similar arguments to (36), and the second equality follows from the definition of å = [/D(e)) Under Assumptions 3 and 5, it is easy to show that the condition a (o ) implies G1 0, which in ensures that [(w) [(1) for any w + 1.
Since for any weight function w, the objective [(w) differs from MSE(w) only by constant independent of w. This in turn.
MSE(w) MSE(1).
which leads to contraction with the property of the optimal weight w" = arg min MSE(w). Therefore.
w +1 11 for all S.
Oracle property. We next show that the difference in MSE between our proposed estimator and MSE(EDO) is smaller than MSE(EDO) itself.
Define the f--norm of (c) (for {0, 1 Jand ((h) as
1/v ||e(e) 1p E |c(-)pp p2
for p = 2,4,8
E [E(())) p)) 1/2
|| =
for p 2,4,8. 8.
Under Assumption 7, we show the proposed estimator achieves the oracle property. By (33), assumptions 3 and 5, MSE(EDO) has such a lower bound.
Var(z Ω [D(e)
(ol (0) + +00 00
MSE(EDO) = MSE(1)
(38)
[D(e)
According to Theorem 1, we can deduce that
MSE(@) MSE(1) I Eibia (1) bias (1)] E Varc Var(1)] O(a
=
V[ (1) Var(1)) + OOaB))
(39)
We next focus on orders of the above terms, respectively. By definition,we we have
E Varu Var(t)] = [Dee]]
E Varu (2() Var (1))
For analytical convenience, express the empirical variance estimator of Z (e) (1) as
Var (1)
Σ
(1) 10 where μ = E [e) (1)
D
(ED(o)
Then Var (1)( Var (1).
Var (1)) - Var (Z)) Through standard
According to (18),we need to calculate E moment expansion, we obtain:
Var (1) Var (2))
E (12 (1²= -)² -Vr(z(() (1)
'I) (40)
1
(Var ((z (1) - -1²))
O
12
(12) (1) (1) 2)² Var(z
(Var (1)
are of the same order, as their leading terms involve the eighth moments of (e) and c (e) For the remaining terms, under
and
42

---

Assumptions and 5, they can be uniformly bounded the same order of Mmax and Therefore, both expressions share the same asymptotic order. So the second term is the dominant one, and the overall expression is of order O (2 )1)=))))))
If choose U
log 1/4 DDe)) Var 1/2 (12 )
- )
U =
DDee)/1/4
we can get:
loog [ar (Z (1) - 11)2)
Var (1) Var (1) = =0
Dle)
Therefore, we obtain the following bound on the expected deviation between the upper-bound estimator and the true variance of the EDO estimator:
E Vary (1) - Var(1)] = (
5/4 ( 1/2 (1Z)) -w)²)
log [D(e]
(41)
We the fact that lower-order moments be controlled by higher-order moments. Specifically, we have
- ((2101)-)))
Var 1/2
= o E (1)
o (1213
s o ( (1
+ +0 (0(e)))()))
(42)
The first equality follows directly from the definition of the 4normm while the final inequality is consequence of the triangle inequality for norms and the definition in (9). According Assumptions 2, and 3, we obtain the following bound:
1/2 ((Z0))))) - -n) S
o 11 (ol + ( (0))))
O (Tmax + o + o0 (1))²
(43)
Furthermore, we have:
(D()) B²) (aBB)=00 (ID()) (
(44)
(e) We treat Tmax. To oj and as constants, and consider the asymptotic regime where [D(e)] + co.
Plugging (41), (43), (42), and (44) into (39), we have
MSE ( (SE S O log 1/1 |D(e)] (Fimxx + 000) -o( + D())) log -I
[D(4)]5/4
Comparing with the MSE EDO estimator given (38), one can deduce that
MSE( - MSE(1)
MSE(1)
as [D(e] - co. We conclude that the gap between our proposed method and the EDO baseline vanishes a faster rate than the MSE of the EDO estimator itself.
C.5 Proof of Corollary 2
Oracle property. We prove the corollary by contradiction. Suppose that optimal w(S) does not converge uniformly to zero. Then, there must exist non-empty set S2 := {S w(S) > A, A > 0}.
43

---

Under this assumption, we can derive lower bound for the objective L(w") (35) as follows:
(0)6 (0))2
[(w)=
1
7(0 ((6) Pr(()))) max))
Strtes
1
[E((1 w(((h))) ))² ( (
|DiA)]
w(s(())((o)))
1
*T0 s(+)) Pe(()t)) -O(r
[D())
[F[(1 - u((((h))) ( (al1)22)
|D(h]
Here, the first inequality holds since L1 and Ls in (35) are always non-negative, and the remaining terms can be upper bounded by Of according Assumption 5. On the other hand, we can derive an upper bound for [()), which corresponds to assigning zero weights to all historical data. In this case, the objective reduces to the variance from historical data and the squared bias(35):
[D(M (())=
1
(Var(u(s(h) (1( (0, (())) E))(s())) (o (1)) + (I((((((0)1
|D(N]] (B) (0, sT0 (B( (0, (1)))))))
Elu(St))))((0 (1))2 + (B1o(Stet)) ?
|DIN]
Dinl (0, s(h)))2 (( Elu(s()))3)] (o()1)2) + O((maan)
(0 (ma))) + E[n(S(N) (o(M))2) + (raaa))
|D(1)]
The reasoning behind this conclusion as follows. The squared bias term (E(((S()))) can be bounded by (r Similarly, the expectation term El((S(()) ((())))))))))))))))))))))))))))) ((()((()))) is bounded by (rmax)) which implies that its square is of order O(r as well.
Direct calculations lead to
(((( (e)) <00 S(e) (s(e))) - - mmx)
1
((w*) 2())
2
Die)
S
1
B[(1 (1 - w(((hl)) ()1(())))))
D
w( (0)))))))
1
a(0 S(e)) PS()) 0 max)
[D(e)
Sta)ES
(pe(S(e))3
Σ
(1 w(S [
(1)2
D
Pn((in))
ES2
(()))pp(S(e)
Σ
[D())|((0) Ste))
Strees
( (0))) (1- (1- (5(s))
(0) (a()))
2
w(S pm
(S(())))) (S(N))
where the first equality directly follows from the calculation of expectation, and the second equality
uses (u()) = p. (g(e))
(Sth) and = ( I/
According to the assumption
(h)
( > -1/2
v8 + v [Dei] Tmax
44

---

the order of the first term in ((w) - (() dominates the term of order (f leading to
[(w) (()) 0
for any w. Hence, the lower bound of ((w) strictly exceeds the upper bound of [()). This contradicts the optimality of w", since w² arg min, ((w). Therefore, the assumption must be false, and it follows that w* gap between the MSE of our proposed method and that of the HDB estimator becomes negligible relative to the MSE of the HDB estimator itself.
for all S. In the following, we discuss additional conditions under which the
Optimality of HDB. At first, We give the additional assumption: Assumption 8. The variance 01 (e)
satisfies:
of > max log ( (o + Tmux), log!/4 1/4 |D(M))/D())))))))))) 1/4
D(N) (a(h) + Fiaax)
[D(e)]
We begin by recalling the asymptotic order of the mean squared error (MSE) of the HDB estimator given in (34):
MSE(HDB) bias bias?(0) Var 2 1/ (O(+))
(10 (O())
Var
[D(e)
(D(N)
(45)
(o(=))2)
(oll))z2) [D(A]]
bias 2(0) + Ω
+
[D(e]
The difference between our proposed method and the MSE of the HDB estimator can be decomposed according to Theorem 1,
MSE ( MSE(HDB) < E
2- biasy (0) bias?
+ E
(HDBB Var(HDBB) + OaaB²).
For the variance difference component, have:
E
Vary (HDB) Var(HDB)
Var 4/ (0())) - Var (va(voo)) (1)))
[De]]
Vary 110 (h) (0(h)) Var Var(wb (O("))) (o()))))))))))
[D00]
Similar to the derivation of (41), according to the fourth-moment version of Markov's inequality(17) and assumption8, we can take:
log 1/4 [D()) |- V/2 1/2
(w110) - m21)²)
a1 = -O( |D(e)))
U1 =
1/4
U2 (D(M) 1/2 (O(1) 122)2
02 o (ID(M) log -DD!))))
D/4
where in E (o (Ote)) and μ22 = v0th (O(N) denote the corresponding population means.
(h)
Furthermore, the order of E Vary (HDB) Var Var(HDB) be derived as follows:
(D(e) ) (p())
1/2
(wle)())) 111)2)
E Vary (HDB) Var(HDB)
[D(e]/5/4
(46)
+0 1og log2/1|D(M)) |D(N)]
1/2 /2 (w) (Oth)) - (22
D/4
45

---

For the bias difference component, we directly compute the decomposition as: E biasy (0) bi)) (biast:()) (0) - bias (0)
+ 2 bias(0) E biasy (0) bias(0)
More precisely, the bias estimation error can be decomposed into two components,
[D(A]
ID
(O Eleb ( (O(h)))
Σ
Vo. (e) (O(e))
[D(A]
D
each which can be controlled via (40):
log 2/4 | Var
vo - (0(e))
a3 = ( "oog - |D((.))
Ubi =0
[D10)/1/4
1/4 |D(N) Var 1/2
(10 (1) (O()))
44 0 (ID(N))-2 log -1 |D(()))
U12 0
[D(N]1//4
Denote Us = Ubi + Ub2 it is easy to deduce that bias(0)
< Ub. Therefore, we obtain:
U. =0 log 1/4 (D(e) Var ( (O(s))
+0 log 1/4 |D(h) Var (006"
DDIjj1/44
Wext,we can get:
E biasy (0) bias 20)) = O(U3) + O(bias()) Ub) = O(U3)
=o log" 1/2 (Dee) Var v/ (O(o)))
+ 1og (P(()))) Var v/ (N) (OW)))
(47)
1/2
Diij
By summing all as's obtain
|D(E) 0
|1 log -1 (D(()))
(48)
Assumption holds, namely,
(c) max log (D(e) 1/44)) + (mmaa)e 1/2
D(e)) D(1)]
log 1/4 (D()) ()() /D/
(o + rmmaaee 1/2
(49)
This condition is equivalent to:
log 12 (De))
DDee)p//2 (o) + Frmax) P 1/2 1/2 |D(N)
(b)
+ -Tmax 126 Po-1
D
max
(D(N) 1/2
Furthermore, we know that:
Var (e)) (O())) < o (o) (c) + rmaa))
o (o(1) + Fmaax)
Var (w) (1) (O())) s <0
(50)
By combining the bounds in (45), (47), (48), and (50), under Assumption and comparing it with the order in (45), we have:
log 1/4
|D(e]
(rmax + o(o)g
2/4 |D(A)
(Fimax + 21)
MSF(@) MSE
<
+
[D(e)
|D(4)/6/4
B
B
+ log [De) [
log ( D(iii
+ (47)
46

---

which is much smaller than MSE(O) such that
MSE( MSE(0) MSE(O)
- 0
This completes the proof.
C.6
Proof of Corollary 3
Oracle property. Recall that in (27),
MSE(w) (w) Var(Z(*) (w)) [Dee)
+ Var (ar)Z((())
+ E (1- (((t)))((())))
[D(b]
Since Minimum Variance Estimator (MVE) is designed minimize total variance, it suffices to show that the squared bias (the third term) is negligible compared to the variance terms (the first two). In this case, the bias contributes asymptotically little the overall error.
Therefore, our analysis focuses on deriving a nontrivial lower bound for the variance component. For the variance contribution from the experimental data, we have the lower bound from Lemma 2, and Assumptions and 5, one can derive that
Var 2 (w)) D
(w))
1 [D
T S())) (o)))))
(S(e)))
E
This inequality holds by regrouping the terms involving ((t) (0, S(e)) and (()) ((,)), and applying basic inequality that ensures the non-negativity of the remaining components. Similarly, for the variance contribution from the historical data, we obtain the following lower bound from Lemma and Assumptions 3 and 5,
Var (w)) [D0)]
(1 w(Sth))) (((()))))(((( (o)))
D(N
Note that both S [(O] and E [L1 - w(Sth) ( ()) can not be simultancously zero for any nontrivial weight function w. Therefore, the total variance is lower bounded by:
Var(Z(=((w))
Var(Z()(w)) min DDA)
(a()))² (o D
[D(A]] f
for any wo
[D(e]
For the bias term, the condition
o(e)
(b)
(MS())) < min ViDiej
VIDDAJI
implies
(o(e))² (o(h))a
E (1 w(s(e)
min
for any w.
[D(e]]
(D())
In this scenario, minimizing MSE is equivalent to minimizing the total variance. Therefore, the optimal weight w² given by:
(Var(Z()(w)) (Die)
Var (1))
w = arg min
(51)
[D(1)
which corresponds exactly to the MVE formulation. This completes the proof of the corollary.
Optimality of MVE. The conditions required here are identical to those stated in Assumption 8. In this setting, the order of the MSE of the MVE estimator is given by
MSE(MVE) MSE(w") = Ω ( (()))
(e)}²
(a (1()))))
+ 22
(52)
(De))
D(h)]
47

---

We can derive the following MSE gap by plugging in w" in (51) in Theorem 1,
MSE MSE(w") S biasy (w") bias
"las?(w") (w") + E Vare((") Var(w") O(al3)).
For the bias component, we further obtain:
biasy (w") bias (has((w") O(U?) + +0 (bias(w") UUb
by using the 4-th moment Markov equality similarly to derivations in Sections C.4-C.5, where
( 1og |D(e)
(fine
Fmaxx + (1660))
log |D(M)
2 max + (a(11)2)
Ub =
+
(53)
and 01 O (o [D(e]] + |D(())-1 |D(N)))
Under the small shift assumption, the bias term satisfies [bias(w)) (w
< Ub. Therefore, we obtain:
E bias (w") bias (wo) 0 loog 1/2 [D(e]
max + (0)
[D(e)
(54)
log 1/2 ((0)
+ (a(1))²)
max
1//2
For the second term in MSE gap, similar to the derivation for (41), it is easy to deduce that
E Varu(w") Var(w")
(55)
(rmax + o00)) + (0))
llog 1/4 |De)) (Tmax + o(h))2
c. (Dib)]6/4 logg 1/4 |D(W))
0(U)
c. (/())//5/4
02 = O ((D(1)) log 1 | + |D(1)) log log- |D(N)). (o(-)l)"" Simple calculations show that the decay |D(c))
rate of E Varo(w") - VVr(w") (2*) is faster than that in MSE(w") in (52). The term
aB² = (an + a2) = 0 (B2ID(e)- -oog |D(e) (3) log (1(()))
(56)
which also vanishes faster than the leading MSE terms in (52).
Given the additional assumptions that
(o (e) [
log 1/2 1/ (D(e)) [D(e]]1/2 (o( + 2 log 1/2 [D())
(a( Tmmax) )
max
[D(A)
which is asymptotically smaller than Furthermore, by summing three gap terms in (54)-(56),
MSE( ( MSF(w") = =0 log 2/2 |D(e))
(rmax + 00 + o(1)]2
+
Die))//4
log 1/4 |D(A| (Fmax + (M)32
B²
B²
+
log (D(N)) + +(54)
(D(1)///
log (D(e) |
follows directly that
MSE ()) MSE(MVE) + 0
MSF(MVE)
as the sample size goes to infinity. This completes the proof.
48

---

C.7 Proof of Corollary 4
Oracle property. We prove the corollary by contradiction, considering both the moderate shift and large shift cases.
Mo) shift. We define the set S3 == (S 1 - w(S) 2 A > 0}. We consider the
experiment-related component for the objective function in (35),
According to the condition (r(e) (0, (())) I Tmax, w(5 ) [0, (00 5 2 for any S(e)
and triangle inequality, we obtain the upper bound
Maxx + +00
Cexp(1r) - [Dee] O
Similarly, the historical rellted component be bounded by
1
Lhis (w)
(L4 + Ls)
Dih
Var ((1 w(((h))) (((())) (0)) (0,
Dihjj
1
E[(1 w(s(h)))² ( (at11)²2
1
0
/maxx + (0(())32
Diinl
6
For the bias item:
La = B (1 ((SO)
21 Σ (e(())
[b((())))) 2( Σ A ((((e))))
gtolEs
SSOEEs
where the inequality follows from the fact that b(((()) is sign-consistent over S, either non- non-negative or non-positive for all 5(e) E S, and the coefficient of ((((c))) is strictly positive in S3.
As for [(1), it can be upper bounded as:
1- max + (o (o())22 e/D(E)
((1)
In the moderate case, for any ((e), we have [b(S(=))]
g +rmax
o(h) +hh++mmms
ve
ViDie)
VID(al
(0)))
2 max
+ (o(1))33
L6 Σ A-M(S()))
D
6
Dihjj
then the bias dominates the total MSE. Compare the order of L(w) with L(1), we have [ep((w) (w)( + L6 > C(1),
((w)
for any w * 1,which leads to a contradiction. Hence MSF(w) in w, the optimal weight function w" arg min,. MSE(w) satisfies
w + 1 for all S.
(2) Large shift. The proof follows the same reasoning as in the moderate shift case by using the condition
log |D(e] ( (1i (afe))z [D(e]
+ log D(A) [Dle)]
(r2 (naax (o(A))2))
E A A-(6(S()))) g(e)gS
In this case, where the EDO method remains optimal, the gap between the MSE of our proposed estimator and that of EDO becomes asymptotically negligible. This completes the proof.
Optimality of EDO. Conditions for small MSE difference Since the assumption and proof strategy the same as those used in C.4, we omit the detailed proof of this result for brevity.
49

---

D Limitation
The current paper considers settings without carryover effects where each action affects only its immediate reward and does not influence future outcomes. However, in many real-world applications, treatments are sequentially assigned over time, and such carryover effects can arise [104, 131]. This represents potential limitation of our work, as it does not account for such effects. In scenarios with carryover effects, the weight function for data integration may depend not only on contextual variables but also vary over time. Determining an optimal time-dependent weight function remains an open question, which we leave for future research.
50


Reviews:
[{"review_id": "7BebegR9lH", "reviewer": "Reviewer_KjHs", "summary": "The authors study the problem of ATE estimation by combining experimental data (in which a binary treatment is randomized) and observational data observed under the control treatment only. The contribution is a new estimation method that linearly combines experimental and observational estimators with data-dependent weights that are optimized for minimum MSE.", "questions": "- Could extensions to the sequential decision-making problem not be done similarly to Li et al. [3]?\n- The presentation, distribution shift scenarios, related work, experiments, and theory follow quite closely the work of Li et al. [3]. I understand that the authors define their weights as functions of covariates rather than constant values, is there a theoretical argument to favour one method over the other? Are there scenarios, e.g. low data regimes, in which we should expect Li et al. [3] to outperform?\n- How exhaustive are the distribution shift scenarios considered?\n- Are there any differences between your Theorem 1 and the Theorem 1 of Li et al. [3]?", "limitations": "- There is a disconnect between the presentation of the method and theory with the experimental results, the former developing an importance sampling estimator while the latter implementing a doubly robust estimator. This is a little bit unsatisfying.\n- I find that most of the presentation, distribution shift scenarios, related work, experiments, and theory to be very similar to that of Li et al. [3] which in my view detracts from the novelty and originality of the paper.\n- There is no conclusion.\n- The historical data is limited to observations of control outcomes.", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths.\n- The proposed method genuinely solves an important problem, it is shown to have oracle properties in many different settings and performs well experimentally.\n\nWeaknesses.\n- The conceptual and methodological advancement is quite limited, especially in relation to related work such as Li et al. [3] that already include many of the elements of the proposed recipe.", "was_revised": true, "final_justification": "I didn't have any particular concerns with the work and most of my questions were addressed during the rebuttal period. I still believe that much of the presentation and results follow closely from Li et al. [3] and it is not clear to me that the contribution significantly improves on previous work, especially since the idea of estimating weights using covariate information is quite well-established and seems like a small improvement upon previously proposed constant weights. That being said, I am not very familiar with the literature and thus my confidence level is low.", "rebuttal": "Thank you for your thoughtful and critical assessment. Many of your comments will help us produce a more readable and self-contained version of the paper. Below, we address each of your specific concerns in turn.\n* **Extension to Sequential Setting**\n  Our method naturally extends to sequential decision-making, differing significantly from Li et al. [3]. Given a time horizon $T$, we split the process into $T$ sub-sequences of lengths $1, 2, \\ldots, T$, where only the reward at the final step $t$ is retained, preserving the original reward structure.\n   - For each sub-sequence, we build a state-varying double robust estimator using both experimental and historical data. At each time $t$, we compute pessimistic estimates of bias, variance, and account for cross-time correlations, aiming to minimize the sum of pessimistic MSEs over all $T$ steps.\n  - To illustrate the key difference between our method and that of Li et al.[3], we present a simplified mathematical formulation.\nSuppose we have value function estimates at each of the $T$ time steps: $\\phi_t(S^{(e)})$ based on experimental data and $\\phi_t(S^{(h)})$ based on historical data.\n    \n    Our weighted estimators:  \n       - For experimental unit data: $\\sum_{t=1}^{T} w_t(S^{(e)}) \\cdot \\phi_t(S^{(e)})$  \n       - For historical  unit data: $\\sum_{t=1}^{T} \\left(1 - w_t(S^{(h)}) \\right) \\cdot \\phi_t(S^{(h)})$\n    \n    Li et al.'s fixed-weight estimators:  \n       - For experimental unit  data: $w \\sum_{t=1}^{T}  \\cdot \\phi_t(S^{(e)})$  \n       - For historical unit data: $(1-w) \\sum_{t=1}^{T}  \\cdot \\phi_t(S^{(h)})$\n\n  - Advantages over Li et al. [3]:\n    - **State-adaptivity**:  Sample weights vary with local state information, reducing MSE. \n\n    - **Time-adaptivity**:   Unlike Li et al. [3], who use fixed weights across time, our method allows time-varying weights. It reduces the MSE by leveraging information from specific moments in the dataset. Even if an estimator has a large overall bias across $T$ moments, certain moments may exhibit smaller biases.\n\n    - **Computational challenges**: Estimating time- and state-varying weights involves handling multi-dimensional state spaces and complex dependencies, while accounting for cross-time correlations adds further computational burden.\n  \n\n* **Differences with Li et al. [3]**. We appreciate the reviewer’s observation and clarify key distinctions highlighting our novel contributions beyond Li et al. [3].\n  \n  - **Covariate-Adaptive vs. Constant Weights**. A central distinction lies in how the integration weight is defined and optimized. \n    - Li et al. [3] propose a constant weight that linearly combines historical and experimental estimators by minimizing a pessimistic bound on the MSE. This approach performs well in cases of large or zero reward shift, but may not adapt well in intermediate regimes.\n    - Our method extends this framework by allowing the weight to be a function of the covariates (i.e., covariate-adaptive), treating it as a policy in a contextual bandit framework. This design enables local adaptation to distributional heterogeneity, leading to more refined and robust integration.\n\n  - **Theoretical Justification**. Our theoretical analysis (see Theorem 1 and Corollaries 1–4 in our paper) provides rigorous justification for the superiority of covariate-adaptive weighting in several challenging settings.\n\n    - In moderate posterior shift scenarios (which are typically the hardest to detect and address), our covariate-adaptive weighting achieves near-oracle performance, while the constant-weighted estimator may suffer from substantial bias if the fixed weight is not well-tuned.\n\n    - The proposed estimator is adaptive: it recovers the optimal performance of the scenario-specific best estimator without knowing the shift regime a priori. This is not achievable by constant-weighted methods, including Li et al. [3].\n\n    - We show that our estimator retains robustness even when the historical or experimental data are heavy-tailed, or when the posterior shift is moderate and non-identifiable from the data.\n\n  - **Low-data regimes**. We agree that in very low-data regimes, a simple constant-weight approach (e.g., Li et al. [3]) may exhibit lower variance due to reduced model complexity. However, we highlight that\n\n    - Our estimator includes constant weighting as a special case, meaning it can recover the behavior of Li et al. [3] when that is optimal.\n\n    - Moreover, the pessimistic principle we adopt prevents overfitting even in small samples by minimizing an upper bound of the estimated MSE, providing a safeguard in low-data settings.\n\n    - In our experiments, our estimator consistently outperforms or matches the best baseline across all data regimes, including low-sample scenarios.\n\n\n\n\n* **Exhaustiveness of distribution shift scenarios**. This is an excellent comment. As summarized in Section 3.1 of the main text, all potential distributional shifts can be broadly categorized into three types: (i) covariate shift, (ii) policy shift, and (iii) posterior shift. The first two types can be easily handled using importance sampling. In contrast, posterior shift inevitably introduces bias when incorporating historical data -- a challenge we specifically focus on addressing in this paper.\n\n   - Posterior shift primarily concerns changes in the conditional distribution of the reward given a covariate-policy pair. Since this distribution is quite general, and our main interest lies in minimizing the mean squared error (MSE) of the estimator, we restrict attention to its first two moments: the conditional mean and variance of the reward. The conditional mean is captured by the mean reward difference function $b(s)$, while the conditional variances of the historical and experimental rewards are denoted by $(\\sigma^{(e)})^2$ and $(\\sigma^{(h)})^2$, respectively.\n\n   - By focusing on these two key quantities, all relevant scenarios can be systematically categorized as follows, based on whether one dominates the other or whether they are of comparable magnitude:\n\n     (i) **$\\sigma^{(h)}$ deminates both $|b(s)|$ and $\\sigma^{(e)})$**: This corresponds to the heavy-tailed historical reward scenario (Scenario (i)) considered in the paper.\n\n     (ii) **$\\sigma^{(e)}$ deminates both $|b(s)|$ and $\\sigma^{(h)})$**: This corresponds to the heavy-tailed experimental reward scenario (Scenario (ii)).\n\n     (iii) **$\\sigma^{(e)}$ and $\\sigma^{(h)})$ are comparable and both dominate $|b(s)|$**: This corresponds to the small mean shift scenario (Scenario (iii)).\n\n     (iv) **All three quantities are comparable**: This corresponds to the moderate mean shift scenario (Scenario (iv)).\n\n     (v) **$|b(s)|$ dominates both $\\sigma^{(e)}$ and $\\sigma^{(h)})$**: This corresponds to the large mean shift scenario (Scenario (v)).\n\n   - Accordingly, all the aforementioned scenarios -- or closely related settings -- are systematically studied in this paper. In summary, our analysis considers all three types of distributional shifts: shifts in the covariate distribution, the policy distribution, and the reward distribution. For the reward distribution, we focus specifically on changes in its first two moments (mean and variance). While this does not exhaustively cover all possible posterior shift scenarios, we believe it captures the most practically relevant scenarios for the purpose of MSE minimization.\n* **Differences from Theorem 1 in Li et al.[3].**  \nLi et al. [3] analyze constant weights without pessimism. Their result relies on $\\hat{w}$, a data-dependent random variable that can induce high variance. In contrast, our conclusion considers a pessimistic, state-dependent constant weight $w$, which avoids such randomness and reduces variance. Moreover, we explicitly address heavy-tailed settings, which are not covered in Li et al. [3].\n\n* **Disconnect between the method and theory with the experimental results**. We emphasize that the theoretical foundations of importance sampling and doubly robust estimation are closely linked in assumptions and statistical properties. To better align theory with experiments, we have expanded the discussion of the doubly robust framework in our revision, clarifying its relation to our method. This strengthens the connection between our theoretical claims and empirical performance. For further details, please see our response to **Reviewer TPvk** due to character limits.\n\n* **Extension to Historical Treatment & Control Data**\nOur method is initially developed for scenarios where only control outcomes are available in the historical dataset—a common case in A/B testing. It can be naturally extended to more general settings where **both treatment and control outcomes** are observed.\nIn this case, we estimate individual treatment effects for historical samples and construct a corresponding estimator. Each experimental unit is assigned a personalized weight $w(S^{(e)})$, while each historical unit receives a complementary weight $1 - w(S^{(h)})$. This enables flexible integration of both datasets in the estimation process and allows us to compute a pessimistic MSE accordingly.\n\n* **Conclusion.** We will include the following conclusion in the final version. \n  - We propose a pessimistic data integration method for causal effect estimation that leverages both experimental and historical datasets. Our method formulates sample-wise weight assignment as a learnable policy. This allows us to apply pessimistic policy optimization to minimize the estimator’s mean squared error .The resulting state-adaptive weights adjust flexibly to individual covariates, improving efficiency and balancing bias-variance tradeoffs. We evaluate the method theoretically and empirically across five scenarios, showing that it achieves near-optimal performance and superior robustness under distribution shift or limited experimental data.\n\nThank you again for your thoughtful review. We look forward to further discussion."}, {"review_id": "K8DRKS6zis", "reviewer": "Reviewer_TPvk", "summary": "This paper investigates the problem of Average Treatment Effect (ATE) estimation in an off-policy evaluation setting by leveraging both historical (observational) and experimental data. The authors propose an estimator that combines estimates derived from each data source via a learned weight parameter. A key aspect of their method is the use of a contextual bandit algorithm to adaptively optimize this mixing weight with the goal of minimizing the Mean Squared Error (MSE) of the final ATE estimate. The performance of the proposed estimator is evaluated on synthetic and semi-synthetic datasets, where it is shown to achieve a low MSE empirically.", "questions": "- Could you elaborate on the use of a Doubly Robust (DR) estimator in the experiments when the theoretical analysis is provided for an Importance Sampling (IS) estimator? Do the theoretical guarantees extend to the DR setting? \n- I understand that some datasets are proprietary. However, have you considered validating your findings on a publicly available benchmark dataset for policy evaluation?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- Simplicity and Intuition: The proposed technique is simple and intuitive. The concept of a weighted mixture of estimators is straightforward to understand and appears relatively simple to implement.\n- Novel Optimization Method: The application of a contextual bandit to learn the optimal mixing weight is a creative and interesting contribution.\n- Strong Empirical Results: The experiments, though on synthetic data, demonstrate that the proposed estimator is effective. The reported low MSE suggests that the method holds significant promise.\n\nWeaknesses:\n- Incremental noveltry: The general idea of combining estimators from observational and experimental data is well-established in the literature. While the use of a bandit is novel, the paper could benefit from a clearer positioning of its contribution relative to this existing body of work.\n- Reproducibility concerns: The semi-synthetic experiments are based on a closed-source dataset, which hinders the reproducibility of those results.\n- Gap between theory and practice: There appears to be a disconnect between the theoretical analysis and the empirical implementation. The theory is developed for Importance Sampling (IS) based estimators, while the experiments are conducted using a Doubly Robust (DR) estimator. \n- Experimental setting: The evaluation relies only on synthetic and semi-synthetic data. The absence of experiments on a real-world, public benchmark makes it challenging to assess how the method would perform in practical scenarios.", "was_revised": true, "final_justification": "The rebuttal addressed most of my concerns\n\nI appreciate the authors adding new experiments, however, I believe the claim of \"consistently strong performance\" is overstated. The new results suggest the method's primary strength lies in the low-to-moderate posterior shift regime, as its advantage diminishes for larger shifts and it is not uniformly superior to all baselines.", "rebuttal": "Thank you for your thoughtful and critical assessment. Many of your comments will help us produce a more readable and self-contained version of the paper. Below, we address each of your specific concerns in turn.\n* **Contribution relative to existing work on data combination.**  \nWe thank the reviewer for raising the connection to prior work on combining observational and experimental data—a well-studied topic.\n\n  - Existing methods largely fall into two categories:  \n  (1) those addressing **unobserved confounding** in observational data using experimental data (e.g., Kallus et al., 2018; Athey et al., 2020; Bartolomeis et al., 2024; Imbens et al., 2025), and  \n  (2) those improving **efficiency under distributional shifts** by combining datasets (e.g., Degtiar & Rose, 2023; Shi et al., 2023; Colnet et al., 2024).  \n   - Our method belongs to the latter and focuses on mitigating distribution shift to enhance estimation accuracy.Within this group, existing methods can be further divided into:\n\n     - **Category 1**: Assume **no posterior shift**, and only address covariate and policy shifts (e.g., Li & Luedtke, 2023; Yang et al., 2023; Gao et al., 2025).  \n      - **Category 2**: Handle **posterior, covariate, and policy shifts**, often via $\\ell_1$-penalized selection (e.g., Xiong et al., 2023; Sheng et al., 2024; Wu et al., 2025; Cheng & Cai, 2021; Han et al., 2023, 2025).\n\n  - Our approach differs from both in three key aspects:\n\n    - **Posterior shift**: Unlike Category 1, we explicitly model and account for it.\n  \n    - **Robustness**: Unlike Category 2, which may rely on large posterior shifts, our method is robust to **moderate shifts** and **heavy-tailed data**.\n  \n    - **Estimation principle**: We go beyond naive pooling or penalized influence functions by directly **minimizing estimated MSE** via **covariate-adaptive, pessimistic weighting**.\n    - While the idea of data combination is well-established, our contribution lies in its **robust handling of posterior shift**, **adaptive sample-wise weighting**, and a **bandit-inspired framework** suited to real-world heterogeneous settings.\n   \n* **Gap Between Theory and Practice**\n\n  First, we emphasize that **Theorem 1 is generic**. As noted (Page 6, Line 234), it applies not only to IS but also to the DR estimator used in our experiments. Details for DR construction are provided in Appendix B.3 to save space. Thus, our theory aligns with our practical implementation.\n  Although the corollaries focus on IS for clarity, our theoretical analysis shows that **all results can be extended to DR**. Below, we outline the necessary assumptions and provide sketches of the main results.\n\n   - **Additional Assumptions for DR**\n\n       - **Assumption 6 (Doubly Robust Specification)**: Either the reward function or the density ratio is correctly specified.\n        - **Assumption 7 (Boundedness of Nuisance Functions)**:\n             - (i) Reward functions are bounded by $r_{\\max}$.\n             - (ii) Density ratios are bounded between $\\epsilon$ and $\\epsilon^{-1}$.\n\n    Assumption 6 ensures consistency of DR estimators and can be replaced by requiring both nuisance functions to converge to oracle targets. Under Assumptions 3 and 5, the oracle functions satisfy Assumption 7, making it reasonable to impose boundedness on their estimates.\n\n    - **Corollaries for DR**\n\n       Let $\\delta$ represent the posterior shift magnitude and $\\sigma^{(e)}, \\sigma^{(h)}$ be reward variances in experimental and historical data.\n\n     - **Corollary 1 (Heavy-tailed Historical Rewards)**  \n  DR-EDO remains optimal if:   $ \\sigma^{(h)} \\gg \\epsilon^{-2} \\sqrt{\\delta} [\\sigma^{(e)} + r_{\\max}] $  and the MSE difference between DR and EDO is negligible.\n\n    -  **Corollary 2 (Heavy-tailed Experimental Rewards)**  \n  DR-HDB remains optimal if:  $\\sigma^{(e)} \\gg \\epsilon^{-2} \\sqrt{\\delta} [\\sigma^{(h)} \\delta^{-1/2} + \\sqrt{n^{(e)}} r_{\\max}] $\n\n    - **Corollary 3 (Small Posterior Shift)**  DR-MVE remains optimal under the same condition as the main text.\n\n    - **Corollary 4 (Moderate/Large Posterior Shift)**  \n  DR-EDO remains optimal if:  $ |b(s)| \\gg \\epsilon^{-2} \\left[ |D^{(e)}|^{-1/2} (\\sigma^{(e)} + r_{\\max}) + |D^{(h)}|^{-1/2} (\\sigma^{(h)} + r_{\\max}) \\right] $\n\n     Compared to IS-based results, the main difference is the dependence on $\\epsilon^{-2}$ due to potential density ratio misspecification.\n  \n    - **Sketch of Proof**\n\n       The proof follows that of IS but differs in bias and variance control:\n\n   - **Bias**: In IS, correct density ratios are needed. In DR, the bias still equals $\\mathbb{E}[b(S^{(e)})]$ under Assumption 6 due to its doubly robust nature (see Chernozhukov et al., 2018).\n\n   - **Variance**: IS is a special case of DR with zero reward function. For general DR, variance bounds still hold under Assumption 7. However, misspecified density ratios yield heavier dependence on $\\epsilon^{-1}$, explaining the stronger conditions in DR corollaries.\n\n* **Additional Experiments**.\nWe add  experiments on a public real-world dataset.We evaluate our method on the ACTG175 dataset with 2,139 HIV-positive individuals randomized to four treatments. We focus on comparing ZDV+ddI ($n=522$) and ZDV+zal ($n=524$), using rescaled CD4 count as the outcome and three covariates: age ($S_1$, rescaled by 10), homosexual activity ($S_2$), and hemophilia ($S_3$). The outcome model is:\n$$\nR = f(S)=\\beta_0 + \\beta_1 S_1^2 + \\beta_2 S_1 + \\beta_3 S_2 + \\beta_4 S_3 + \\gamma \\cdot  A,\n$$ Using fitted model parameters, we generate real-data-based synthetic outcomes:\n$$\nR_e = f(S') +  0.8 \\delta \\epsilon_e, \\quad\nR_h = f(S') + \\mu_d + 0.05\\mu_d S_1' + 0.8 \\delta \\epsilon_h,\n$$\nwith $\\epsilon_e \\sim \\mathcal{N}(0,1)$ and $\\epsilon_h \\sim \\mathcal{N}(0,1)$ or $t_6$. In experiments, actions are randomized ($A \\sim \\text{Bernoulli}(0.5)$) in experimental data, while fixed at $A=0$ in historical data. Both datasets have 48 samples. We vary $\\boldsymbol{\\mu}_d\\in [0,5]$ (25 points) and report the $ \\textbf{empirical average MSE}$ over 100 simulations. Results $\\textbf{stabilize }$for large $\\boldsymbol{\\mu}_d\\$ and are omitted $\\textbf{for brevity}$.\n\n   -  **EDO  0.3277 N-N and N-t**\n      | CWE (N-N) | NonPessi | Proposed | LASSO  | CWE (N-t) | NonPessi | Proposed | LASSO  |\n      |:---------:|:--------:|:--------:|:------:|:---------:|:--------:|:--------:|:------:|\n      | 0.3154    | **0.2868** | 0.2912 | 0.3162 | 0.3189    | 0.3014   | **0.3013** | 0.3232 |\n      | 0.3153    | **0.2919** | 0.2927 | 0.3205 | 0.3186    | 0.3013   | **0.3007** | 0.3223 |\n      | 0.3172    | 0.3032   | **0.2997** | 0.3262 | 0.3198    | 0.3056   | **0.3041** | 0.3274 |\n      | 0.3203    | 0.3152   | **0.3088** | 0.3306 | 0.3220    | 0.3147   | **0.3098** | 0.3355 |\n      | 0.3231    | 0.3225   | **0.3164** | 0.3341 | 0.3234    | 0.3196   | **0.3140** | 0.3304 |\n      | 0.3245    | 0.3259   | **0.3210** | 0.3290 | 0.3241    | 0.3213   | **0.3165** | 0.3278 |\n      | 0.3258    | 0.3285   | **0.3246** | 0.3302 | 0.3245    | 0.3231   | **0.3178** | 0.3264 |\n      | 0.3263    | 0.3293   | **0.3252** | 0.3296 | 0.3251    | 0.3236   | **0.3199** | 0.3277 |\n      | 0.3264    | 0.3275   | **0.3248** | 0.3277 | 0.3256    | 0.3243   | **0.3212** | 0.3290 |\n      | 0.3264    | 0.3266   | **0.3241** | 0.3277 | 0.3257    | 0.3241   | **0.3216** | 0.3277 |\n\n\n   We further consider a $t_9$ target distribution with sources either standard normal or $t_9$, increasing $\\delta$ to 2.5 for stronger signals.\n\n   - **EDO 1.42 t-N and t-t**\n      | CWE(t-N) | NonPessi | Proposed | LASSO | CWE(t-t | NonPessi | Proposed | LASSO |\n      |:---:|:--------:|:--------:|:-----:|:---:|:--------:|:--------:|:-----:|\n      | 1.38 | 1.29 | **1.28** | 1.41 | 1.39 | 1.32 | **1.30** | 1.41 |\n      | 1.38 | 1.29 | **1.28** | 1.41 | 1.39 | 1.33 | **1.30** | 1.42 |\n      | 1.38 | 1.30 | **1.29** | 1.41 | 1.39 | 1.34 | **1.31** | 1.44 |\n      | 1.39 | 1.31 | **1.30** | 1.41 | 1.39 | 1.33 | **1.32** | 1.43 |\n      | 1.39 | 1.33 | **1.32** | 1.41 | 1.40 | 1.34 | **1.33** | 1.42 |\n      | 1.40 | 1.36 | **1.34** | 1.44 | 1.40 | 1.36 | **1.34** | 1.43 |\n      | 1.41 | 1.38 | **1.37** | 1.45 | 1.41 | 1.38 | **1.36** | 1.43 |\n      | 1.41 | 1.40 | **1.38** | 1.43 | 1.41 | 1.40 | **1.37** | 1.43 |\n      | 1.42 | 1.41 | **1.39** | 1.44 | 1.41 | 1.41 | **1.38** | 1.42 |\n      | 1.42 | 1.42 | **1.40** | 1.44 | 1.42 | 1.42 | **1.40** | 1.42 |\n      | 1.42 | 1.44 | **1.41** | 1.44 | 1.42 | 1.43 | **1.40** | 1.43 |\n      | 1.43 | 1.44 | **1.42** | 1.43 | 1.42 | 1.43 | **1.41** | 1.44 |\nAcross all settings and $\\mu_d$ values, our method consistently achieves strong performance.\n\nThank you again for your thoughtful review. We look forward to further discussion.\n* **References**.\n- Athey et al. (2020). Combining experimental and observational data. arXiv.\n- Cheng & Cai (2021). Adaptive combination of RCT and obs. data. arXiv.\n- Colnet et al. (2024). Causal inference combining trials and observational studies. Stat. Sci.\n- De Bartolomeis et al. (2024). Lower bound for confounding strength. AISTATS.\n- Degtiar & Rose (2023). Generalizability and transportability. Annu. Rev. Stat. Appl.\n- Gao et al. (2025). Data-adaptive borrowing in RCTs. Biometrika.\n- Han et al. (2025). Federated causal estimation (FACE). JASA.\n- Han et al. (2023). Multiply robust federated estimation. NeurIPS.\n- Imbens et al. (2025). Long-term causal inference under confounding. JRSSB.\n- Kallus et al. (2018). Removing hidden confounding. NeurIPS.\n- Li & Luedtke (2023). Efficient estimation under data fusion. Biometrika.\n- Sheng et al. (2024). Sequential data integration under shift. Technometrics.\n- Shi et al. (2023). Data integration in causal inference. Wiley Interdiscip. Rev. Comput. Stat.\n- Wu et al. (2025). Estimating ATE via data combination. JASA.\n- Xiong et al. (2023). Federated causal inference in heterogeneous data. Stat. Med.\n- Yang et al. (2023). Elastic integrative analysis. JRSSB."}, {"review_id": "YohsZ4NNWF", "reviewer": "Reviewer_XVXr", "summary": "This paper studies the optimal integration of experimental data (with both treated and control units) and historical data (only control units, subject to covariate, policy, and posterior shift compared with the experimental data). The authors propose to use a weighted combination of the dataset, where the weights are conceptualized as a \"policy\" that determines the importance of each data point. Then, the policy is learned by minimizing an estimated combination of MSE. The resulting estimator is analyzed in several scenarios, and experiments show the superior performance of the resulting estimator.", "questions": "1. It's a bit confusing to use $\\hat{\\mathbb{E}}_n$ for both average in experimental data and average in historical data. It might be helpful to use different symbols for the two sample averages.\n\n2. Theorem 1 appears incorrect to me. The issue is that on Assumption 1, one doesn't have the claimed equation $$bias(\\hat{w}) + Var(\\hat{w}) \\leq \\hat{bia}(\\hat{w}) + \\hat{Var}(\\hat{w})$$ in the proof, since Assumption 1 only concerns a single, fixed $w$, whereas $\\hat{w}$ is data-dependent. Indeed, in the literature of pessimism one typically needs uniform concentration for the estimated upper bound (i.e., uniformly valid for all $w$ with high probability). Also in the proof of Theorem 1, it's unclear what $MSE(\\hat{w})$ means -- is the MSE marginalized over everything? \n\n3. Apart from MSE, inference is also important for A/B testing applications. Could you comment on inference with the proposed estimators?", "limitations": "Yes.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n\n1. The problem is important and the solutions is interesting\n2. The experiments show superior performance \n3. The paper is clearly written and easy to follow\n\nWeakness:\n\n1. The theory is incorrect (see the Questions section), which is my biggest concern.", "was_revised": true, "final_justification": "authors have provided responses and revisions to my questions.", "rebuttal": "Thank you for your thoughtful and critical assessment. Many of your comments will help us produce a more readable and self-contained version of the paper. Below, we address each of your specific concerns in turn.\n\n* **Notation for $\\mathbb{E}\\_{n}$** . We use $\\\\mathbb{E}\\_{n}$ to denote the empirical average computed over a dataset. To distinguish between different data sources, we let $\\\\mathbb{E}\\_{n_e}$ and $\\mathbb{E}\\_{n_h}$ represent the empirical averages over the experimental and historical data, respectively.\n\n* **Theory concerns**. \nWe sincerely thank the reviewer for the insightful comments regarding the theoretical analysis. Upon revisiting the proof, we agree with the observation and have strengthened Assumption 1 to ensure uniform concentration guarantees. We also clarified the definition of $\\text{MSE}(\\hat{w})$ by explicitly treating it as a conditional expectation given $\\hat{w}$. Based on these revisions, we provide a more rigorous proof of Theorem 1.\n\n  - **Revised Assumption1**. Define the following events:\n    $$ \n     \\\\begin{aligned}\n     A &:= \\\\left\\\\{ \\\\widehat{\\\\text{bias}}_U(w) \\\\geq |\\\\text{bias}(w)| \\\\text{ for all } w \\\\in \\\\mathcal{W} \\\\right\\\\}, \\\\\\\\\n     B &:= \\\\left\\\\{ \\\\widehat{\\\\text{Var}}_U(w) \\\\geq \\\\text{Var}(w) \\\\text{ for all } w \\\\in \\\\mathcal{W} \\\\right\\\\}, \\\\\\\\\n     \\\\end{aligned}\n  $$\n     We assume, for some $\\alpha \\in (0,1)$, that $\\mathbb{P}(A) \\geq1 - \\alpha$, $\\mathbb{P}(B) \\geq 1 - \\alpha$.\n\n  - **Clarify the definition of $\\\\text{MSE}(\\\\hat{w}) .$** We define $\\text{MSE}(\\hat{w})$ as a conditional expectation—once $\\hat{w}$ is fixed, the expectation is taken over the remaining randomness. Thus, $\\text{MSE}(\\hat{w})$ is itself a random variable. The overall estimator's MSE is: $ \\text{MSE}(\\widehat{\\text{ATE}}(\\hat{w})) = \\mathbb{E}\\left[\\text{MSE}(\\hat{w})\\right], $where the expectation is taken over the randomness of  $ \\hat{w} $. \n\n   - **Proof of  Theorem1**. We decompose the MSE difference into two parts based on whether the event $   C := A \\\\cap B $ holds:\n      $$ \\text{MSE}(\\widehat{\\text{ATE}}(\\hat{w})) - \\text{MSE}(\\widehat{\\text{ATE}}(w)) = M_1 + M_2, $$\n          \n      where  \n      $ M_1 = \\mathbb{E} \\left[(\\text{MSE}(\\hat{w}) - \\text{MSE}(w)) \\cdot \\mathbf{1}_{C } \\right], $ \n\n      and \n     $ M_2 = \\mathbb{E}\\left[(\\text{MSE}(\\hat{w}) - \\text{MSE}(w)) \\cdot \\mathbf{1}_{C ^c} \\right]. $ \n\n     For $M_1$, since $\\hat{w}$ minimizes the pessimistic upper bound within event $C$, we have:\n     $$ \\mathrm{bias}^2(\\hat{w}) + \\mathrm{Var}(\\hat{w})  \\leq  \\widehat{\\mathrm{bias}}_U^2(\\hat{w}) + \\widehat{\\mathrm{Var}}_U(\\hat{w})  \\leq  \\widehat{\\mathrm{bias}}_U^2(w) + \\widehat{\\mathrm{Var}}_U(w). $$\n\n     Thus,\n     $$ M_1 \\leq \\mathbb{E} \\left[ \\left( \\widehat{\\text{bias}}_U^2(w) - \\text{bias}^2(w) + \\widehat{\\text{Var}}_U(w) - \\text{Var}(w) \\right) \\cdot \\mathbf{1}_C \\right] \\leq \\mathbb{E} \\left[ \\widehat{\\text{bias}}_U^2(w) - \\text{bias}^2(w) \\right] + \\mathbb{E} \\left[ \\widehat{\\text{Var}}_U(w) - \\text{Var}(w) \\right]. $$ \n\n     For $M_2$, we apply the triangle inequality and Assumption 2 to get\n     $$ \\text{MSE}(\\hat{w}) = \\mathbb{E}[(\\widehat{\\text{ATE}}(\\hat{w}) - \\text{ATE})^2] \\leq \\mathbb{E}[(|\\widehat{\\text{ATE}}(\\hat{w})| + |\\text{ATE}|)^2] \\leq \\mathcal{O}(B^2). $$\n\n     Similarly, we have $\\text{MSE}(w) \\leq \\mathcal{O}(B^2)$ for any $w \\in \\mathcal{W}$.\n\n     By the union bound on probabilities\n     $\\mathbb{P}(C^{c}) = \\mathbb{P}(A^{c} \\cup B^{c} ) \\leq \\mathbb{P}(A^{c}) + \\mathbb{P}(B^{c}) \\leq 2\\alpha.$ Hence, we bound the term $M_2$ as:  \n     $$ M_2 = \\mathbb{E}\\left[(\\text{MSE}(\\hat{w}) - \\text{MSE}(w)) \\cdot \\mathbf{1}_{C^c} \\right] \\leq 2B^2 \\cdot \\mathbb{P}(C^c) \\leq \\mathcal{O}(\\alpha B^2). $$\n\n     Combining both parts yields the final bound\n     $$ \\text{MSE}(\\widehat{\\text{ATE}}(\\hat{w})) - \\text{MSE}(\\widehat{\\text{ATE}}(w)) \\leq \\mathbb{E} \\left[ \\widehat{\\text{bias}}_U^2(w) - \\text{bias}^2(w) \\right] + \\mathbb{E} \\left[ \\widehat{\\text{Var}}_U(w) - \\text{Var}(w) \\right] + \\mathcal{O}(\\alpha B^2). $$\n\n* **Statistical inference**. This is an excellent comment, and we fully agree on the importance of conducting valid inference in A/B testing, as A/B testing is essentially a statistical inference problem. Fortunately, when employing DR -- used in our numerical studies -- for ATE estimation, valid p-values can be readily obtained. We have conducted both theoretical analysis and empirical evaluations during the rebuttal to demonstrate this.\n\n   - Specifically, when combined with sample-splitting -- using one half of the data to estimate the weight function $\\widehat{w}$ and nuisance functions (the reward and density ratio), and the other half to construct the ATE estimator -- following Chernozhukov et al. (2018), one can show that the resulting ATE estimator is asymptotically normal under suitable regularity conditions (i.e., the reward and density ratio functions converge at certain rates).\n\n  - As a result, standard $z$-tests with normal approximation can be used to obtain valid p-values. The only issue that may invalidate the p-value is the bias introduced through the incorporation of historical data. However, our proposed data integration procedure is explicitly designed to minimize MSE and control this bias. We elaborate on this below.\n\n  - For a given $\\widehat{w}$, the bias of the ATE estimator is given by $E(1-\\widehat{w}(S^{(e)}))b(S^{(e)})$ where the expectation is taken over $S^{(e)}$. In the first three scenarios considered in our paper, the variance of the estimator dominates $b^2(s)$, so the bias $E(1-\\widehat{w}(S^{(e)}))b(S^{(e)})$ is also negligible compared to the variance. This ensures the validity of p-values in these three scenarios. In the last two scenarios, $b^2(s)$ might dominate the variance. However, since our method is designed to minimize the MSE, the estimated weight function $\\widehat{w}$ will converge to 1 to ensure the bias $E(1-\\widehat{w}(S^{(e)}))b(S^{(e)})$ remains small. Our numerical studies presented below confirm this behavior: the bias is negligible in magnitude, and the resulting p-values remain valid across all scenarios.\n\n* **Additional simulation studies**. For completeness, we conducted additional simulations to evaluate the finite-sample performance of hypothesis testing using the proposed estimator. Specifically, we consider the following hypothesis testing problem:\n$H_0: \\text{ATE} = 0 \\quad \\text{vs.} \\quad H_1: \\text{ATE} \\ne 0$.\n  \n  - To assess the performance of the EDO, Pessi, and Proposed methods, we first set the true ATE to 0 and perform 1,000 Monte Carlo simulations to estimate the Type I error rate of each method at the 5\\% significance level. Next, to evaluate the empirical power, we gradually increase the signal strength by setting the ATE to values in the set ${0.5,\\ 1.0,\\ 1.5,\\ 2.0,\\ 2.5,\\ 3.0}$, and again conduct 1,000 simulations for each setting to compute the probability of correctly rejecting the null hypothesis under the alternative. We also examine three levels of bias: small, moderate, and large.\n  \n  - The tables below report the empirical rejection rates. Under the null hypothesis, all three methods maintain Type I error rates close to the nominal 5% level, albeit slightly conservative. Under the alternative hypothesis, the proposed method consistently achieves higher power than the alternatives, with power approaching 1 as the signal strength increases.\n\n\n   | ATE | Metric | EDO (Small) | CWE (Small) | Proposed (Small) | EDO (Moderate) | CWE (Moderate) | Proposed (Moderate) | EDO (Large) | CWE (Large) | Proposed (Large) |\n   |:---:|:------:|:-----------:|:-----------:|:----------------:|:--------------:|:---------------:|:-------------------:|:-----------:|:-----------:|:----------------:|\n   | 0.0 | Type-I Error |    0.030    |    0.032    |      **0.024**    |     0.030      |      0.035      |      **0.029**       |    0.030    |    0.030    |     **0.027**     |\n   | 0.5 | Power  |    0.081    |    **0.097**    |       0.093   |     0.081      |      0.103      |      **0.114**       |    0.081    |    0.084    |     **0.088**     |\n   | 1.0 | Power  |    0.267    |    0.310    |      **0.320**    |     0.267      |      0.311      |      **0.335**       |    0.267    |    0.276    |     **0.286**     |\n   | 1.5 | Power  |    0.546    |    0.592    |      **0.611**    |     0.546      |      0.586      |      **0.613**       |    0.546    |    0.558    |     **0.572**     |\n   | 2.0 | Power  |    0.810    |    0.847    |      **0.867**    |     0.810      |      0.835      |      **0.863**       |    0.810    |    0.815    |     **0.825**     |\n   | 2.5 | Power  |    0.955    |    0.967    |      **0.974**    |     0.955      |      0.965      |      **0.970**       |    0.955    |    0.960    |     **0.962**     |\n   | 3.0 | Power  |    0.994    |    0.996    |      **0.997**    |     0.994      |      0.996      |      **0.997**       |    0.994    |    0.994    |     **0.994**     |\nTo conclude, we sincerely thank you for your detailed and constructive review. Your comments on the theoretical aspects have guided us in refining the arguments and completing a more rigorous proof framework. We also found your suggestion regarding the potential for statistical inference based on our method to be highly insightful. In response, we have revised the manuscript to incorporate your suggestions and would welcome any further discussion. Thank you again for your thoughtful review."}, {"review_id": "cD3UEADeR5", "reviewer": "Reviewer_b6bo", "summary": "The paper showcases a reweighing procedure to include historical data (not randomized) together with randomized data to compute an average treatment effect.\n\nApart from the standard assumptions on historical data they add a possible shift on the reward function between the experiments and the historical data.\n\nEDO is the ATE computed only data that has been randomized and then a proxy for the ATE using historical data, where the remaining expected value of the reward is computed with the experimental data.\n\nThe final estimator is a convex combination between the ATE and the estimator computed using the historical data", "questions": "No questions.", "limitations": "yes, albeit in the appendix.", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n- The assumptions are not too crazy and the paper presents a bound on the best possible MSE attained. \n- They give sufficient conditions to guarantee optimality of the EDO\n- Good numerical simulations\n\nWeaknesses\n\n- I do not understand why the new estimator is an improvement over the ATE. It is clearly biased and only in a special case will recover the ATE.\n- They do not perform a characterization for when the EDO is optimal", "was_revised": true, "final_justification": "It is a well executed paper although heavily based on Li et al. [3]. However because it is an interesting and well presented result I decided to maintain my score.", "rebuttal": "Thank you for your thoughtful and critical assessment. Many of your comments will help us produce a more readable and self-contained version of the paper. Below, we address each of your specific concerns in turn. \n\n* **Improvement over the ATE.**\nThank you for the insightful comment. We respectfully clarify that the proposed estimator is designed to improve the mean squared error (MSE) of the standard ATE estimator by integrating experimental data with potentially large-sample historical data. The core idea of our method is to adaptively combine potentially biased but low-variance historical estimators with unbiased experimental estimators—striking a principled balance between bias and variance in finite-sample settings. This leads to consistently improved performance across a broad range of practical scenarios.\n\n\n  - When the bias introduced by the historical data is small or negligible, the proposed estimator effectively leverages both datasets, benefiting from the larger sample size and achieving a smaller MSE compared to the experimental-data-only (EDO) estimator.\n\n  - When the bias is moderate, our method balances bias and variance in a way that reduces overall MSE. Minimizing MSE—reflecting the fundamental trade-off between bias and variance—is widely recognized as a more practical and robust objective in real-world applications. While the proposed estimator may introduce some bias, it is carefully calibrated to ensure that the reduction in variance outweighs the increase in bias, particularly in settings with distributional heterogeneity between datasets.\n  \n  - When the bias is substantial, the learned weight on historical data approaches zero, causing the proposed estimator to behave similarly to the EDO estimator. This adaptive behavior ensures robustness even under significant posterior shifts.\n  \n  - As demonstrated in our toy example (Table 2 on Page 5), even a slightly biased estimator can outperform the standard ATE estimator significantly in terms of MSE. Our theoretical analysis rigorously characterizes this trade-off, and our empirical evaluations across diverse scenarios consistently show that the proposed method achieves near-oracle performance by effectively managing the bias–variance balance.\n\n* **Perform a characterization for when the EDO is optimal.**\n  We thank the reviewer for the helpful comment. Below, we provide a more detailed explanation of the conditions under which the EDO estimator tends to perform best.\n\n  In Section 4 of the paper, we analyze five representative scenarios and formally characterize the settings where the experimental-data-only (EDO) estimator achieves the lowest mean squared error (MSE). These conditions are explicitly stated in:\n  - Corollary 1 (Page 7): In the presence of heavy-tailed historical rewards, EDO becomes optimal because the high variability in historical data outweighs its variance-reducing benefit. Under mild regularity conditions, we show that the optimal weight function converges to 1, effectively defaulting to EDO.\n  - Corollary 4 (Page 7): When there is moderate or large posterior shift between historical and experimental data (i.e., substantial discrepancies in conditional outcome distributions), our analysis demonstrates that the optimal weight again converges to 1. In such settings, incorporating biased historical data would result in significant estimation error, making EDO the preferred choice.\n\n  In both cases, we derive upper bounds for the excess MSE of our proposed estimator relative to EDO, and prove that this excess is negligible under the specified conditions. These results demonstrate that the proposed method is adaptive: it defaults to EDO when EDO is indeed optimal.\n\nThank you again for your thoughtful review. We look forward to further discussion."}]

Author rebuttal:

Meta-review: This paper introduces a novel weighted Average Treatment Effect (ATE) estimator designed to integrate experimental data (in which a binary treatment is randomized) with observational data (collected from a control treatment only). The key contribution of this work is a new technique that linearly combines experimental and observational estimators using data-dependent weights optimized to minimize MSE. The authors evaluated the empirical performance of the proposed method in synthetic and semi-synthetic datasets, with results indicating that the proposed estimator achieves lower MSE than competitors.

---
All reviewers agreed that this paper tackles an important and challenging problem. They found the proposed method to be simple, intuitive, and novel, highlighting, for example, that the use of contextual bandits to optimize mixing weights is an interesting and unexplored direction in the literature. The reviewers also praised the strong empirical results, with the proposed estimator achieving superior performance compared to baselines.

---
During the discussion phase, reviewers raised a number of concerns. Most were relatively minor and were adequately addressed by the authors in their thorough rebuttal. That said, two central concerns remained and were the primary focus of the discussion. First, two reviewers commented on the perceived limited novelty of this method relative to the one introduced by Li et al. [3]. The authors provided a detailed response, clearly explaining both similarities and the ways in which their approach extends Li et al.’s. The proposed estimator, for instance, dynamically determines how to integrate data through a learned covariate-adaptive policy explicitly optimized to minimize MSE. This contrasts with the constant-weight scheme leveraged by Li et al., which merely minimizes a pessimistic bound on MSE. In addition, the proposed approach extends Li et al.’s framework by enabling the estimator to adapt locally to distributional heterogeneity. Achieving this capability required substantial new mathematical analysis, which resulted in a technique capable of improving efficiency under distributional shift through a scheme that adapts combination weights as a function of the state and time—a key capability for addressing sequential settings. Finally, the authors emphasized in the rebuttal that their estimator is optimal across five key scenarios, thus covering a wide range of practically relevant cases. Taken together, these distinctions support the paper’s theoretical and methodological novelty.

The second major concern raised during the discussion phase related to the correctness of one of the paper’s theoretical results—in particular, Theorem 1. The authors fully addressed this point by strengthening Assumption 1 to ensure uniform concentration over the weight class. They also provided a more rigorous proof sketch and outlined two construction strategies for obtaining uniform upper bounds on bias and variance.

---
Overall, the reviewer–author exchanges were constructive, and the authors appeared to provide strong and clear responses to all key concerns raised. Post-rebuttal, the reviewers indicated that their main questions were resolved. The reviewers encouraged the authors to revise the manuscript to address all major points raised during the discussion and to incorporate key requested clarifications (for instance, by more explicitly discussing how this work’s Theorem 1 differs from the one introduced by Li et al.).
Decision: Accept (poster)