Adaptive Data-Borrowing for Improving Treatment Effect Estimation using External Controls
Authors: Qinwei Yang
Jingyi Li
Peng Wu
Venue: NeurIPS 2025 poster ()
Keywords: Adaptive Selection
Causal Inference
Data Fusion
Efficiency Improvement
Primary area: probabilistic_methods
TL;DR: We propose a novel influence-based adaptive sample borrowing approach to enhance treatment effect estimation in RCTs by effectively incorporating external controls.

Abstract: Randomized controlled trials (RCTs) often exhibit limited inferential efficiency in estimating treatment effects due to small sample sizes. In recent years, the combination of external controls has gained increasing attention as a means of improving the efficiency of RCTs. However, external controls are not always comparable to RCTs, and direct borrowing without careful evaluation can introduce substantial bias and reduce the efficiency of treatment effect estimation. In this paper, we propose a novel influence-based adaptive sample borrowing approach that effectively quantifies the "comparability'' of each sample in the external controls using influence function theory. Given a selected set of borrowed external controls, we further derive a semiparametric efficient estimator under an exchangeability assumption. Recognizing that the exchangeability assumption may not hold for all possible borrowing sets, we conduct a detailed analysis of the asymptotic bias and variance of the proposed estimator under violations of exchangeability. Building on this bias-variance trade-off, we further develop a data-driven approach to select the optimal subset of external controls for borrowing. Extensive simulations and real-world applications demonstrate that the proposed approach significantly enhances treatment effect estimation efficiency in RCTs, outperforming existing approaches.

Adaptive Data-Borrowing for Improving Treatment Effect Estimation using External Controls
Qinwei Yang', Jingyi Li?, and Peng Wul-
Beijing Technology and Business University?, National University of Singapore
Abstract
Randomized controlled trials (RCTs) often exhibit limited inferential efficiency in estimating treatment effects due small sample sizes. In recent years, the combination of external controls has gained increasing attention as a means improving the efficiency of RCTs. However, external controls are not always comparable to RCTs, and direct borrowing without careful evaluation can introduce substantial bias and reduce the efficiency of treatment effect estimation. In this paper, we propose a novel influence-basee adaptive sample borrowing approach that effectively quantifies the "comparability" of each sample in the external controls using influence function theory. Given a selected set of borrowed external controls, we further derive a semiparametric efficient estimator under an exchangeability assumption. Recognizing that the exchangeability assumption may not hold for all possible borrowing sets, we conduct a detailed analysis of the asymptotic bias and variance of the proposed estimator under violations of exchangeability. Building on this bias- variance trade-off, we further develop a data-driven approach to select the optimal subset of external controls for borrowing. Extensive simulations and real- world applications demonstrate that the proposed approach significantly enhances treatment effect estimation efficiency in RCTs, outperforming existing approaches.
Introduction
Randomized controlled trials (RCTs) are regarded as the gold standard for estimating treatment effects, as randomization effectively eliminates confounding bias ential efficiency is often limited by small sample sizes, due to high costs and lengthy recruitment periods 6 7 To address this challenge, there has been growing interest in novel clinical trial designs that leverage external real-world datasets containing only control arms, referred to as external controls, to improve treatment effect estimation in RCTs. For example, in digital marketing, platforms may combine small-scale A/B test data with historical single-arm user behavior logs to enhance infer- ence (9) In medical research, oncology trials often incorporate historical control data to strengthen causal conclusions [10]. Similarly, rare disease studies frequently augment small randomized trials with matched external controls from disease registries to improve statistical power
However, their infer-
To fully exploit external controls, most studies rely on the exchangeability assumption 12 which posits that the distribution of potential outcomes under control remains invariant between RCTs and external controls when conditioned on baseline covariates [13]
[4 [5 [2] [6] However, the exchangeability assumption requires that individuals in both RCTs and external controls follow the same pattern between the covariates and the potential outcome under control, which may be less
plausible in real-world applications due to individual heterogeneity 17 controls usually have significantly larger sample sizes than RCT's and often exhibit greater individual heterogeneity [15 I2 20). When some individuals in the external controls display patterns that differ from those in the RCTs, the exchangeability assumption is violated and leads to biased conclusions.
18 19]. practice, external
*Corresponding author: vengwu@btbu.cutt
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

In this paper, we aim to improve treatment effect estimation in RCTs by adaptively borrowing external controls without relying on the exchangeability assumption. The key lies in (a) measuring the "comparability" of each sample in the external controls for treatment effect estimation, thereby distinguishing between comparable and non-comparable samples, and (b) defining and identifying the optimal set of comparable samples.
For the first goal (a), we propose an influence-based adaptive sample borrowing approach. Specifically, We employ the influence function [2] [22] to quantify how each external control sample perturbs the outcome model fitted on the RCT's control group. yielding influence scores that reflect the comparabilityy of each sample in external controls. Intuitively, sample with a smaller influence score has less impact on the outcome model in the RCT and is therefore considered more comparable. Including such samples in the RCT can effectively increase the control group's 's sample size without introducing bias. In contrast, a higher influence score suggests that including the sample would substantially affect the outcome model, thereby introducing bias into the treatment effect estimation. Using ranked influence scores, we construct nested candidate subsets of external control samples.
For the second goal (b), we begin by deriving the semiparametric efficient estimator [23 that combines RCT data with an arbitrary candidate set of external control samples under the exchangeability assumption. Such an estimator minimizes the asymptotic variance among all regular estimators and is often considered optimal 24 2.5
under the given assumptions. We establish the consistency and asymptotic normality of the proposed estimator. Then, recognizing that the exchangeability assumption may not hold for all candidate external control samples, we analyze the asymptotic bias and variance of the proposed estimator under violations of exchangeability. Finally, to determine the optimal external control samples for treatment effect estimation, we propose a data-driven approach that minimizes the estimator's 's mean squared error. The main contributions are summarized as follows.
We reveal the limitations of existing approaches for estimating treatment effects by combining RCT data with external controls.
We propose an influence- beatee sample borrowing approach, which can effectively quantify the comparability of each sample in the external controls.
We develop a data-driven approach to select the optimal subset of external control samples based on the MSE of the proposed estimator.
We conduct extensive experiments on both simulated and real-world datasets, demonstrating that the proposed approach outperforms the existing baseline approaches.
2 Related Work
There has been an increasing amount of research in settings where RCT data augmented with external controls to improve efficiency in estimating treatment effects since [26]. termed external control, historical control, or history borrowing in related literature. Under the exchangeability assumption (or its analog) for the potential outcome under control, several approaches [13] 15 27
have been proposed to estimate treatment effects RCTs using external controls. However, the exchangeability assumption may be less plausible in practice due to individual heterogeneity. As a result, combining the RCT data and external controls directly will lead to bias. To address the
issue, various methods have been developed, including matching and bias adjustment (28]. power
priors (29, meta- analytia predictive priors 30), and conformal prediction (3T] More recently,
ofate-of-the-art approach proposed by [17] revealed the limitations of previous approaches and
developed an adaptive lasso-based borrowing approach, which borrows a comparable subset of
external controls through bias penalization. Unlike these studies, we propose novel influence-
sample borrowing approach that utilizes influence functions to quantify the perturbation effect of each external control sample the outcome model in the RCT controls. This enables the derivation of individual-level influence scores that capture the compatibility of external controls. Also, we develop approaches for borrowing the optimal subset of external controls. In this sense, our approach complements existing methodologies.
Beyond combining RCT data with external control data, there are many other settings for data
combination 12
16,
32
3
34 For example, one can combine RCT (or experimental) data with
external data that contain only covariates (35 36) In such settings, the goal is typically to generalize the causal effect from the RCT to the external population, rather than to improve causal effect estimation within the RCT itself. Another common setting involves combining RCT data with
2

---

confounded observational data that suffer from unobserved confounders 37 38 In addition, several studies have combined multiple datasets for causal inference [6 datasets, it is important not only to consider which identifiability assumptions to adopt in order to improve causal effect estimation, but also to address how to preserve privacy. Unlike these studies that either use the entire external dataset or do not use external data at all, our method performs individual-level selection of external data.
When combining multiple
3 Preliminaries
Let X E X C Iº denote the observed pre-treatment covariates and Y E Y C. R the outcome of interest. The binary treatment indicator A E {0, 1]. where A indicates treatment and = indicates control. Under the potential outcomes framework 41 42 each individual has two potential outcomes (0) and (1), corresponding to control and treatment, respectively. We maintain the stable unit treatment value assumption 43], the observed outcome is Y = (1 A)) + AY
Suppose we have access RCT data and external control data (external controls) containing only control samples. The RCT data and external controls are denoted by
(X,, A., Y, R2 L, E} and (X,,
Y, R2 0, O}.
where Rt is a data source indicator: R = 1 corresponds to the RCT data & and R = 0 to the external controls O. Let Ne and No denote the sample sizes of the RCT data and external controls, respectively. Let P(- R = 1) and (- R = 0) represent the population distributions of the RCT data and external controls, respectively, and let denote the expectation under
The causal estimand of interest the average treatment effect (ATE) in the RCT population, which is defined R[Y (1) Y R 1j. For identification, invoke the common assumption in the causal inference
45 46
Assumption (Strong Ignorability for RCT Data) (a) IL (Y(O).Y(1)) I (X, R 1); (b) < el (z) - P(A=
x, R = 1) for all x, where e (x) is the propensity score.
Assumption 1 suggests that given the covariates X, treatment A is independent of the potential outcome (a). This implies that confounding between and Y can be eliminated by conditioning on X. Assumption [b) ensures that individuals with X = x have a positive probability of receiving treatment. Assumption is inherently satisfied in RCTs due to the randomization mechanism. Under Assumption I. T can be identified based only on the RCT data,
= Elu4 X (x( R= 1),
0, the outcome models in the RCT data. When only RCT data are available, the semiparametric efficient estimator of under Assumption
where Ma (X) = E|y X,A a, R for
1
A.YY (X.))) (1 - A-A.)(Y: -)n(X)) + (X,( 1 - (X,)
Paipw
- (oo())))
No
(X.))
3E
where (z(zz) (x), and é (I) are estimates of (o(x), (x))), and ej (z). respectively. This is the classical augmented inverse probability weighting (AIPW) estimator, widely studied in prior work [7 48 49 50 51 52 53 54). The Fnipw serves an asymptotically unbiased benchmark for T. Nevertheless, estimating T using only the RCT data often lacks efficiency due to small sample sizes, resulting from high costs, long recruitment periods, and ethical or feasibility constraints. This paper aims to improve the estimation efficiency of r by fully leveraging external controls.
Motivation
To leverage external controls, most studies 13 14 12
rely on the exchangeability assumption [55
Assumption (Exchangeability) R IL Y(0)
X in P..
Assumption 2 states that the conditional mean of Y()) is identical between the RCT data and external controls, i.c. B (0) X = R =] E|y (0) X R 1]. This establishes connection
3

---

5
RCT
RCI
1
0
0.00 0.25
0.75
1.00 1.75 0.00 0.25
1.60
1.55
X
x
(a)
(b)
Figure 1: Borrowing behavior comparison of different approaches in synthetic example. (a) The
borrowed samples of the adaptive lasso-based borrowing approach [17]. (b) The borrowed samples of the proposed influence-basee borrowing approach. The influence-basee basedd approach identifies set of samples with highly comparable. Details of the synthetic example can be found in Section 7
between RCT data and external controls. Under this assumption, we can use the entire external controls to improve the treatment effect estimation in RCTs [1]
However, the exchangeability assumption is usually implausible in practice due individual het- erogeneity. Thus, directly integrating external controls without proper scrutiny can lead to biased estimates. To mitigate this issue, state-of-the-arr approach has been proposed by Gao al. (2025) 17 to find the optimal subset of external controls for integration to improve treatment effect estimation in RCTs. In this section, provide brief overview of this approach and analyze its limitations, thereby motivating this work.
Specifically, Gao al. (2025) [17 introduced a vector of bias parameter by (b1.0..... bNo. .) for all j E O, where 0 E(Y; X,, R 0) EE(Y, X,. A, 0, R 1) == (o.o(X)) mo(X)) The bias parameter by quantifies the difference in conditional mean outcomes between external controls and RCT's controls. The authors treated external control samples with zero bias "comparable" and aimed identify such unbiased samples from the pool of external controls. Specifically, let 6y. = o (Xx) Xx) - (X,) be consistent estimator for by.u and let (b1.0.... bNo..) be an initial estimator for bu- Then, they obtained an sparse estimator of by via optimizing the adaptive lasso loss:
{ (6-))"E,'6-6) +)
16,o0l
b = arg min
Σ
13,01
JEO
where En is the variance estimate of b, (A, v) two tuning parameters. Finally, they borrowed external control samples with estimated zero bias.
To better understand the borrowing behavior of the adaptive lasso-based approach, conduct simulation using a simple synthetic dataset (see Section [ for details). As shown in Figure [a), where green points denote the external control samples borrowed by the adaptive lasso-based approach, observe that this approach exhibits several limitations:
Suboptimal comparability: As shown in Figure aa), the samples (marked in green) borrowed by the adaptive lasso-based approach may exhibit patterns that differ substantially from those in the RCT's controls, resulting suboptimal comparability. Intuitively, the adaptive lasso-bace approach borrows samples with small values of b(x) := (z) - po(z), where b(x:) represents average discrepancy that overlooks individual-less heterogeneity in the neighborhood of Consequently, the approach may include samples with high variability (even picking out several outliers), simply because their mean outcomes close to those in RCT's controls.
Sensitivity to outliers: The final estimator i heavily depends on the estimate of ( which is learned using all samples from the external controls. When outliers present in the external

---

controls, the estimate of (x)) may become biased, introducing estimation errors for individual samples and exacerbating con-comparability issues.
These limitations greatly hinder the effective use of external controls, limiting their potential to improve the efficiency of treatment effect estimation in RCTs. Thus, developing improved data borrowing methodology is essential.
5 Adaptive Sample Borrowing via Influence Function
In this section, we propose a novel influence-basee based borrowing approach designed to more effectively incorporate external controls, thereby enhancing the efficiency of treatment effect estimation.
The key lies in how measure the comparability of cach external control sample to the RCT's controls. We formalize this problem by asking the counterfactual: how do the parameters of the outcome model of the RCT's controls change if we add external control sample? If the model parameters show minimal change, the sample can be considered comparable and suitable for borrowing.
Denote C = (X,, A; 0, Y, R; 1, E E as the RCT's controls, and N: is the sample size. Given RCT's controls {Z, = (X,, Y.), Ze E C}. To train the model Nó (parameterized by 0), learn the model parameters by the empirical risk minimizer, Ô def arg minsge Σx,ec L(Z;; 0) where [(Z;;) = (Y: - {Xo 0))² that wice-differentiable and convex in 0, and À > controls regularization strength. For an added external control sample
(z, y), we denote the modified parameters as f+.. which is obtained by retraining the model after adding 2 as 0+z def arg minggee Σz, ECUa L(Z;;)) 0). Then, we can obtain the influence score of the external control sample z on loss over RCT's controls,
IF I.Floss
loss
def
Z.EC
The influence score I.Fiuss measures the actual influence of A large value of I.F indicates that 2 has a significant impact on the model No-
However, retraining the model for each added is prohibitively slow and expensive. Fortunately, instead retraining the model, the influence function gives an approximation of how much the model changes when external control samples influence function is to weight the external control sample 2 by infinitesimal steps to produce new
added to the RCT data (57). The main idea behind the
model parameters 0z.s def arg mingge No Σz.ec [(Z,,)) 0) + eL(z, 0). A classic result 22 57 tells us that the influence of upweighting on the parameters Ó is given by
I Iparams (2) def ddc.=
de
l==0
where Hj def No NE' (Z, 0) is the Hessian matrix. Then, the influence of adding an external control sample the prediction loss of single sample Z₂ E C can be expressed in closed-form expression by applying the chain rule, as follows,
I (z, Zi) def dZ. de
do de
O)
--VoL(Z,,) T
According to Eq. 4. [z, Zi) is equivalent to "the first-order derivatives of the model parameters Z," multiplied by "the parameter change when we add the external control sample z". Furthermore, we can quantify the total influence of an external control sample z on the entire RCT's controls as the influence score, which is given as,
IFoo (2) ~
Σ HH
2,E0
For all external control samples (Z, (X,, E O). we can calculate the influence score set
{IF loss [Z,),j i E O} according to Eq. ( The influence scores measure the comparability of each
5

---

external control sample. The proposed adaptive data-borrowing method is based on the influence scores and consists of two main steps:
Step 1. Based on the ranking of influence scores, we construct a series of nested subsets of external controls, where each subset comprises the top-K K most comparable samples (i.e., the top-K samples with the smallest influence scores).
Step 2. Find the optimal K that minimizes the mean square error (MSE) of the estimator based on the RCT data and the selected top-K external controls. This method will be detailed in Section 6
Unlike the adaptive lasso-based approach, the inffuence- based approach does not rely on modeling No,o(z). In addition, it noteworthy that the inftuence score is defined at the individual level, and the influence score for each point is unaffected by other points in the external controls. As result, it robust to outliers external controls. As shown in Figure [b), the proposed method effectively selects the top-K K points (in red) that are close to the RCT controls, while remaining robust to outliers in the external controls.
Improved Estimation by Leveraging of Borrowed External Controls
In this section, based on the ranking of influence scores, our goal is to select the optimal subset of external controls that minimizes the mean squared error (MSE) of the proposed estimator.
Let S C O be a subset O borrowed by the influence function, and Ngs is the sample size. For ease of presentation, we denote Is as the combined population that Ps(- R = 1) and I-s( R 0) denote the distributions of RCT data and borrowed external controls. The expectation operator of I's denoted by EEs. For clarity, summarize the nuisance parameters in Table that utilized in the following theory analysis, and all of them can be identified from the observed data.
In subsection 6.1 construct the estimator of r by combining the RCT data E and the borrowed set S, under the weaker version Assumption [ [Assumption beloww. In subsection 6.2 we first analyze the bias and variance of the estimator under the violation of Assumption 3 and then select the optimal K that minimizes the corresponding MSE.
6.1 Fused Estimator Based on Exchangeability for Borrowed External Controls
To fully utilize the borrowed data, we aim to derive the semiparametric efficient estimator of r, which regarded as optimal since it achieves the semiparametric efficiency bound-yield cound-yyelding the smallest asymptotic variance under standard regularity conditions [23] 24). To construct the semiparametric efficient estimator, we typically first derive the efficient influence function and the semiparametric efficiency bound based the available data and assumptions.
Assumption (Exchangeability for Borrowed External Controls) R JL ( X in Ps.
Assumption 3 is analogous to, but weaker than, Assumption 2 differing only by replacing PP with
Ps. This implies that the outcome regression functions for Y (0) given covariates are the same in E and S. that is, (Y() X, R 1] = Es (0)) X, R 0]. Assumption is plausible when S is appropriately selected by the influence function. As shown in Figure (b), the RCT controls (blue
points) and the selected external controls (red points) are close, and follow a similar distribution,
supporting the plausibility of Assumption 3
Lemma Under Assumptions 7 and 3
the efficient influence function of T is
T(X) RA(Y mmi(()) (1 A)(Y mX)(X) + (m1(X) mo(X) - r}. ex(X)
1 exes(X)
) are defined in Table and q Ps(R- = 1). The associated semiparametric efficiency bound for is Var(ø).
where es (X)), T (X), (X)X), and mx(X)
Lemma D presentss the efficient influence function of under Assumptions T [and] can construct the estimator of T as follows
Based on it,
1
*(X.) ( R,A, mx()))) (1 - A,) -mn((.)) - ês(X.)
} + g
R. (m) (X;) (x,))))))
is NNE + No
Σ SEEUS
1
(X,),

---

Table 1: Nuisance parameters in faipy and fg.
Nuisance parameters
Description
(1(X) =
1 1| =
propensity score in outcome regression function in & sampling score in & U propensity score in USS oulcome regression function in & US
(X)) = -EE(Y X = a R = Bs X,A= R=
(XX) = Ps(RR
X)
(s(X) Ps(A=11 X) (X)x((()
(X) Es(Y X, A a).
Note that by definition, [1(X) (X) but mo( # mo(X).
where (x)), (x)), ma (x) estimates of (1 ma ma()) for 0, and (s()) = (x)
Lemma Under Assumptions
- (2))112 op (oppnn11///) 11/2) and [1*(x) - (>) [ ma (z) = OF (n for all x E X and a E {0, 1}, then is satisfies VNg N -) S NN(,, where o² is the semiparametric efficiency bound of T, and 4
and (z)
means convergence in distribution.
Lemma 2 establishes the consistency and asymptotic normality of the estimator ñs. In addition, it shows that ēs is semiparametric efficient, provided that the nuisance parameters are estimated at convergence rate faster than n -1/4, These conditions common in causal inference and are easily satisfied using a variety of flexible machine learning methods [58]
6.2 Find the Optimal Borrowed Set
given borrowed set S, we construct the semiparametric efficient estimator řs by combining data from & and S. However, the proposed estimator is relies on the key Assumption 3 which may hold for all choices S.
In Section 6.1
When Assumption 3 is violated, fs is no longer a consistent estimator of t and yields a bias. Intuitively, selecting S involves bias-variance trade-off: increasing the sample size in S tends to reduce variance but may also introduce "non-comparable samples, thereby increasing bias. The proposed approach is based the bias-variance analysis of fs-
Theorem (Bias- Variance Analysis) Under Assumption only, (e) e (x))12 (m) ma(x)12 1/33) and [|*(2) (z))lle [[ma(x)) m. (2)]l2 o 1/2) for all x E X and E {0, 1}. then is satisfies VNe + Ns T - bias(fs)) 5 (5((0,00) o3), where 0² = Var(ø), is defined in Proposition and bias(fs) = Es 1 (mo()) mX]]]
Theorem exxeends Lemma by allowing Assumption to be violated. When Assumption Tholds, the bias term vanishes, and Theorem I reduces to Lemma with the asymptotic variance o² achieving the semiparametric efficiency bound for under Assumptions 3 In contrast, when Assumption is violated, bias arises and is proportional to the expectation of (X mx(X), and the asymptotic variance a² retains the same form but is no longer the semiparametric efficiency bound.
Based on Theorem [ we propose to select the optimal S that minimizes the MSE of is from the candidate sets when Assumption 3 is violated. Specifically, the MSE of the estimator is is MSE(fs) bias? (fs) + var(fs). We estimate the bias using ts fapp and estimate the variance using the sample variance of J, where @ is the estimate of d obtained by substituting the nuisance parameters with their estimated values. Our goal is to find the optimal subset S defined by
S = arg min MSE(fs.).
S,ES
where S = (Sx : -= 1,.., No).
S₁ denotes a subset of external controls corresponding to the top-k smallest influence scores, No is the sample size of external controls.
Importantly, the proposed selection strategy for does not rely on Assumption Or nor does impose any restrictions on the distribution of the external controls. Therefore, our method exhibits high applicability across a variety of settings. RCT data typically have relatively small sample sizes, as their collection is often time-consuming and costly, and further limited by the inclusion
7

---

criteria set by the experimenter [39] In comparison, external controls are observational, more readily available, and often drawn from larger and more diverse sources, exhibiting greater individual As a result, it is inevitable that some individuals (or outliers) in the external controls will exhibit patterns that differ significantly from those the RCT controls. this case, the adaptive lasso- approach can only achieve sub-optimal performance (as discussed in Section 4), whereas our proposed method can effectively accommodate such scenarios.
heterogeneity [13
20 60
7 A Synthetic Example
To better analyze the borrowing behavior of the proposed influence-based borrowing approach, we first compare it with the adaptive lasso-based borrowing approach [17] on a simple synthetic dataset.
Y = n + GT= Ye = = + 2. + Cec-
where Xn, X ~ UO, 2). and ~ N(0, 0.2?), Cec ~ (O, 0.5²). The Xeu is the 1-dimensional covariates. And the external controls contain five outlier samples.
As shown in Figure [ the adaptive lasso-based approach proposed by [17] achieves only suboptimal comparability and is sensitive outliers. This suggests that relying solely the conditional mean difference may not be an optimal strategy for identifying comparable samples. In contrast, the proposed anfluence-based borrowing approach achieves better performance by employing the influence function theory to quantify the comparability of each external control.
Experiments
To demonstrate the effectiveness of the proposed approaches, we conduct experiments on three datasets, including two synthetic datasets and a real-world dataset. Experimental details (e.g.. parameter settings), are provided in the Appendix C
8.1 Experimental Setup
Simulation Study. We generate synthetic datasets mimic both RCTs and external controls, with each observation characterized by d = 8 dimensional covariates X. The synthetic datasets consist of Ng 400 samples with N, 300 in the treated group and N; 100 the control group, while the external control dataset includes No 800 samples. The treatment assignment A for the RCTs completely at random. Following [17. we designed two data- data-generating mechanisms: linear outcome model (denoted as "Linear") and a nonlinear outcome model (denoted as "Nonlinear").
As outlined in Table 2 for the "Linear" mechanism, we simulate the RCTs baseline covariates Xn ~ Nu,, of) and the external controls baseline covariates Xe: ~ N ( az). Here, μ4 # p2, with |P2 Mal quantifying the magnitude of covariate shift between the two groups. The standard reflects greater heterogeneity in the external controls compared to the RCTs. We initially set μ1 = 0, 1 and P2 0.1, 02 2. The coefficient A8 U(O.8, 1.2]d) characterizes the structural differences in outcome models between the external controls and the RCTs. and constitute the effect of concurrency bias, in which 7;, i O, is simulated by taking values of (0. 1, 2) with probability 1/3 and 6 represents the level of inconcurrency, where we set 8 = 0.1. Similarly, for the "Nonlinear" mechanism, X N[-2,2 (f) and X N1-4, oz), both follow a truncated normal distribution. The A,32 characterizes the structural differences and 8 represents the level inconcurrency. We set A82 U((.88, 1.2)d) and å = 1.0.
deviation 02
Table Simulation settings: model choices (linear and nonlinear).
generate mechanism
parameters
Y B Xn Aog (1, Xn) Ents
B1 U ([-1,11) )r N(O, 1.0%)
Linear
Yo.. (B1 AB) X T Ceas
Lec 1.52)
Y, = a exp X Aa (1, Xx) + Eres
B2 ~ ([ N(0, 1.0²)
Nonlinear
Yec = a expl A3,) Xxc 5. Eecr
Eec N(0, 2.02)

---

Real-Data Application. In addition to synthetic datasets, we also utilize two real-world datasets from the National Supported Work (NSW) program (RCTs) [6 and the Population Survey of Income Dynamies (PSID) (external controls) 66 The NSW dataset consists of Ne = 345 samples with N₂ 185 in the treated group and 260 in the control group, where we randomly selected N, = 80 samples as the control group, while the PSID dataset includes No = 123 samples. The NSW program investigates whether providing intensive job training and supported work experience could improve employment outcomes for economically disadvantaged populations. This dataset contains a treatment indicator (i.e., the training program, A: I for treated, 0 otherwise), demographic covariates ( age, education years, race indicators), Socioeconomic variables (marital status, education attainment), and earnings (1974, 1975, and 1978 outcomes). We take the earnings in 1978 as the interesting outcome Y. The external control dataset, PSID, is used for external comparison with the NSW dataset, which includes the same 10 columns as the NSW dataset, with all observations representing untreated individuals (A 0 for all individuals).
Baselines and Evaluation metrics. We compare our proposed influence- based approach (frr with the following baselines in the above datasets. (a) the augmented inverse probability weighting (AIPW) estimator without borrowing (Fiip) [63] (based only on the RCT data); (b) the AIPW estimator with full borrowing (famn) [5.. (c) the AIPW estimator with adaptive lasso-based borrowing approach (fasso) [7]. We report three evaluation metrics, including standard deviation (std), bias, and mean
squared errors (MSE), with detailed definitions provided in Section 6.2
8.2 Results
The adaptive lasso-based approach assesses the comparability of external control samples based on the magnitude of bias b, where smaller bias indicates stronger comparability, and vice versa. In contrast, the influence-basee approach evaluates comparability using influence scores, with smaller scores reflecting stronger comparability. By ranking bias b, identify the Top-K most comparable external control samples borrowed by the adaptive lasso-based based approach. Similarly, by ranking influence scores, identify the Top-K K most comparable external control samples borrowed by the proposed influence-based approach.
0.250
0.0005
0.028
0.5007
0.016
0.0008
0.150
0.0003
0.022
6.90
0.100
0.0004
0.075
0.050
0.0007
300
200
200
410
500
600
700
D02
200
200
2000
400
500
600
B02
100
120
Tap.
Top.
(a) Linear (Synthetic Data)
(b) Nonlinear (Synthetic Data)
(c) NSW & PSID (Real- World Data)
Figure Comparison of performance of different approaches at different Top-K.
Sample Borrowing Behavior Analysis. Figure 2 presents the changes in MSE when borrowing different Top-K samples. First, we can find that the full borrowing approach दाu is invariant across Top-K. It has larger MSE than other approaches as it integrates the biased external controls for estimation. Second, the proposed f consistently achieves a lower MSE than the adaptive lasso-based approach faseo This is attributed to the better comparability of the external control samples borrowed by the proposed approach. Third, increasing Top-K leads to an initial decrease (or flattening) in MSE, followed by upward trend, which indicates that approach effectively prioritizes the more comparable external control samples, compared to the adaptive lasso-based approach.
Performance Comparison. To thoroughly evaluate the validity of the proposed approach, we present detailed experimental results in Table 3 including estimated standard deviations and biases across different Top-K values. We can find the following conclusion. First, due to the biases of the external controls, the full borrowing estimator full is significantly different from faipw. leading to
Thhis data is available at attt :ttpps:///ureee nber wwwp:///wdd//aaaa/anttttese

---

Table 3: Comparison of our approach (fr), Faipws Pill and Aasso three datasets, with standard deviation (std) and bias| as evaluation metrics.
Linear std
Top-K=10
Top-K=50
Top-K=100
Top-K=150
Top-K=200 std
Top-K=250
Top-K=300
[nins]
std
[bass]
std
[bias]
sdd
[bias]
[bias]
std.
[bias]
sid
[bias
Taars
0.1075
0.1075
0.1075
0.1075
0.1075
0.1075
0,1075
Tul
0.9912
0,0951
0.0912
0,0951
0.0912
0.0951
0,0912
0.0951
0,0912
0.0851
0.0912
0.0951
0,0912
0.0951
0.994
0
0.1032
0.0964
0.1031
0.9655
0.1022
0.0998
0.1039
D.0851
0.1023
0.0665
0,1006
0,0784
0.0963
0.0005
0.0902
0.0075
0.0841
0.0185
0.0819
0.0240
0.0808
0.0307
0.0807
0.0252
0.0808
0.0251
Nonlinear
Top-K=10
Top-K=50 Km50
Top-K=100
Top-K=150
Top K
Top-K=250
Top-K=301
std
bias
[bias
bias
[blos
[bias]
Ibas
0.2232
0.2232
0.2232
0,2232
0.2232
0.2232
0.2232
0.1862
0.449/
0.1862
0.4492
0.1862
0.4992
0.1862
0.4492
0.1862
0.4492
0.1862
0,4492
0.1862
0.4492
9
0,2093
0.0130
0.2124
0,0318
0.2218
0.0648
0.2236
0.1 192
0.2202
0.1350
0.2270
0.1370
0.2280
0,1709
0.2090
0.0179
0.2117
0.0485
0.2165
0.0602
0.2152
0.0595
0.2128
0.1028
NSW & PSTD
Top-K=10
Top- Km15
Top-K=20
Top-K=25
Top-K=30
Top-K=35
Top-K=f0
std
Inios]
[Nas
[blas]
[bias]
std
[hins]
41d
[hias]
[bas
Fage
0.0131
0.0134
0.0134
0,0134
0.0131
0.0134
0.0134
0.0149
0.0226
0.0149
0.8226
0.0149
0.0226
0.0149
0.0226
0.01499
0.0236
0.0149
0.0226
0.0149
0,0226
0.0142
0.0119
0.0144
0.0144
0.0144
0.0152
0.0136
0.0106
0.0140
0.0152
0.0140
0.8174
0.0145
0.01811
0.0131
0.0123
0.0127
0.0078
0.0130
0.0074
0.0129
0.0026
0.0130
0.0013
0.0132
a significantly larger bias. Therefore, we exclude it from further comparisons below. Second, the proposed fir is closer to the benchmark faipw with smaller bias compared with all basclines. Third, the proposed für has a smaller standard deviation than both Aaso and Faipw. This is attributed to the high-quality external control samples borrowed by the proposed approach.
0.0200
6.022
0.07%%
0.000
0.0203
6.175
60100
0.000
60073
6.000
100
200
1002
700 100
100
200
5250
6400
10C
1912
180
200
100
(a) Linear, /12 0.2
(b) Lincar, M2 0.3
(c) Linear, f 0.4
Figure 3: Comparison of the performance of different approaches at different covariate shifts.
Sensitivity Analysis. We also conducted experiments to explore the robustness of the for to different U2 (i.e., covariate shift), where we consider μ2 {0.2, 0.4]. As shown Figure 3 we observe that: 1) the proposed für consistently outperforms the adaptive lasso-based approach; 2) as Top-K addition, we conducted experiments on
increases, MSE tends to decrease and then increase. different RCT control group sizes. We retain the size of the treatment group, but create different sub-samples by randomly selecting N samples from its control group of RCT. In our simulation study, consider control group sizes of N e {70, 80, 90), while for the real-data application, examine a range of N E {70, 75, 85). The corresponding results are presented in Appendix B. These results further demonstrate the robustness of the proposed approach fe.
Conclusion
In this paper, we first reveal the limitations of existing approaches in borrowing external control sam- ples for integration in estimating treatment effects. Then, we propose an influence-based borrowing approach that can overcome these limitations. The proposed approach consists of two key steps: measuring the "comparability" of each external control sample and identifying the optimal set of comparable samples. For the first step, we employ the influence function to obtain sample-specifee influence scores. For the second step, we propose a data-driven approach to select the optimal set by minimizing the estimator's 's mean squared error (MSE). A limitation of this work is that computing the Hessian matrix for the influence function may have a high computational burden for large-scale models, particularly deep neural networks with millions even billions of parameters.
10

---

Acknowledgments and Disclosure of Funding
The authors thank Professor Shu Yang of North Carolina State University for her insightful comments and constructive discussions on this work. This research was supported the National Natural Science Foundation of China (No. 12301370), the BTBU Digital Business Platform Project by BMEC, the Beijing Key Laboratory of Applied Statistics and Digital Regulation, Academy for Interdisciplinary Studies at Beijing Technology and Business University, and the BTBU Research Foundation for Youth Scholars (No. BRFYS2025).
References
[1] Guido W Imbens and Donald B Rubin. Causal inference in statistics, social, and biomedical sciences. Cambridge university press, 2015.
[2] M.A. Hernán and J. M. Robins. Causal Inference: What If. Boca Raton: Chapman and
Hall/CRC, 2020.
[3] Peng Wu, Haoxuan Li, Yuhao Deng, Wenjie Hu, Quanyu Dai, Zhenhua Dong, Jie Sun, Rui Zhang, and Xiao-Hua Zhou. On the opportunity of causal learning in recommendation systems: Foundation, estimation, prediction and challenges. In IJCAI, 2022.
[4] Peng Wu, Haoxuan Li, Chunyuan Zheng, Yan Zeng, Jiawei Chen, Yang Liu, Ruocheng Guo, and Kun Zhang. Learning counterfactual outcomes under rank preservation. arXiv preprint arXiv:2502.066 06398, 2025.
[5] Susan Athey and Guido W Imbens. The state of applied econometries: Causality and policy evaluation. Journal of Economic perspectives, 31(2)33-32, 2017.
[6] Wenjie Hu, Xiaohua Zhou, and Peng Wu. Identification and estimation of treatment effects on long-term outcomes clinical trials with external observational data. Statistica Sinica, 35:1-22, 2025.
[7] Qinwei Yang. Xueqing Liu, Yan Zeng. Ruocheng Guo, Yang Liu, and Peng Wu. Learning the optimal policy for balancing short-term and long-term rewards. In Proceedings of the 38th International Conference on Neural Information Processing Systems, 2025.
[8] Peng Wu, Ziyu Shen, Feng Xie, Zhongyao Wang, Chunchen Liu, and Yan Zeng. Policy learning for balaneing short-term and long-term rewards. In Proceedings of the 41st International Conference on Machine Learning, 2024.
[9] lavor Bojinov, David Simchi-Levi, and Jinglong Zhao. Design and analysis of switchback experiments. Management Science, (9())3355-3377, 2023.
[10] Brian Hobbs, Bradley Carlin, Sumithra Mandrekar, and Daniel Sargent. Hierarchical commensurate and power prior models for adaptive incorporation of historical information in clinical trials. Biometrics, 6773)11047 2011.
[11] Shirley V Wang, Sebastian Schnceweiss, Joshua J Gagne, Thomas Evers, Christoph Gerlinger, Rishi Desai, and Mehdi Najafzadeh. Using real-world data to extrapolate evidence from randomized controlled trials. Clinical Pharmacology & Therapeutics, (0565):1156-1163 2019.
[12] Peng Wu, Shanshan Luo, and Zhi Geng. On the comparative analysis of average treatment effects estimation via data combination. Journal of the American Statistical Association, 120(552):225--2666) 2025.
[13] Issa Dahabreh, Sarah E Robertson, Eric J Tchetgen, Elizabeth A Stuart, and Miguel A Hernán. Generalizing causal inferences from individuals in randomized trials to all trial-eligible individuals. Biometrics, (5(2):6556694 2019.
[14] Issa J Dahabreh, Sarah E Robertson, Jon A Steingrimsson, Elizabeth A Staart, and Miguel A Herman. Extending inferences from randomized trial a new target population. Statistics in medicine, 39( 1991:449:9999999) 1999-20144 2014, 2020.

---

[15] Xinyu Li, Wang Miao, Fang Lu, and Xiao-Hua Zhou. Improving efficiency of inference in clinical trials with external control data. Biometrics, 79(1):394 403, 2023.
[16] Peng Wu and Xiaojie Mao. The promises of multiple experiments: Identifying joint distribution of potential outcomes. arXiv preprint W5Xiv:2504.20470, 2025.
[17] Chenyin Gao, Shu Yang, Mingyang Shan, Wenyu YE, llya Lipkovich, and Douglas Faries. Improving randomized controlled trial analysis via data-adaptive borrowing. Biometrika, i2(2))asac069, 2025.
[18] Peng Wu, Peng Ding. Zhi Geng, and Yue Liu. Quantifying individual risk for binary outcome. arXiv preprint arXiv:2402.1000 2024.
[19] Peng Wu, Qing Jiang, and Shanshan Luo. Safe individualized treatment rules with controllable harm rates. arXiv preprint 20Xi::2505005308 2025.
[20] US FDA. Considerations for the design and conduct of externlly controlled trials for drug and biological products guidance for industry< httpss//www. fda. htt///edda/164990 O/down! oad2(2023). Accessed April, 2023.
[21] Frank R. Hampel, Elvezio M. Ronchetti, Peter Rousseeuw, and Werner A. Stahel. Robust Statistics: The Approach Based on Influence Functions. John Wiley & Sons, Inc., 2005.
[22] Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning. pages 1885-1894. PMLR, 2017.
[23] Anastasios Tsiatis. Semiparametric theory and missing data, volume Springer, 2006.
[24] Whitney K Newey. Semiparametric efficiency bounds. Journal of Applied Econometrics, 5(2):99-13) 135, 1990.
[25] Aad W Van der Vaart. Asymptotic statistics, volume 3. Cambridge university press, 2000.
[26] Stwart J Pocock. The combination of randomized and historical controls in clinical trials. Journal of chronic diseases, 1953)1175-188, 1976.
[27] Michael Valancius, Herbert Pang. Jiawen Zhu, Stephen R Cole, Michele Jonsson Funk, and Michael R Kosorok. A causal inference framework for leveraging external controls in hybrid trials. Biometrics, 80(4)uja 2024.
[28]
Elizabeth A Stuart and Donald B Rubin. Matching with multiple control groups with adjustment for group differences. Journal of Educational and Behavioral Statistics, 33(3):279-300 276-30,, 2008.
[29] Beat Neuenschwander, Michael Branson, and David Spiegelhalter. A note the power prior. Statistics in medicine, 3562-356662366 2009.
[30] David Alan Schoenfeld, Dianne M Finkelstein, Eric Macklin, Neta Zach, David L Ennist, Albert A Taylor, Nazem Atassi, and Pooled Resource Open-Access ALS Clinical Trials Consor- tium. Design and analysis of a clinical trial using previous trials as historical control. Clinical Trials, 16(5):531-538, 2019.
[31] Ke Zhu, Shu Yang, and Xiaofei Wang. Enhancing statistical validity and power in hybrid controlled trials: A randomization inference approach with conformal selective borrowing. International Conference on Machine Learning. 2025.
[32] Irina Degtiar and Sherri Rose, A review of generalizability and transportability. Annual Review of Statistics and Its Application, 10:501 2023.
[33] Nathan Kallus and Xiaojie Mao. On the role of surrogates in the efficient estimation of treatment effects with limited outcome data. Journal of the Royal Statistical Society Series B: Statistical Methodology, page qkae099, 2024.
[34] Bénédicte Colnet, Imke Mayer, Guanhua Chen, Awa Dieng, Ruohong Li, Gaal Varoquaux, Jean- Philippe Vert, Julie Josse, and Shu Yang. Causal inference methods for combining randomized trials and observational studies: review. Statistical Science, 199:165-19,,
2024. 2024.
12

---

[35] Issa J. Dahabreh, Sarah E. Robertson, Eric J. Tchetgen, Elizabeth A. Stuart, and Miguel A. Hernán. Generalizing causal inferences from individuals in randomized trials to all trial-eligible individuals. Biometrics, 75(2):685-66) 2019.
[36] Issa J. Dahabreh, Sarah E. Robertson, Jon A. Steingrimsson, Elizabeth A. Stuart, and Miguel A. Hernán. Extending inferences from a randomized trial to a new target population. Statistics in Medicine, 39: 19:1999-2014, 2020.
[37] David Cheng and Tianxi Cai. Adaptive combination of randomized and observational data. arXiv preprini 2111.15012, 2021.
[38] Shu Yang, Chenyin Gao, Donglin Zeng, and Xiaofei Wang. Elastic integrative analysis of randomised trial and real-world data for treatment heterogeneity estimatio. Journal of the Royal Statistical Society Series B: Statistical Methodology, 85(3):575-566 2023.
[39] Larry Han, Zhu Shen, and José R. Zubizarreta. Multiply robust federated estimation of targeted average treatment effects. In Conference on Neural Information Processing Systems, number 3087, pages 70453-70482, 2023.
[40] Larry Han, Jue Hou, Kelly Cho, Rui Duan, and Tianxi Cai. Federated adaptive causal estimation (face) of target treatment effects. Journal of the American Statistical Association, 2025.
[41] Donald B Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of educational Psychology, 66(5):688, 1974.
[42] Jerzy Neyman, Dorota M Dabrowska, and Terrence P Speed. On the application of probability theory to agricultural experiments. essay on principles. section 9. Statistical Science, pages 465-472, 1990.
[43] Donald B Rubin. Randomization analysis of experimental data: The fisher randomization test comment. Journal of the American statistical association, 75(371):591-59) 1980.
[44] Paul R Rosenbaum and Donald B Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41-55 1983.
[45] Guido W Imbens. Nonparametric estimation of average treatment effects under exogeneity: A review. Review of Economics and statistics, 86(1):4-29 2004.
[46] Shanshan Luo, Peng Wu, and Zhi Geng. Pseudo-strata learning via maximizing misclassification reward. arXiv preprint 2rXiv:2511.203/16 2025.
[47] Heejung Bang and James M. Robins. Doubly robust estimation in missing data and causal inference models. Biometrics, 61:962-972, 2005.
[48] Joseph D.Y. Kang and Joseph L. Schafer. Demystifying double robustness: comparison of alternative strategies for estimating a population mean from incomplete data. Statistical Science, 22:523-55: 539, 2007.
[49] Zhiqiang Tan. Comment: understanding or, ps and dr. Statistical Science, 22:560-568. 2007.
[50] Peng Wu, Shasha Han, Xingwei Tong, and Runze Li. Propensity score regression for causal inference with treatment heterogeneity. Statistica Sinica, 34:747-769, 2024.
[51] Peng Wu, Zhiqiang Tan, Wenjie Hu, and Xiao-Hua Zhou. Model-assisted inference for covariate- specific treatment effects with high-dimensional data. Statistica Sinica, 34:459-479, 2024.
[52] Peng Wu, Pengtao Zeng, Zhaoqing Tian, and Shaojie Wei. Matching-based nonparametric estimation of group average treatment effects. arXiv preprint arXiv:2508. 2025.
[53] Zhaoqing Tian and Peng Wu. Semiparametric efficient inference for the probability of necessary and sufficient causation. Statistics in medicine, 44(18-199:c70222,2 2025.
[54] Jiayi Guo, Haoxuan Li, Ye Tian, and Peng Wu. A relative error-based evaluation framework of heterogeneous treatment effect estimators. arXiv preprint arXiv:2510.16411 2025.
13

---

[55] M.A. Hernan and J.M. Robins. Causal Inference: What If. Chapman & Hall/CRC Monographs on Statistics & Applied Probab. CRC Press, 2025.
[56] Elias Bareinboim and Judea Pearl. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences, 113(27):3355-3352 2016.
[57] Dennis Cook and Sanford Weisberg. Characterizations of an empirical influence function for detecting influential cases in regression. Technometrics, 22(():495-55) 5088 1980.
[58) V. Chernozhukov, D. Chetverikov, M. Demirer, E. Duñlo, C. Hansen, W. Newey, and J. Robins. Double/dcbiased machine learning for treatment and structural parameters. The Econometrics Journal, 21:1-68, 2018.
[59] Nathan Kallus, Aahlad Manas Puli, and Uri Shalit. Removing hidden confounding by experi- mental grounding. Conference on Neural Information Processing Systems, 2018.
[60] Bénédicte Colnet, Imke Mayer, Guanhua Chen, Awa Dieng, Ruohong Li, Gael Varoquaux, Jean- Philippe Vert, Julie Josse, and Shu Yang. Causal inference methods for combining randomized trials and observational studies: a review. Statistical science, (9(1):6651191 2024.
[61] Robert LaLonde. Evaluating the econometric evaluations of training programs with experi- mental data. The American economic review, pages 604-620, 1986.
[62] Rajeev H Dehejia and Sadek Wahba. Propensity score-matching methods for nonexperimental causal studies. Review of Economics and statistics, 84(1):151-161 2002.
[63] Weihua Cao, Anastasios A Tsiatis, and Marie Davidian. Improving efficiency and robustness of the doubly robust estimator for population mean with incomplete data. Biometrika, 96(3):723- 734, 2009.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
See the abstract and [3-5] paragraphs in Section 1.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
See the conclusion (especially the last sentence.)
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

Justification: In Section 3 and Section 6.1, we provide detailed discussion of the adopted assumptions. Additionally, we present the complete proof's in Appendix A Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: In Section and Section 8, we provide a detailed description for the experi- mental datasets. In addition, we provide the datasets and codes in supplemental material to ensure easy reproduction all reported results.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
16

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide the supplemental material for datasets and codes in zip file to ensure easy reproduction of all reported results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to rum to reproduce the results. See the NeurIPS code and data submission guidelines (https
//ips.cc/publle... wwcppbllccgguuueeeeoo http//uddese/eee/ https//CoSSSubbiisce stibbbmmmssoooooccccc. for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: See Section for the detailed description simulating studies and experimen- tal details. In Appendix C, we provide the implementation Details. In addition, we provide the supplemental material for datasets and codes in a zip file to ensure easy reproduction of all reported results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We report the standard deviations for all experimental results in section 8.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [NA]
Justification: All experimental results can be easily reproduced on personal computer.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https://neurrps. wwtpppbbicc/tiicccuceeeeeeoeeeee
Answer: [Yes]
Justification:
All experiments are conducted on simulation datasets and publicly available
datasets.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: In the first paragraph of Section (Introduction), we outline various potential applications of our work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
18

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Real-world application is conducted on publicly available datasets.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: In Section 8, provide references for the datasets and the simulation setups of the Bata-gnnegating process.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/dataseta
datasets.papersstt
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [Yes]
Justification: In Section 8, we provide references for the datasets and the simulation setups of the data-generating process. addition, we provide the supplemental material for datasets and codes in zip file to ensure easy reproduction of all reported results.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We don't use a crowdsourcing service.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification:
This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
20

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLM is not an important, original, or non-standard component the core methods in this research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https://neurd) aatto:://eearrig..c/c//ooffeeeer / for what should or should not be described.
21

---

A Technical Proofs
Lemma 1. Under Assumptions and 3 the efficient influence function of T is
( RA(Y (x()))
A) mo(())
R
+ (mı (X) mo(X) - 7]:
ex(X)
1 - (X)
where es (X), (X)), mi mX),,, and mx(X) defined in Table I of the main text. The associated semiparametric efficiency bound for r is Var(
Proof of Lemma 1. Recall that (x( = e1 ( (X) = Ps(R
x). m (X) = E X, A 1], (x() Es|Y X, = 0j, and e (X) P(A = X, R = Ps =
=1 R
X, = 1) is the propensity score in the RCT data.
Let f(x) [(xr = 1), and [x/r 0) represent the density functions of X in the combined data Ps. RCT data ( = 1) and selected external data P. (- R 0), respectively. Denote f'Vo-- 31 (1)))) as the joint distribution of (0) Y (1)) conditional on X
First, we derive the tangent space. The observed data distribution under Assumptions | and (R (Y(0), X in Ps) is given as 2,,,,,,11) vlaa
ff(x) x (x)) [[aa))))
[1(0l=)(1 16000 (x))]
where [1(1))) f [wy, (1r)))) Jdyo and fo(-|x) f y1 (æ))dy are the marginal density of (1) and (()) given X = x. respectively. Consider regular parametric submodel indexed by given as
p(a, x. y. r. 0) = f(x, 0) x [f( 0) fo(ul2, 0)2--9)1- x [e 0) (z) (z,
[ 0) Txx, ((z,
which equals p(a, xr y, r) when On- Also, Sa fa(y|e, 0) =
1, = faly,,r 0,0)
by Assumption 3 Then, the score function for this submodel is given by
2(, Olog p(a, x U. r)
00
t(z, 0) + ra (1/ 0) (1 a) so so (,]2,,))
a- (1, è1 (1 0) + T[x,0)(1- - -(, 0))
r- (2,
#xx 0),
el (r, (1 ei (),
where
t(x, - = (z 0)/08
sa 0) = 0 0log fi (y|æ, 0//00 for a = 0,
()x,0) de, (z, 0)
#(z (I 0)/00
Thus, the tangent space is given
T= [1(x) ras (v/²) + (1 (y/k)
+ r(a - e1 (x)) (x( + x(x) b2))),
where sa(y(x) satisfies (Y| X s] = =
for = 0, 1, (xx) satisfies
([)( = ʃ f()) 0, (x) and (z( are arbitrary square-intergrable measurable function
of x. In addition, s. (y(x)) s. (vlæ, = 1) =ylx, 0) according to fa (ylæ) = (u122, 1) fyllzr 0).
Second, we calculate the pathwise derivative of r. Under the above parametric submodel, the target estimand = r()) can be written as
7 (0) (1) Y()) R 1]
(1( (()) R R=
EE|RRY()) (1)(RYY))-RYI RY (0)]
Ps(R=1)
[ (ylo, fz, d)dydx
[f O)yfo(ylz, [ S (x, 0)f(2, 0)dr
(z, dyddd
fx fz, O)dr
22

---

By calculation, the pathwise derivative of (6)) at 0 = 0g is given as
Es x()) (Y) i1 (Y (X)))
θT(0)
08
10=8p
q
Es (X)) Es(Y(0) so (o
Es (x))((x)+ + +x() { (mi(x))moox) mx) - -7}
9
Third, we construct the efficient influence function o. Let
(()) RA(Y mi (X)) (1 A) -mo
R
+ [m(X) mo(X) -T)
(x(
(X)
q
where es(X) = Ps(A = 1x) = Ps(A = 1|x, R = () = 1|x
+ Ps(A = 1|
0|X) = OR = X) = ei -xe((x)))ess(( = Ps
(x)))(())=(x)--- (1 ((1=x()))))
Fourth, we verify that an influence function of r. This holds if satisfies the following equation
θт(0)
Eslø ( X, Y, R2000),
(A.1)
00 88=0
Next, we give a detailed proof of (A.I).
Eslø s( A, X, Y, R; = H1 + H12 + H33
where
(X) RA(Y- - mx))))
H1 = Es
s(A, (A,,X,YRR:OO)
9.
es(X)
T (X) RA(Y - mm
EE
xx(
T(X) RAIY - mx(x)
cs(X) (1(())) x
Es
Es
9
xX)²
(X) - m1 (X)] (((x/))))/XX--111
Es
Es
q
es(X)
T
Es ((())-m((x)) ( x}]
Es
q
Es
(X)
(1 si (X())
the first term of θT(0)
09 10=80"
(X) (1 A) mm
H₂ Es
s(A, X,A, R:
ex(x)
(X) (1 A) mo
Es
(x)
1- (X(
Es
(X) ex(X)
q ex(() Es ((())
So(Y |
Es (XX) Es so (O
the second term of θT(
00 1=000
23

---

and
R
(X)(X) -mo(X) - -73} x x(A,,),
H3 Es
R
A A-
x(X) (10]1117(x] (X)) ((x) xx)))x(x)) }
R - T(X)
Es
(m1 mo( 7} ((X) + +
(x) (1](1eee))) (x)
9
R
A (x)
a - xX)) #(x)) }
1 - x(X)
Es
{m mx(x) 7} x(X)
e1(x) ex 61(X)
9
T (X)
{m (X) mx) - =3} x [(x)) x(x)
,(x) H(x) }
1
E
q
Es (X)((X + #(X) fmx mx
q
the third term of θr(0)
00
1==0p
Thus, equation A. holds.
Finally, we show that 6 is efficient influence function by verifying & E T. Let
((X) =
{m (X) (X)-mo
(X) - }}
x(x) (x-m;(x))
s1(1[x)=
xx(x)
(8 (V-mg(X))
1 1-es(X)
mi(X)
mo (X) -
b2(X)=
1 q
then @ can be written as
Φ = - (1-))))))())))))))))()))()) (R (X)))62(X).
0 for 0, and (z)= which implies that á T. and thus o is the efficient influence function of r.
Clearly,
Lemma Under Assumptions and ( (2) m, (x)112 op(n-1/2 1/2) and ||*(x) (<x) lll [[m m. (x)
or(n for all X and a E {0, 1}. then is satisfies VNz Ns(fs - -) 4 (O, where o² is the semiparametric efficiency bound of r, and 4 means convergence in distribution.
Proof of Lemma 2. Let Z (X, A, R1 Y), and denote
x =X)
RA mx)) (1 A)(Y - mo (X))
o(Z; mo, m1,
q
ex(X)
1 1- ex)s
+ [mi (X) mo(X))
as the non-centralized efficient influence functions т. where mo, mi, 0e are the nuisance parameters mz)x), my (x), e1) (x). respectively. We introduce them for ease of presentation.
Then can be written as
1
Σ iEELS
1 (Z; mi, R, ē1]]
NE + No.
24

---

where (mo, m1, ft é1) are estimates of (mu, M1, n. (1).
Let Ne + Ns. we decompose 7
f - ---- = Unn + U2n,
where
Unn
Σ
(Z; No: No. (1, - 7),
71
JEEUS
U2n
Σ iEEUS
Lo(Z; No: P1, f,, - -0(Zi Mo: M1.
Note that UI, a sum independent variables with zero means, and its variance equals V*/n, where V is the semiparametric efficiency bound. By the central limit theorem, we have
In 4 V*),
where S denotes convergence in distribution. Thus, it suffices to show that U2n (1(21112)) Unn can be be further decomposed as
U2n
U2n Es [U2n] Es [U2n].
By Taylor expansion for [[)2n] yields that
Es(U2a| Es 1 ma, mi # E1 o(Z mo. m1, n, x,6) ma, mi, T,))]
0 mo m n.C. E. (0) ma,
277 m,)],
02
2 Mio mo..
The first-order term
pio mi mi. f -- Bslø Z; mo. m1, x, e1)1
R T(X)(1 (x)(1-A) A)
Es
(mo)(X mo(X))
1- exs(X
9
R - T(X)RA RA
Es
((12(x))m1())))
es(X)
RARY- - mi (X)3
(1 A)LY - mo())
(t(x)=x(x))
es(X)
1 eX()
T (X)
RA(Y - (x))
(1 A)LY mo(X}}
Es
(x)( - (x))
es(X)²
{1 cx(
T(X)
RA(Y - mx())}
(1 - A)ff mo(X}}
Es
(X)
9
es(X)²
{1 ex())
0.
where the equation follows from Es[A|X] ex(X), Es(x(X)A x] = =X))e(XX). E[RA(Y - (X)))x = 0, and E[(1 A)(Y mo(X)) =
25

---

For the second-order term, we get
0
mo.01 m1. -.
-o mo. m,,,,
1
Es
es(x) (mo - -oo - - -x())
1 xXX)(1 A)
+ Es
19 [ c -mo
xxx))
[1 T A)
+ Es
1 ex(()] (X)(mo(())
mo 10000mmm0()0))))) (x)
- ei()))
1 RA
(xs( (mn (X) m1 (X)) - x(x1)
Es
(X)RR
Es
es(X)² B1 -
mi
=(X))
((X)RA x cs(X)2
(X) - -m00mmm1(XX()) (x))(()) ex))
+ Es
(1 A)
+ Es
(f(X) (x)) - mox)))
ex())
(X) (f(X) (X)) -xX))(((()))m((
RA
Es
1 (1 (1-A)
+ Es
- ((())(()))))))) -x)
9 - {s()
1 RA
(x) e1 (X) -X)(()))==(x)))( -m (X (x()))
+ Es
[
A)
[ (x) (X) (xx))) - mo(x))
+ Es
(X) A
+ +Ks
es(X)² (X)(E)( el )
- mx
=Op
(l)(xx) - ei (x)]] (1) (X) m. m1(X)ll + 11 - mo mo(X)ll)
+ +|(+(X) (x)]l2 (X) - mi (x) + | - mx(
= =o((n11/2)
All higher-order terms can shown to be dominated by the second-order term. Therefore, Es[U2n] or(n- -1/2). In addition, we get that U2. Es[U2n] = op(n -1/2) by calculating Varl (T(2n Es(U2.1)) oz) This proves the conclusion.
Theorem (Bias- Variance Analysis). Under [ only, if [x) ei (2) 1| m.(x)) ( o1(2) and [x) [x))]12 II (x) m,, (x)]l2 on for all E X and E {0, 1). then is satisfies /Ne + Ns(fs bias(fs)) 4 o(0, where 0² Var(ø),
is defined in Proposition and bias(fs) EEs 1 (mo(X)) [X)]]]]
26

---

Proof of Theorem 1. By the proof of Proposition 2, if [z) e1 (z)]l2 (Ima - ma(x)ll2
0p(n 0/2) and [x(()) - (2))22 (x) ma (x) 22 = or (1/2), we have
(Xi)
RAA.(Y; - mX.())) (1 A.)(Y, -
is No Ns
Σ
(x();)
1- (x)s(X,)
iESUS
E
Rí
{m (X;) ma (X:)) + or(n (1/2)
No Ns
9
VEEUS
Note that by definition, (x mi (X) but (X() mo(X). Then, under Assumption !
R (u)(X) mmo())
R
R (X) - -x)-
E
{m (X) -mo(X))
LEs
q
q
The bias of is is
(X,)
R,A,(Y, (x,)) (1-
RS (mi(X.) -mu(X;)] -
Es
+
ex(X,)
1 e=(
g
R
R
EEs
[mm (X) mo mx({)]
IEs
{ma {mm(X) (X) Ho(x))
q
q
R
[mo(X) -mo(x))
EE
q
Therefore,
v Ne + Ns - bias(fs))
Σ T(X))
RY, mi()))
(1 - A)), mo (X.)))
VNe + Ns iESUS
q
es(X,)
cX())
Σ
Ra {ml (X;) mX()))]
Esl a
R {mi (X.) - mo((i)))
VN N iEELS
(l))
This implies the conclusion by central limit theorem.
27

---

B Additional Experimental Results
B.1 Sensitivity Analysis on Control-n
To further demonstrate the robustness of the proposed approach îr We consider different control group sizes in RCT, in the simulation study, consider control group sizes of N {70, 80, 90) for "Linear" and "Nonlinear", while for the real-data application, we examine a range of N {70, 75, 85). The results is shown in the Figure Al A2 and A3
6.0250
0.040
6.721
64221
0.085
Tune
68203
0.022
6.0150
0.000
60125
c0125
60303
0.030
0.0035
200
Top.
Top.
Linear, Control-n 70
(h) Linear, Control-n 80
Linear, Control-n 90
Figure A1: Comparison of the performance of different approaches in "Linear" at different N
0.225
0.125
0.200
0.70
0.175
6.375
0.150
0.150
0.855
0.500
0.050
150
200
160
150 Top.k
500
600
300
D0C
150
200
300
100
500
600
30C
M65
180
200
300
4013
500
600
70C
R05
Top.k
10p_
(a) Nonlinear, Control-n 70
(b) Nonlinear, Control-n = 80
(c) Nonlinear, Control-n = 90
Figure A2: Comparison of the performance of different approaches in "Nonlinear" different No.
Suace
5,0000
Tuure
Tasi
0.501
0.8033
0.0000
6.005
0.0005
6.0044
0.0004
0.0004
Top.
Top.
kk..k
(a) NSW & PSID, Control-n = 70
(b) NSW & PSID, Control-n = 75
(c) NSW & PSID, Control-n = 85
Figure A3: Comparison of the performance of different approaches real-data at different No.
B.2 Sensitivity Analysis on Covariate Shift
To further demonstrate the robustness the proposed approach बr. We consider different degrees of covariate shift μ2 {0.2, 0.3, 0.4) in "Nonlinear" shown in Figure A4
28

---

6.30
0.0
0.25
6.25
0.15
0.20
6.35
0.15
6.0
6.30
0.5
6.05
0.05
100
700
160
180
750
4000
100
600
(a) Nonlinear, f12 0.2
(b) Nonlincar, Pl2 0.3
(c) Nonlinear, f 0.4
Figure A4: Comparison of the performance of different approaches in "Nonlinear" at different
C Implementation Details
For real-world data, we train the outcome model of the RCT data or external control through Multi- Layer Perceptron (MLP). In the estimation phase, the outcome regression Poo f41 and ma, Ao modeled using a Multi-Layer Perceptron (MLP), while both the propensity score and selection score estimated via logistic regression. As shown in Table AT presents the hyperparameter space for outcome regression joo jú, and ma, Ão, in both simulated and real-world datasets.
Table A1: Implementation Details
Hyperparameter
Ao
mo
Pu,o
Leaming
0.0005
0.0005
0.0005
0.0005
Batch size
32
32
32
32
Architecture
hidden layers [16]
2 hidden layers [16.8]
hidden layers [16.8)
hidden layers [16.8]
Optimizer
Adam
Adam
Adam
Adam
Early stopping patience
20
20
20
20
Activation function (all layers)
ReLU
ReLU
ReLU
ReLU
29


Reviews:
[{"review_id": "eHw6TSHKID", "reviewer": "Reviewer_NK6K", "summary": "This paper proposes an algorithm that borrows information from external controls using an influence-based criterion to improve the estimation of treatment effects. Some theoretical results are developed for the proposed algorithm, and its numerical performance is evaluated through a synthetic example and a real-data application. Overall, the paper is well written and the presentation is clear.", "questions": "1. Assumption 3 appears to be very strong. It essentially assumes that the selection procedure can identify unbiased external controls with probability tending to one. However, the paper does not demonstrate that the sample-borrowing procedure described in Section 5 guarantees this assumption in practice. Given this, the resulting theoretical results in Section 6.1 seem to reduce to those for a standard AIPW estimator with an extended control set, which limits their novelty and practical value.\n\n2. Although Assumption 3 is relaxed in Theorem 1 (Section 6.2), the theory still treats the candidate set $\\mathcal{S}$ as deterministic. In reality, the selection process for $\\mathcal{S}$ is stochastic. Ignoring this stochasticity may lead to an inaccurate or overly optimistic theoretical characterization of the estimator’s properties.\n\n3. How was the propensity score estimated in the numerical experiments? Introducing external controls could potentially complicate the estimation of the propensity score, which may in turn increase the variability of the proposed borrowing algorithm. Some discussion or diagnostic results would help clarify this point.\n\n4. In Figures 2 and 3, why not compare the proposed algorithm to the standard AIPW estimator that does not use external controls? This would provide a clearer benchmark to evaluate the actual benefit of borrowing external information.\n\n5. In Table 3, why is there no comparison of mean squared error (MSE) across different estimators? In several cases, it appears that the proposed estimator performs no better — or even worse — than the regular AIPW estimator. \n\n6. In the experiments, there are 300 treatment units and 100 controls, so it is reasonable to expect that borrowing additional external controls may be beneficial. However, how would the method perform if the number of controls already exceeds the number of treatment units — for example, 100 treatment units versus 300 controls? Would you still expect any improvement in the estimation of the treatment effect? It would be useful to demonstrate, either theoretically or numerically, how the method performs in this setting. Understanding such limitations is important for practical applications.", "limitations": "Yes.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength: the paper is well-written and the presentation is clear.\nWeakness: the theory part seems a bit weak.", "was_revised": true, "final_justification": "I have read all reviews/response and would like to keep my score as is.", "rebuttal": "We thank you for your efforts in evaluating our work. Below, we hope to address your concerns and questions.\n\n> **Q1:** Assumption 3 appears to be very strong. However, the paper does not demonstrate that the sample-borrowing procedure described in Section 5 guarantees this assumption in practice. Given this, the resulting theoretical results in Section 6.1 seem to reduce to those for a standard AIPW estimator, which limits their novelty.\n\n**Response to Q1:**  Thank you for your comments. **We would like to kindly remind the reviewer that there may have been a misunderstanding regarding our method.** \n\n**First, Assumption 3 is plausible when $\\mathcal{S}$ is appropriately selected by the influence function.** As shown in Figure 1(b), the controls in RCT (blue points) and in $\\mathcal{S}$ (red points) are close, and follow a similar distribution, and thus Assumption 3 is plausible.  \n\n**Second and most importantly, the proposed method does not rely on Assumption 3.** Specifically,\n- The proposed method is mainly based on Theorem 1, where we derive the estimator’s bias and variance without Assumption 3 (i.e., under Assumption 1 only);  \n- From Theorem 1, we could find the appropriate/optimal subset $\\mathcal{S}$ by minimizing the MSE of the estimator $\\hat\\tau_{\\mathcal{S}}$.\n\n**Third, since we select the $K$ samples with small influence scores to form the candidate subset, and we determine the $K$ by minimizing MSE. This design is subtle and guarantees that the selected $K$ samples in external controls and the RCT's controls are close, rendering Assumption 3 holds or close to holds or only slightly violated.** We could intuitively understand it from the following two aspects:  \n- By the definition of influence score, when a sample in external controls is far from the controls in RCT, the influence score tends to be large.    \n-  As $K$ increases, if several incomparable samples (the samples are not close to the controls in RCT) appear, then these incomparable samples will yield a bias in the final estimator $\\hat\\tau_{\\mathcal{S}}$, and increase the MSE. \nFrom the above discussion, our proposed method ensures that Assumption 3  is close to holding. This is why we begin our discussion with Assumption 3 in Section 6.1. \n\n**Fourth, for the novelty of Section 6.1 of this article, we would like to clarify that Section 6.1 is not the main contribution of this article.** Section 6.1 is just the beginning of our method. In lines 61--68 (the end of Introduction) of this paper, we only claimed the following main contributions:\n- We reveal the limitations of existing approaches for estimating treatment effects by combining RCT data with external controls. \n- We propose an influence-based sample borrowing approach, which can effectively quantify the comparability of each sample in the external controls.\n-  We develop a data-driven approach to select the optimal subset of external control samples based on the MSE of the proposed estimator. \n- We conduct extensive experiments on both simulated and real-world datasets, demonstrating that the proposed approach outperforms the existing baseline approaches.\n\n**We hope that the above clarification addresses your concerns and sincerely invite you to kindly re-evaluate our work in light of our explanation.**\n\n  \n> **Q2:** Although Assumption 3 is relaxed in Theorem 1 (Section 6.2), the theory still treats the candidate set  $S$ as deterministic. In reality, the selection process for $S$  is stochastic.\n\n**Response to Q2:** Thank you for your insightful comments. We acknowledge that the selection stochasticity could theoretically influence the results. Nevertheless, we would like to clarify that our empirical findings consistently show that the proposed method significantly outperforms existing approaches across various settings. We agree that this theoretical aspect warrants further investigation, particularly regarding the uncertainty introduced by the selection process. Accordingly, we added a discussion on selection stochasticity as a limitation of our work. \n \n> **Q3:** How was the propensity score estimated in the numerical experiments? Introducing external controls could potentially complicate the estimation of the propensity score.\n\n**Response to Q3:** Thanks for your comments, and we apologize for the lack of details. By the definition of $e_{\\mathcal{S}}(X) = \\mathbb{P}_{\\mathcal{S}}(A=1|X)$, in our experiments, we use logistic regression to estimate it based $\\mathcal{E}\\cup \\mathcal{S}$. Please see our codes in the Supplementary Material. \n\n > **Q4:** In Figures 2 and 3, why not compare the proposed algorithm to the standard AIPW estimator that does not use external controls? \n\n**Response to Q4:** Thanks for your comments. In Table 3, we compared the proposed algorithm to the standard AIPW estimator in terms of |bias|  and S.E. (standard error). From Table 3, we can see that the standard error of the standard AIPW estimator is larger than all the methods using external controls. \n**Following your suggestions, we calculate the MSE of the standard AIPW estimator (i.e., $\\text{S.E.}^2$ due to zero bias) and compare it with our method, the associated results are given below.** \n\n|Linear|Top-K=300|Top-K=350|Top-K=400|\n|--|--|--|--|\n|$\\hat{\\tau}_{aipw}$|0.0115|0.0115|0.0115|\n|$\\hat{\\tau}_{if}$|**0.0071**|**0.0072**|**0.0066**|\n|Nonlinear|Top-K=50|Top-K=100|Top-K=150|\n|$\\hat{\\tau}_{aipw}$|0.0498|0.0498| 0.0498|\n|$\\hat{\\tau}_{if}$| **0.0458**|**0.0477**| **0.0487**|\n|NSW \\& PSID|Top-K=30|Top-K=35|Top-K=45|\n|$\\hat{\\tau}_{aipw}$| 0.000181|0.000181|0.000181|\n|$\\hat{\\tau}_{if}$|**0.000174**|**0.000171**| **0.000174**|\n\nFrom this table, we could see that the MSE of the standard AIPW estimator is significantly larger than that of the proposed methods. \nIn Figures 2 and 3, we don't compare the  standard AIPW estimator in terms of MSE for the following two reasons:\n- Our goal is to compare various competing methods for sample selection, while the AIPW estimator does not conduct sample selection. \n- Visual clarity: For optimal graphical clarity, we selected three representative data-borrowing methods. And it will better highlight our method's performance trends. \n\n\n > **Q5:** In Table 3, why is there no comparison of mean squared error (MSE) across different estimators? In several cases, it appears that the proposed estimator performs no better — or even worse — than the regular AIPW estimator. \n\n**Response to Q5:**  Thanks for your comments. \n**Here are the reasons why we don't  include MSE across different estimators:**\n- Limited space for presentation. \n- As the MSE between different estimates is already presented in Figure 3, it is not necessary to repeat it in Table 3. \n\n**For the comment that \"in several cases, it appears that the proposed estimator performs no better than the regular AIPW estimator.\" We would like to kindly remind the reviewer that there may have been a misunderstanding about Figure 3. Below, we provide a detailed explanation.**  \n- Figure 3 illustrates the trend of the MSE of $\\hat\\tau_{S}$  across a wide range of $K$ (corresponding to different $S$), rather than showing only the optimal $K$. Thus, **we do not require $\\hat{\\tau}_{S}$ to perform better for all values of $K$.** \n- From Figure 3, we can see that a better estimator can be obtained when $K$ is appropriately selected. This indicates that our method has great potential to improve upon previous approaches. \n\n> **Q6:** In the experiments, there are 300 treatment units and 100 controls, so it is reasonable to expect that borrowing additional external controls may be beneficial. However, how would the method perform if the number of controls already exceeds the number of treatment units — for example, 100 treatment units versus 300 controls? \n\n**Response to Q6:** Thanks for your comments. **First, we would like to clarify that the ratio between treatment and control (i.e., $N_t:N_c$) is not the most critical factor. Below, we provide an intuitive explanation of this.** \n- We focus on settings where the overall RCT data sample is relatively small (the most common cases in applications); this means that both the treatment and control groups in the RCT have smaller sample sizes relative to external control data.\n- In this setting, the ratio between treatment and control (i.e., $N_t:N_c$) is not the most critical factor. Logically speaking, the most critical factor is the number of comparable samples available in the external controls, compared with $N_c$. \n\n**Second, we conducted additional experiments to verify this assertion.** We explored different ratios $N_t:N_c$ and different numbers of external data $N_\\mathcal{O}$, where the data-generating mechanisms remain the same in the manuscript. The experimental results are shown in the table below, where $K$ is the optimal value corresponding to the minimum MSE. The experimental results show that the method we proposed is consistently optimal. \n \n|$N_{\\mathcal{O}}=800$| | | |\n|--|--|--|--|\n|MSE(optimal-K=400)|$N_t:N_c=3:1$|$N_t:N_c=1:1$|$N_t:N_c=1:3$|\n|$\\hat{\\tau}_{\\textup{aipw}}$|0.0115|0.0122|0.0161|\n|$\\hat{\\tau}_{\\textup{full}}$|0.0173|0.0205|0.0190|\n|$\\hat{\\tau}_{\\textup{lasso}}$|0.0126|0.0123|0.0200|\n|$\\hat{\\tau}_{\\textup{if}}$|**0.0066**| **0.0089**|**0.0131** |\n| | | | |\n| $N_{\\mathcal{O}}=2000$| | | |\n|MSE(optimal-K=1000)|$N_t:N_c=3:1$| $N_t:N_c=1:1$ | $N_t:N_c=1:3$|\n|$\\hat{\\tau}_{\\textup{aipw}}$|0.0275|0.0122| 0.0162|\n|$\\hat{\\tau}_{\\textup{full}}$|0.0206|0.0135| 0.0111|\n|$\\hat{\\tau}_{\\textup{lasso}}$|0.0492|0.0147| 0.0235|\n|$\\hat{\\tau}_{\\textup{if}}$| **0.0104**|**0.0082**|**0.0073**|\n| | | | |\n|$N_{\\mathcal{O}}=4000$| | | |\n|MSE(optimal-K=2000)|$N_t:N_c=3:1$|$N_t:N_c=1:1$|$N_t:N_c=1:3$|\n|$\\hat{\\tau}_{\\textup{aipw}}$ |0.0275|0.0190|0.0162|\n|$\\hat{\\tau}_{\\textup{full}}$ |0.0368|0.0288|0.0238|\n|$\\hat{\\tau}_{\\textup{lasso}}$|0.0192|0.0134|0.0101|\n|$\\hat{\\tau}_{\\textup{if}}$|**0.0073**|**0.0050**|**0.0042**|"}, {"review_id": "1QUf6qOxT7", "reviewer": "Reviewer_ctUy", "summary": "This paper introduces an influence-function-based framework for adaptively borrowing external control samples to improve treatment effect estimation in RCTs, particularly when sample sizes are small. The method assigns influence scores to external samples based on their perturbation to the RCT control model, allowing selection of the most comparable data without assuming full exchangeability. A semiparametric efficient estimator is then developed, and a data-driven approach selects the optimal subset of external controls by minimizing mean squared error. Through both simulation and real-world experiments, their proposed method demonstrates superior accuracy and robustness compared to existing approaches including adaptive lasso-based borrowing.", "questions": "it would be helpful to clarify 1) the influencial validity after adaptive selection; 2) how are the influence score estimation implemented in practice especially in high dimensional settings; 3) provide justification for the approximation of bias as the difference from the AIPW estimator, which tends to have high variability in small samples and hence unreliable.", "limitations": "There is a notable limitation in their inference - although the author acknowledges a bias-variance-trade-off due to the selection of the comparable subset, ignoring this selection or simply minimizing it may not adequately address the post-selection-inference issue. There is no formal accounting for the variability introduced by the selection part. The theoretical results are derived as if the subset were fixed. The paper sidesteps this with asymptotics and MSE estimation, a rigorous treatment would require additional methods or caveats.  \n\nthe simulation settings are overly simplistic - in most RCTs, we observe complex covariates including counts, skewed distributions. It would also be important to consider heterogeneous treatment effects. The authors should compare to other methods such as AIPW, causal transfer or domain adaptation approaches. Authors should consider more modern real datasets such as UK Biobank or MIMIC.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The overall quality of the work is high. The authors provide both theoretical analysis and empirical results, establishing consistency, asymptotic normality, and semiparametric efficiency of their proposed estimator. They extend their results to accommodate violations of the exchangeability assumption, and use a bias-variance trade-off to drive a data-adaptive selection of borrowed samples. The experimental results are thorough, and the method demonstrates robustness across different covariate shifts and RCT sample sizes. One limitation is the computational burden associated with calculating influence scores, particularly for high-dimensional models requiring second-order derivatives. While this is acknowledged, a deeper exploration of scalability and approximate alternatives would strengthen the practical applicability of the method.\n\nThe clarity of the manuscript is generally strong. The structure is logical, with clear motivation, definitions, and progression through theoretical development and empirical evaluation. Mathematical notation is precise, though occasionally dense, and might benefit from simplified exposition in a few sections. Visuals such as borrowing diagrams and performance plots are well-designed and help convey the main results effectively. Implementation details and reproducibility criteria are addressed in full, with code and data availability confirmed.\n\nThe significance of the work is reasonably high. The method targets a well-known limitation in modern trial design and real-world evidence generation—the inability to use external data when the exchangeability assumption is questionable. By moving beyond global assumptions to local, sample-level borrowing decisions, the authors propose a strategy with clear practical implications for trials in rare diseases, oncology, and digital experimentation, where small sample sizes are common. However, the application remains focused on a single classical dataset (NSW/PSID). Demonstration of the method in more contemporary, high-dimensional settings such as electronic health records or biobanks would further enhance its relevance.\n\nThe originality of the paper is also fine. The use of influence functions to guide data borrowing in causal inference settings is a novel contribution that differentiates this work from prior lasso-based and bias-penalized approaches. The estimator unifies ideas from semiparametric efficiency theory, causal inference, and robust learning, and contributes new tools to an important methodological frontier. The method has the potential to inspire further work in areas such as transfer learning for treatment effect estimation or adaptive trial design. \n\nIn summary, this paper makes some contribution to the literature on treatment effect estimation and external data integration. It presents a theoretically grounded, empirically validated, and practically relevant methodology.", "was_revised": true, "final_justification": "The authors answered most of my questions but I believe that my original score reflects my overall assessment.", "rebuttal": "Thank you very much for your positive evaluation of our paper. Below, we hope that our clarification addresses your concerns. \n\n> **W1:** One limitation is the computational burden associated with calculating influence scores, particularly for high-dimensional models requiring second-order derivatives. While this is acknowledged, a deeper exploration of scalability and approximate alternatives would strengthen the practical applicability of the method.\n\n**Response to W1:** We sincerely appreciate your insightful comments regarding the computational challenges associated with influence score calculation in high-dimensional settings. We acknowledge this limitation and would like to elaborate on both the challenges and potential solutions:\n\n* **Computational Challenges in High-Dimensional Models:** \nThe calculation of influence functions indeed faces significant computational and memory burdens when dealing with high-dimensional models, particularly due to the need for second-order derivative computations. As noted by the reviewer, these computational costs scale quadratically with parameter dimensionality, becoming especially prohibitive for large-scale models such as deep neural networks.\n\n* **Scalability Solutions and Approximation Methods:**\nTo address these challenges, the research community has developed several efficient approximation approaches, including:\n  - Iterative methods: Conjugate Gradients (CG) for Hessian-vector products [1]\n  - Stochastic estimation techniques [2,3]\n  - Linear-time Stochastic Second-Order Algorithm (LISSA) [4]\n  - Kronecker-Factored Approximate Curvature (K-FAC) for efficient curvature approximation[5]\n  - Diagonal Hessian approximation [6]\n\nThese methods can significantly reduce the computational overhead while maintaining reasonable accuracy in influence function estimation. In future work, we plan to implement and evaluate these approximation techniques to improve the scalability of our method for large-scale models. \n\n\n> **W2:** However, the application remains focused on a single classical dataset (NSW/PSID). Demonstration of the method in more contemporary, high-dimensional settings such as electronic health records or biobanks would further enhance its relevance.\n\n**Response to W2:** Thanks for your helpful suggestions. The influence function method has demonstrated its applicability to high-dimensional datasets empirically, such as images. While our work focuses on different objectives, a notable example is Pang et al. (2017)[3], who investigated the role of influence functions in interpreting black-box models on the MNIST dataset and other image datasets. We will explore high-dimensional settings in future work. Thanks again.  \n\n> **W3:**  There is a notable limitation in their inference - although the author acknowledges a bias-variance-trade-off due to the selection of the comparable subset, ignoring this selection or simply minimizing it may not adequately address the post-selection-inference issue. There is no formal accounting for the variability introduced by the selection part. The theoretical results are derived as if the subset were fixed. The paper sidesteps this with asymptotics and MSE estimation;  a rigorous treatment would require additional methods or caveats.\n\n**Response to W3:** Thank you for your insightful comments and helpful suggestions. We acknowledge that the post-selection process could theoretically influence the results. Accordingly, we will add a discussion on post-selection as a limitation of our work. \n\n> **W4:** The authors should compare to other methods such as AIPW, causal transfer or domain adaptation approaches. Authors should consider more modern real datasets, such as the UK Biobank or MIMIC.\n\n**Response to W4:** Thank you for your helpful suggestions. We would like to kindly remind the reviewer that we have included AIPW as one of the baselines in our experiments. AIPW (based only on the RCT data) is the standard baseline (Lines 286-287), which serves as an asymptotically unbiased benchmark for $\\tau$ (line 107). In addition, we also included the semiparametric efficient causal transfer method (i.e., $\\hat \\tau_{\\text{full}}$) as the baseline, which relies on the exchangeability assumption (Assumption 2). \n\n**Following your suggestions, we conducted additional experiments using the MIMIC and eICU datasets**, as outlined below. \n- **Data description:**\nThe MIMIC dataset [7] consists of $N_\\mathcal{E}=2124$ samples with $N_t=902$ in the treated group and $N_c=1222$ in the control group. This dataset contains a treatment indicator (i.e., vasopressor treatment indicator, A: 1 for treated, 0 otherwise), covariates ( age, gender, weight, temperature, glucose, creatinine, bun, wbc, pco2, ph, chloride, potassium, sodium, spo2, heartrate1, and so on). We take the log of the cumulative balance as the interesting outcome $Y$. \nThe external dataset, eICU dataset [8], is used for external comparison with the MIMIC dataset, which includes the same structure as the MIMIC.  We only use the control group data (i.e., $A=0$) of a as the external controls ($N_{\\mathcal{O}}=2595$).\n- **Results:** Due to space limitations, we present the results near the optimal point in the table below (e.g., K=200, 300). We can see that our proposed method consistently outperforms other borrowing methods.\n\n| MIMIC \\& eICU ($10^{-3}$)     | Top-K=200 | Top-K=300  |\n|---------|-----------|------------|\n| $\\hat{\\tau}_{\\textup{aipw}}$  | 0.069     | 0.069      |\n| $\\hat{\\tau}_{\\textup{full}}$  | 1.221     | 1.221      |\n| $\\hat{\\tau}_{\\textup{lasso}}$ | 0.599     | 0.295      |\n| $\\hat{\\tau}_{\\textup{if}}$    | **0.062** | **0.063**  |\n\n\n\n> **Q:** It would be helpful to clarify\n\n>(1) the influential validity after adaptive selection;\n\n>(2) how is the influence score estimation implemented in practice, especially in high-dimensional settings;\n\n>(3) provide justification for the approximation of bias as the difference from the AIPW estimator, which tends to have high variability in small samples and is hence unreliable.\n\n**Response to Q(1):** Please see our response to W3;  \n\n**Response to Q(2):** Please see our response to W1;  \n\n**Response to Q(3):** Thanks for your insightful comments. Below, we show that $\\hat \\tau_{\\mathcal{S}} - \\hat \\tau_{\\text{aipw}}$ is a consistent estimator of the bias. \n\nUnder the conditions of Theorem 1,\\\n\\\\begin{align\\*}\\\n\\\\hat \\\\tau\\_{\\\\mathcal{S}} ={}& \\\\frac{1}{ N\\_{\\\\mathcal{E}} +\nN\\_{\\\\mathcal{S}} } \\\\sum\\_{i\\\\in \\\\mathcal{E}\\\\cup\\\\mathcal{S}}\n\\\\frac{\\\\pi(X_i)}{q} \\\\left \\[ \\\\frac{ R_iA_i(Y_i - m_1(X_i))}{\ne\\_{\\\\mathcal{S}}(X_i) } - \\\\frac{(1-A_i)(Y_i - m_0(X_i)) }{1 -\ne\\_{\\\\mathcal{S}}(X_i)} \\\\right \\] \\\\\\\\\\\n{}&+ \\\\frac{1}{ N\\_{\\\\mathcal{E}} + N\\_{\\\\mathcal{S}} } \\\\sum\\_{i\\\\in\n\\\\mathcal{E}\\\\cup\\\\mathcal{S}} \\\\frac{R_i}{q}\\\\{ m_1(X_i) -\nm_0(X_i)\\\\} + o\\_{P}(n\\^{-1/2}),\\\n\\\\end{align\\*}\\\nwhere \\$e\\_{\\\\mathcal{S}}(X) = e_1(X) \\\\pi(X)\\$. In addition, by a similar proof, let \\$q =\nN\\_{\\\\mathcal{E}}/(N\\_{\\\\mathcal{E}}+N\\_{\\\\mathcal{S}})\\$, we have\\\n\\\\begin{align\\*}\\\n\\\\hat \\\\tau\\_{\\\\textup{aipw}} ={}& \\\\frac{1}{N\\_{\\\\mathcal{E}}}\n\\\\sum\\_{i\\\\in \\\\mathcal{E}} \\\\left \\[ \\\\frac{ A_i\\\\{Y_i -\n\\\\mu_1(X_i)\\\\}}{e_1(X_i)} - \\\\frac{(1-A_i)\\\\{Y_i - \\\\mu_0(X_i)\\\\}}{1 -\ne_1(X_i)} + \\\\{ \\\\mu_1(X_i) - \\\\mu_0(X_i) \\\\} \\\\right \\] +\no\\_{P}(n\\^{-1/2}) \\\\\\\\\\\n={}& \\\\frac{1}{ N\\_{\\\\mathcal{E}} + N\\_{\\\\mathcal{S}} } \\\\sum\\_{i\\\\in\n\\\\mathcal{E}\\\\cup\\\\mathcal{S}} \\\\frac{R_i}{q} \\\\left \\[ \\\\frac{\nA_i(Y_i - \\\\mu_1(X_i))}{ e\\_{1}(X_i) } - \\\\frac{(1-A_i)(Y_i -\n\\\\mu_0(X_i)) }{1 - e\\_{1}(X_i)} \\\\right \\] \\\\\\\\\\\n{}&+ \\\\frac{1}{ N\\_{\\\\mathcal{E}} + N\\_{\\\\mathcal{S}} } \\\\sum\\_{i\\\\in\n\\\\mathcal{E}\\\\cup\\\\mathcal{S}} \\\\frac{R_i}{q}\\\\{ \\\\mu_1(X_i) -\n\\\\mu_0(X_i)\\\\} + o\\_{P}(n\\^{-1/2}).\\\n\\\\end{align\\*}\\\nSince \\$m_1(X) = \\\\mu_1(X)\\$,\\\n\\\\begin{align\\*}\\\n\\\\hat\\\\tau\\_{\\\\mathcal{S}} - \\\\hat \\\\tau\\_{\\\\text{aipw}}\\\n={}& \\\\frac{1}{ N\\_{\\\\mathcal{E}} + N\\_{\\\\mathcal{S}} } \\\\sum\\_{i\\\\in\n\\\\mathcal{E}\\\\cup\\\\mathcal{S}} \\\\left \\[ \\\\frac{R_i}{q}\n\\\\frac{(1-A_i)(Y_i - \\\\mu_0(X_i)) }{1 - e\\_{1}(X_i)} -\n\\\\frac{\\\\pi(X_i)}{q} \\\\frac{(1-A_i)(Y_i - m_0(X_i)) }{1 -\ne\\_{\\\\mathcal{S}}(X_i)} \\\\right \\] \\\\\\\\\\\n{}&+ \\\\frac{1}{ N\\_{\\\\mathcal{E}} + N\\_{\\\\mathcal{S}} } \\\\sum\\_{i\\\\in\n\\\\mathcal{E}\\\\cup\\\\mathcal{S}} \\\\frac{R_i}{q}\\\\{ \\\\mu_0(X_i) -\nm_0(X_i)\\\\} + o\\_{P}(n\\^{-1/2}) \\\\\\\\\\\n={}& A\\_{1n} + A\\_{2n} + o\\_{P}(n\\^{-1/2}),\\\n\\\\end{align\\*}\\\nWe can see that the first term \\$A\\_{1n}\\$ converges to zero at a rate\nof order \\$1/\\\\sqrt{N\\_{\\\\mathcal{E}} + N\\_{\\\\mathcal{S}}}\\$, and the second\nterm \\$A\\_{1n}\\$ converges to the bias \\$\\\\mathbb{E}[ R (\\\\mu_0(X) -\nm_0(X))/q]\\$. Thus, \\$ \\\\hat\\\\tau\\_{\\\\mathcal{S}} - \\\\hat\n\\\\tau\\_{\\\\text{aipw}}\\$ is a consistent estimator of the bias.\n\n---\n **References** \n\n[1] Martens J. Deep learning via hessian-free optimization[C]//Icml. 2010, 27: 735-742.\n\n[2] Agarwal N, Bullins B, Hazan E. Second-order stochastic optimization for machine learning in linear time[J]. Journal of Machine Learning Research, 2017, 18(116): 1-40.\n\n[3] Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pages 1885–1894. PMLR, 2017.\n\n[4] Agarwal N, Allen-Zhu Z, Bullins B, et al. Finding approximate local minima faster than gradient descent[C]//Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing. 2017: 1195-1199.\n\n[5] Grosse R, Martens J. A kronecker-factored approximate fisher matrix for convolution layers[C]//International Conference on Machine Learning. PMLR, 2016: 573-582.\n\n[6] Guo H, Rajani N F, Hase P, et al. Fastif: Scalable influence functions for efficient model interpretation and debugging[J]. arXiv preprint arXiv:2012.15781, 2020.\n\n[7] Singer et al (2016). The third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA, 315(8):801–810\n\n[8] Chu et al (2023). Targeted optimal treatment regime learning using summary statistics. Biometrika."}, {"review_id": "oDMeqMMTxD", "reviewer": "Reviewer_sU8Q", "summary": "This paper proposes an influence function-based adaptive sample borrowing method to enhance statistical efficiency in randomized controlled trials by incorporating external controls. The paper presents a data-driven selection strategy to optimally manage the bias-variance trade-off under potential exchangeability violation when identifying external control samples for borrowing.", "questions": "1. Section 5, line 181 to 186, how to decide the optimal $K$? What if there are ties among the influence scores? \n2. In practice, how to determine whether Assumption 3 is violated? Could the authors comment on this? \n3. Regarding the top-K selective borrowing approach, is the selected subset consistent? It would be helpful if the authors could provide further explanation or theoretical guarantees regarding the set selection consistency.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\nThe paper is clearly written and well-structured. It makes an effort to provide intuitions and examples to explain the motivation and the proposed methodology. \n\nWeakness:\n- Novelty: It appears that the proposed method is built upon the semiparametric efficiency results derived in Section 6.1. However, similar results can be found in Li et.al, 2023 (reference 9 cited in the paper). Could the authors provide more discussions on this connection? \n- Contributions: The top-k selective borrowing strategy and the relaxation of the exchangeability assumption appear to be the paper's primary contributions, but these points are not sufficiently highlighted. To clarify the selective borrowing component, it might be helpful to explicitly formulating the top-k selection as an objective function, perhaps under Eq. (5)? Regarding the relaxation of the exchangeability assumption, highlighting and elaborating further on lines 235 to 239 could strengthen the presentation of this contribution.\n- Theoretical results: It would be helpful to have some theoretical results (e.g., remarks) on the efficiency gain of the proposed methods.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your positive evaluation of our paper. Below, we hope that our clarification addresses your concerns.\n\n> **W1:** Novelty: It appears that the proposed method is built upon the semiparametric efficiency results derived in Section 6.1. However, similar results can be found in Li et.al, 2023 (reference 9 cited in the paper). Could the authors provide more discussion on this connection?\n\n**Response to W1:** Thank you for your insightful comments. We agree that both studies share common ground in the semiparametric efficiency theory framework. However, **we would like to clarify that our primary contributions are not in the estimation itself, but in developing a novel approach for selecting external comparable samples and determining the optimal selection set.** \n\n- In Section 4, we provided a comparison between our influence-based method and the lasso-based approach introduced in [1], as illustrated in Figure 1. The results from our experiments demonstrate that the influence-based method offers greater reliability in selecting external samples (see the Table below), which is the key motivation behind our work.  \n- In Sections 5--6, we developed the influence-based method and the optimal borrowed set selection method.\n\n|                                 | model-free for external controls | robust to outliers  |\n|---------------------------------|----------------------------------|---------------------|\n| the adaptive lasso-based method | ×                                | ×                   |\n| the influence-based method      | √                                | √                   |\n\n**In addition, in lines 61--68 (the end of Introduction) of the manuscript, we did not claim any contribution to semiparametric efficiency. We only claimed the following main contributions:** \n- We reveal the limitations of existing approaches for estimating treatment effects by combining RCT data with external controls. \n- We propose an influence-based sample borrowing approach, which can effectively quantify the comparability of each sample in the external controls.\n-  We develop a data-driven approach to select the optimal subset of external control samples based on the MSE of the proposed estimator. \n- We conduct extensive experiments on both simulated and real-world datasets, demonstrating that the proposed approach outperforms the existing baseline approaches.\n\nWe hope that the above clarification could address your concerns.\n\n\n> **W2:** Contributions: The top-k selective borrowing strategy and the relaxation of the exchangeability assumption appear to be the paper's primary contributions, but these points are not sufficiently highlighted. To clarify the selective borrowing component, it might be helpful to explicitly formulate the top-k selection as an objective function, perhaps under Eq. (5)? Regarding the relaxation of the exchangeability assumption, highlighting and elaborating further on lines 235 to 239 could strengthen the presentation of this contribution.\n\n**Response to W2:** Thank you for your comments. We appreciate your suggestions to emphasize the selective borrowing strategy and the relaxation of the exchangeability assumption. To this end, we can explicitly formulate the top-k selective borrowing strategy as an objective function as follows: \n\\$\\$S\\^\\*=\\\\arg \\\\min\\_{S_k \\\\in \\\\mathcal{C} } MSE (\\\\hat{\\\\tau}\\_{S_k}),\\$\\$\nwhere \\$\\\\mathcal{C} = \\\\{ C_k: k = 1, ..., N\\_{O}\\\\}\\$, $C_k$ denotes a subset of external controls corresponding to the $k$ smallest influence function values, $N_{O}$ is the sample size of external controls, and $\\mathcal{S}^*$ is the optimal candidate. Regarding the relaxation of the exchangeability assumption, we will revise the manuscript to further elaborate on it. Thanks again. \n\n\n> **W3:**  Theoretical results: It would be helpful to have some theoretical results (e.g., remarks) on the efficiency gain of the proposed methods.\n\n**Response to W3:** Thanks for your helpful suggestions. Under Assumption 1 and Assumption 2 (the exchangeability assumption), Li et al. (2023) [ref. 9 in the manuscript] showed that the estimator $\\hat \\tau_{S}$, where $S$  is the entire external control dataset,  has a smaller asymptotic variance than $\\hat \\tau_{\\text{aipw}}$. \n\nLikewise, **under Assumptions 1 and 3, by a similar proof, we can show that $\\hat \\tau_{S}$ (for $S$ satisfies Assumption 3),  has a smaller asymptotic variance than $\\hat \\tau_{\\text{aipw}}$. And the efficiency gain is given as**  \\\\begin{align\\*}\\ \\\\mathbb{E}\\_{\\\\mathcal{S}} \\\\left \\[ \\\\frac{ \\\\pi(X) }{q\\^2} \\\\frac{\\\\text{Var}(Y (0)\\|X)}{ 1 - e_1(X) } \\\\frac{ \\\\mathbb{P}\\_{\\\\mathcal{S}}(A=0, R=0\\|X) }{ \\\\mathbb{P}\\_{\\\\mathcal{S}}(A=0\\|X) } \\\\right \\],\\ \\\\end{align\\*} where \\$\\\\mathbb{P}\\_{\\\\mathcal{S}}(A=0, R=0\\|X) = \\\\mathbb{P}\\_{\\\\mathcal{S}}(R=0\\|X) \\\\mathbb{P}\\_{\\\\mathcal{S}}(A=0\\|R=0,X) = 1-\\pi(X)\\$. \n\n**However, when only Assumption 1 holds--allowing for arbitrary differences between the RCT and external controls--$\\hat \\tau_{S}$ may be biased, and any efficiency gain is not guaranteed. This is exactly the setting we explored.**\n\n\n> **Q1:** Section 5, lines 181 to 186, how to decide the optimal $K$? What if there are ties among the influence scores?\n\n**Response to Q1:** Thanks for your comments, and we apologize for the lack of clarity. **Below, we provide a detailed description of selecting the \"optimal\" borrowed set. It consists of two steps:** \n- **Step 1:**  Based on the ranking of influence scores, we construct a series of nested subsets of external controls, where each subset comprises the top-K most comparable samples (i.e., the top-K samples with the smallest influence scores), see lines 183--185 of the manuscript.  \n- **Step 2:** Find the optimal $K$ that minimizes the MSE based on Theorem 1, see Section 6.2 of the manuscript.\nFrom Figure 2 of the manuscript, we can see that the MSE curve will show a trend of first decreasing and then increasing. Then the lowest point of the curve corresponds to the optimal K with the smallest empirical MSE.  \n\n**In addition, our method also works if there are ties among the influence scores.** In this case, there will be multiple optimal $K$, but they all have the same MSE. This does not affect the associated numerical results.   \n\n\n> **Q2:** In practice, how to determine whether Assumption 3 is violated? Could the authors comment on this?\n\n**Response to Q2:** Thank you for raising this interesting question. **We would like to clarify that the proposed method does not rely on Assumption 3. Specifically,**\n- The proposed method is mainly based on Theorem 1, where we derive the estimator’s bias and variance without Assumption 3 (i.e., under Assumption 1 only);  \n- From Theorem 1, we could find the appropriate subset $\\mathcal{S}$ by minimizing the MSE, and then construct the estimator $\\hat\\tau_{\\mathcal{S}}$ based on it.  \n\n**In addition, Assumption 3 is plausible when $\\mathcal{S}$ is appropriately selected by the influence function.** As shown in Figure 1(b), the controls in RCT (blue points) and in $\\mathcal{S}$ (red points) are close, and follow a similar distribution, and thus Assumption 3 is plausible.  \n\n**Furthermore, since we select the $K$ samples with small influence scores, we determine the $K$ by minimizing MSE. This design is subtle and guarantees that the selected $K$ samples in external controls and the RCT's controls are close, rendering Assumption 3 close to holding or only slightly violated.** We could intuitively understand it from the following two aspects:  \n- By the definition of influence score, when a sample in external controls is far from the controls in RCT, the influence score tends to be large.    \n-  As $K$ increases, if several incomparable samples (the samples are not close to the controls in RCT) appear, then these incomparable samples will yield a bias in the final estimator $\\hat\\tau_{\\mathcal{S}}$, and increase the MSE. \n\n\n> **Q3:**  Regarding the top-K selective borrowing approach, is the selected subset consistent?\n\n**Response to Q3:** Thank you for your insightful comments. We will further explore the issue of selection consistency. Additionally, we would like to clarify that our empirical results show that the proposed method significantly outperforms existing approaches. We agree that the theoretical aspect of selection consistency merits further investigation, and we will include a discussion on this point in the revised manuscript."}, {"review_id": "IbrPM8GLNp", "reviewer": "Reviewer_mm3d", "summary": "This paper proposes a method to improve treatment effect estimation in RCTs by adaptively borrowing external control samples without relying on the exchangeability assumption. The key innovation is an influence-function-based approach that quantifies each external sample’s comparability to the RCT controls through its impact on the outcome model, ranking samples by influence scores. The method constructs nested candidate borrowing sets and derives a semiparametric efficient estimator combining RCT data with these selected controls. The authors analyze asymptotic bias and variance under violations of exchangeability and propose data-driven selection of the optimal borrowed set by minimizing MSE. Simulations and real-world application (NSW/PSID) demonstrate the method’s superiority over existing approaches, including full and adaptive lasso-based borrowing (Gao et al).", "questions": "1. Inference and Confidence Intervals: The paper does not provide clear guidance on how to construct valid confidence intervals for the proposed estimator, especially given the adaptive selection of borrowed samples. Could the authors clarify whether the estimator is regular and root-n consistent? If not, how should practitioners approach uncertainty quantification?\n\n2. Choice of K (Number of Borrowed Samples): While the paper proposes ranking external controls by influence scores, there is no practical guidance on selecting the optimal K. Should practitioners perform a grid search over candidate Ks? Could the authors propose a data-driven or theoretically motivated stopping rule?\n\n3. Beyond Covariate Shift: The paper considers covariate shift (differences in X distributions) in simulations but does not address other forms of distribution shift between RCT and external controls, such as differences in Y|A,X (conditional outcome) or A|X (treatment assignment mechanism). Could the authors discuss the approach’s robustness to these more general types of shift in theory and simulations?\n\n4. Detectability of Bias and Limits of Borrowing: What are the limits on the degree of mismatch between RCT and external controls before the method’s bias detection and correction fail? Could the authors quantify scenarios where differences across data sources become too large for reliable borrowing?\n\n5. Comparison with Existing Methods: In what scenarios might the adaptive lasso-based approach (e.g., Gao et al.) outperform the influence-based method? A discussion of failure modes or trade-offs would help practitioners choose between methods.\n\n6. Related Literature: The introduction could better & more broadly be situated in the literature on generalizability/transportability of causal effects (e.g., work on extending causal inferences to new populations or data integration/fusion, see Dahabreh et al.). Relevant papers also include methods for federated causal inference (e.g., Xiong et al, 2023 SIM; Han et al., JASA 2025), etc.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n* The paper addresses a practically important problem of borrowing external controls when exchangeability may not hold.\n* The influence-based approach is relatively novel, theoretically grounded, and provides a principled way to assess sample comparability.\n* The paper presents rigorous asymptotic analysis of bias and variance.\n* Extensive simulations and real-data application demonstrate improvements over existing methods.\n* The paper is generally well-organized and clearly motivated, helping readers understand the method’s intuition.\n\nWeaknesses:\n* The paper lacks guidance on inference, including how to construct valid confidence intervals for the proposed estimator.\n* There is no practical strategy for choosing K, leaving practitioners without a clear selection procedure for the optimal number of borrowed samples.\n* The method’s robustness to other distributional shifts (e.g., outcome or treatment assignment mechanisms) beyond covariate shift is not explored.\n* The computational burden of influence score calculation, especially for large-scale or high-dimensional data, is not addressed in detail.\n* The introduction could better situate the work in the broader literature on generalizability, transportability, and federated causal inference.", "was_revised": true, "final_justification": "There have been many papers on combining RCT and observational studies, including in the context of external controls, but this paper proposes a novel idea in quantifying each external sample’s comparability to the RCT controls through its impact on the outcome model by ranking samples via a measure it calls \"influence scores.\" Although there are some limitations to the method, namely in its inference, I find that the authors have engaged with the reviewers and I believe the paper has improved via this process and contributes to the literature via a new idea; hence my increased score.", "rebuttal": "We thank you for your efforts in evaluating our work. **Below, we hope that our clarification addresses your concerns and sincerely invite you to kindly re-evaluate our work in light of our explanation.**\n\n> **Q1:** Inference and Confidence Intervals: The paper does not provide clear guidance on how to construct valid confidence intervals for the proposed estimator, especially given the adaptive selection of borrowed samples. Could the authors clarify whether the estimator is regular and root-n consistent? If not, how should practitioners approach uncertainty quantification?\n\n> **Response to Q1:** Thanks for your comments.  \n\n**Theoretically,** from Proposition 2 and Theorem 1, the proposed estimator is regular and root-$n$ consistent under Assumptions 1 and 2, and may exhibit bias when only Assumption 1 holds. Furthermore, we derive the asymptotic variance in both Proposition 2 and Theorem 1, which can be used to construct confidence intervals for the proposed estimator.\n\n**Empirically,** in our experiments, we presented the standard error (S.E). and the bias of the estimator in Table 3, and we can naturally construct the $95\\\\%$ confidence interval, by $\\text{Est.}\\pm 1.96\\times\\frac{\\text{S.E.}}{\\sqrt{N_{\\mathcal{E}}+N_{\\mathcal{S}}}}$, where \"Est.\" represents \"Point estimates\". We added confidence intervals in the following table. \n\n|  | Top-K=300  |   | Top-K=350  | | Top-K=400     |   |\n|--|--|--|--|--|--|--|\n| Linear | Est.(S.E.) | C.I.| Est.(S.E.)| C.I. | Est.(S.E.) | C.I.  |\n| $\\hat{\\tau}_{\\textup{aipw}}$  | 0.285(0.107) | [0.274,0.296]    | 0.285(0.107)  | [0.274,0.296]    | 0.285(0.107)  | [0.274,0.296]     |\n| $\\hat{\\tau}_{\\textup{full}}$  | 0.19(0.091)   | [0.185, 0.195]   | 0.19(0.091)   | [0.185, 0.195]   | 0.19(0.091)   | [0.185, 0.195]    |\n| $\\hat{\\tau}_{\\textup{lasso}}$ | 0.363(0.101)  | [0.356, 0.371]   | 0.364(0.099)  | [0.357, 0.371]   | 0.339(0.098)  | [0.332, 0.346]    |\n| $\\hat{\\tau}_{\\textup{if}}$    | 0.31(0.08)  | [0.304, 0.316]   | 0.311(0.081)  | [0.304, 0.316]   | 0.293(0.081)  | [0.288, 0.299]    |\n| | | | | | | |\n|  | Top-K=50 | | Top-K=100 | | Top-K=150 |  |\n| Nonlinear | Est.(S.E.) | C.I.    | Est.(S.E.)  | C.I. | Est.(S.E.)    | C.I.              |\n| $\\hat{\\tau}_{\\textup{aipw}}$  | 1.424(0.216)  | [1.404,1.443]    | 1.424(0.216)  | [1.404,1.443]    | 1.424(0.216)  | [1.404,1.443]     |\n| $\\hat{\\tau}_{\\textup{full}}$  | 0.975(0.186)  | [0.964, 0.986]   | 0.975(0.186)  | [0.964, 0.986]   | 0.975(0.0186) | [0.964, 0.986]    |\n| $\\hat{\\tau}_{\\textup{lasso}}$ | 1.392(0.212) | [1.373, 1.412]   | 1.359(0.221)  | [1.340, 1.379]   | 1.305(0.223)  | [1.286, 1.324]    |\n| $\\hat{\\tau}_{\\textup{if}}$    | 1.393(0.211)  | [1.373, 1.413]   | 1.378(0.213)  | [1.359, 1.397]   | 1.376(0.215)  | [1.358, 1.394]    |\n|                               |                          |                  |               |                  |               |                   |\n| | Top-K=30 | | Top-K=35 | | Top-K=45      |                   |\n| NSW \\& PSID | Est.(S.E.) | C.I.  | Est.(S.E.)    | C.I.  | Est.(S.E.)    | C.I.              |\n| $\\hat{\\tau}_{\\textup{aipw}}$  | 0.009(0.013)  | [0.007,0.10]     | 0.009(0.013)  | [0.007,0.10]     | 0.009(0.013)  | [0.007,0.10]      |\n| $\\hat{\\tau}_{\\textup{full}}$  | -0.013(0.015) | [-0.015, -0.012] | -0.013(0.015) | [-0.015, -0.012] | -0.013(0.015) | [-0.015, -0.012]  |\n| $\\hat{\\tau}_{\\textup{lasso}}$ | -0.006(0.014)  | [-0.008, -0.005] | -0.008(0.014) | [-0.010, -0.007] | -0.006(0.015) | [-0.008, -0.005]  |\n| $\\hat{\\tau}_{\\textup{if}}$    | 0.006(0.012)  | [0.005, 0.008]   | -0.008(0.013) | [0.006, 0.009]   | 0.010(0.013)  | [0.009, 0.012]    |\n\n\n> **Q2:** Choice of K (Number of Borrowed Samples): While the paper proposes ranking external controls by influence scores, there is no practical guidance on selecting the optimal K. Should practitioners perform a grid search over candidate Ks? Could the authors propose a data-driven or theoretically motivated stopping rule?\n\n **Response to Q2:** Thanks for your comments. **We would like to kindly remind the reviewer that we have proposed a method for selecting the optimal $K$ by minimizing the MSE of the estimator $\\hat \\tau_{S}$.  For clarity, we provide a detailed description of selecting the \"optimal\" borrowed set. It consists of two steps:** \n\n- Step 1:  Based on the ranking of influence scores, we construct a series of nested subsets of external controls, where each subset comprises the top-K most comparable samples (i.e., the top-K samples with the smallest influence scores), see lines 183--185 of the manuscript.  \n- Step 2: Find the optimal $K$ that minimizes the MSE based on Theorem 1, see Section 6.2 of the manuscript.\n\n**From Figure 2 of the manuscript, we can see that the MSE curve will show a trend of first decreasing and then increasing. Then the lowest point of the curve corresponds to the optimal K with the smallest empirical MSE.**  \n\n> **Q3:** Beyond Covariate Shift: The paper considers covariate shift (differences in X distributions) in simulations but does not address other forms of distribution shift between RCT and external controls, such as differences in Y|A, X (conditional outcome) or A|X (treatment assignment mechanism). Could the authors discuss the approach’s robustness to these more general types of shift in theory and simulations?\n\n **Response to Q3:**  Thanks for your comments. **We would like to kindly remind the reviewer that we have considered both the differences in $Y|A, X$ and $A|X$. In Table 2 (Simulation setting), the term $ \\delta T$ constitutes the differences in Y|A, X (conditional outcome), where $\\delta$ represents the level of inconcurrency. We describe this experimental setup in lines 267-268.**  \n\n**In addition, we additionally conducted experiments with different $\\delta$ to illustrate the robustness of the proposed method.** As shown in the table below, we can see that: (1)the proposed method stably performs well. (2) as $\\delta$ increases, the optimal $K$ becomes smaller. This is intuitive because a larger $\\delta$ means fewer samples in the external sample that meet comparability requirements.\n\n| | $\\delta=0.3$| $\\delta=0.5$ | $\\delta=0.7$ |\n|--|--|--|--|\n| Linear |  MSE(optimal_K = 300) | MSE(optimal_K = 250)| MSE(optimal_K = 200) |\n| $\\hat{\\tau}_{\\textup{aipw}}$ | 0.0116 | 0.0116 | 0.0116|\n| $\\hat{\\tau}_{\\textup{full}}$ | 0.0807 | 0.2046 | 0.3893 |\n| $\\hat{\\tau}_{\\textup{lasso}}$ |0.0155 | 0.0134 | 0.0132 |\n| $\\hat{\\tau}_{\\textup{if}}$ | **0.0067**| **0.0071** |**0.0067**|\n\nFor the A|X (treatment assignment mechanism), the treatment assignment for the trial data is completely at random, and $A = 0$ for all external controls. \n\n\n> **Q4:** Detectability of Bias and Limits of Borrowing: What are the limits on the degree of mismatch between RCT and external controls before the method’s bias detection and correction fail? Could the authors quantify scenarios where differences across data sources become too large for reliable borrowing?\n\n **Response to Q4:** Thanks for raising this interesting question. **Reliable borrowing of our method depends on the difference between RCT and external control not being too large. \nWhen \n\\\\\\[      \\mathbb{E}[Y(0) | X=x,R=1]  - \\mathbb{E}[Y(0) | X=x,R=0]  \\text{ is large for all } x,   \\\\\\]\nour method fails.** This is because, in this case, no comparable samples are available between RCT and external control.  All influence scores are large, and introducing the external controls will incur a large bias compared to its contribution in variance reduction. We also would like to clarify that this problem is not unique to our method. All previous methods also fail.  \n\n**Moreover, we added additional experiments to illustrate this empirically by setting a large value of $\\delta$ (see table below)**. The results show that only a small number of external controls are selected, and the standard AIPW method exhibits the most competitive performance.\n| | $\\delta=5$ |\n|-|-|\n|Linear| MSE(optimal_K = NA) |\n|$\\hat{\\tau}_{\\textup{aipw}}$ |**0.0116** (K=10)|\n|$\\hat{\\tau}_{\\textup{full}}$ |44.124(K=10)|\n|$\\hat{\\tau}_{\\textup{lasso}}$|0.036(K=10) |\n|$\\hat{\\tau}_{\\textup{if}}$|0.033(K=10)|\n\n\n> **Q5:**  Comparison with Existing Methods: In what scenarios might the adaptive lasso-based approach (e.g., Gao et al.) outperform the influence-based method? A discussion of failure modes or trade-offs would help practitioners choose between methods.\n\n **Response to Q5:** Thanks for your suggestions.  **We are not aware of any settings where the lasso-based method is better than ours. If you have further comments on this, we welcome further discussion.** **In addition, we would like to clarify that our method has two key merits compared with Gao et al. (2024): Better comparability and insensitivity to outliers, please refer to Section 4 for more details. These two salient merits stem from the fact that ''our method is model-free for outcome model in the external controls.'**' In contrast, the lasso-based method (Gao et al. (2024)) is model-dependent and relies on the outcome model of the external control data, and the simple difference between the two outcome models cannot fully capture the comparability of the external data, as discussed in Section 4. \n\n\n> **Q6:** Related Literature: The introduction could better & more broadly be situated in the literature on generalizability/transportability of causal effects (e.g., work on extending causal inferences to new populations or data integration/fusion, see Dahabreh et al.). Relevant papers also include methods for federated causal inference (e.g., Xiong et al, 2023 SIM; Han et al., JASA 2025), etc.\n\n **Response to Q6:** We sincerely thank the reviewer for pointing out these helpful references. We fully agree that it would benefit from a more comprehensive discussion of the broader literature on generalizability/transportability and federated causal inference. We will add several discussions for them in our revised version."}, {"review_id": "W9ayDQCt9d", "reviewer": "Reviewer_uy6o", "summary": "This paper studies how to use external control data to improve inferential efficiency in estimating treatment effects for randomized controlled trials (RCTs), especially those with small sample sizes. The main difficulty arises from the individual heterogeneity; that is, the potential outcomes of external control units may be different from those of RCT control units. To address this issue, the authors proposed an influence-based adaptive sample borrowing approach that quantifies the comparability of each unit in the external control and adaptively adds external units to enhance the efficiency of estimating treatment effects. Specifically, the method calculates an influence score for each external unit based on the change of a pre-specified loss function when the external unit is added to learn the model parameters. Then, the authors derived a semiparametric estimator for the average treatment effect (a doubly robust estimator), analyzed its bias, and proved asymptotic normality of the estimation error.", "questions": "1. I am confused about Proposition 1. What is the \"efficient influence function\"? It seems [1] also provides a formula for the efficient influence function under the same assumptions if one replaces $P$ with $P_S$. Is the formula in Proposition 1 different from that in [1]? If so, why are they different? In addition, what is the parameter $q$ in Proposition 1?\n\n2. In Section 5, when learning the model $\\hat{\\theta}_z$ parameters by the empirical risk minimizer, there is a regularizer term; however, when using the influence function to approximate the model change $\\hat{\\theta}$$\\epsilon, z$, there is no regularizer term. Could the authors explain this discrepancy?\n\n3. In the experiment setup for the NSW data, could the authors explain how the unobserved outcomes are generated? Every unit has a missing potential outcome. I assume that one needs all potential outcomes to calculate the MSE?\n\n4. In Proposition 2 and Theorem 1, should $(N_E + N_S) (\\hat{\\tau}_S - \\tau) \\rightarrow N(0, \\sigma^2)$ be $\\sqrt{N_E + N_S} (\\hat{\\tau}_S - \\tau) \\rightarrow N(0, \\sigma^2)$?\n\nReferences:\n[1] Chenyin Gao, Shu Yang, Mingyang Shan, Wenyu YE, Ilya Lipkovich, and Douglas Faries. Improving randomized controlled trial analysis via data-adaptive borrowing. Biometrika, page asae069, 2024.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strengths:\n1. The research problem is well-motivated. Due to small sample sizes of RCTs, one wants to combine external control data to improve the efficiency of estimating treatment effects. However, most existing works rely on a potentially implausible assumption that external control and RCT control units are exchangeable. The authors aim to drop this exchangeability assumption. The introduction is well written.\n\n2. The authors proposed a new influence-based method to adaptively borrow external control units. This new method can handle external outliers effectively.\n\n### Weaknesses:\n1. Compared with [1], it seems that the main difference appears to be how to select the external control units. The semiparametric estimator and the asymptotic normality of the estimation error seem similar to those in [1]. The authors could consider providing a more detailed comparison with [1].\n\n2. While the authors presented numerical evidence showing improvement over [1] when the external control data contain outliers, the authors could consider providing some theoretical results that specify the conditions under which their method has a smaller MSE.\n\n3. The authors should provide more details on how to estimate the parameters in their estimator, such as $\\hat{\\pi}$ (I understand that the authors mentioned $\\hat{m}$ is estimated using standard machine learning methods, but I don't think they discussed $\\hat{\\pi}$). In addition, I think the propensity score of the RCT units (i.e., $\\hat{e}_1$) is known by the RCT design.\n\n4. I am confused about Section 6.2 -- find the \"optimal\" borrowed set. Could the authors elaborate on how the borrowed set S that minimizes the MSE is selected? Do you greedily add external units based on their influence scores? If so, why does this method find the optimal S?\n\n5. The authors may also compare their method with other classical methods, such as matching.\n\n6. I think the authors should mention that they assume that the unit data are i.i.d. samples. This assumption is used in their proof of asymptotic normality of the estimation error.\n\nReferences:\n[1] Chenyin Gao, Shu Yang, Mingyang Shan, Wenyu YE, Ilya Lipkovich, and Douglas Faries. Improving randomized controlled trial analysis via data-adaptive borrowing. Biometrika, page asae069, 2024.", "was_revised": true, "final_justification": "The paper studies a well-motivated research problem and provides new ideas for borrowing external data to improve causal estimation. Although I found the theoretical analysis somewhat limited, I think the paper is overall interesting. I also appreciate the authors' detailed responses to my questions during the rebuttal. I increased my score. A minor suggestion for the authors is to provide a clearer explanation of the \"optimal borrowed set\" in a future version.", "rebuttal": "**We thank you for your efforts in evaluating our work. Below, we hope that our clarification addresses your concerns and sincerely invite you to kindly re-evaluate our work in light of our explanation.**\n\n> **W1:** Compared with [1], it seems that the main difference appears to be in how to select the external control units. The semiparametric estimator seems similar to those in [1]. The authors could consider providing a more detailed comparison with [1].\n\n**Response to W1:** Thanks for your insightful comments. We agree that both studies share common ground in the semiparametric theory. However, **we would like to clarify that our primary contributions are not in the estimation itself, but in developing a novel approach for selecting external comparable samples and determining the optimal selection set.** \n\n- In Section 4, we provided a comparison between our influence-based method and the lasso-based approach introduced in [1], summarized in the table below.  \n- In Sections 5--6, we developed the influence-based method and the optimal borrowed set selection method.  \n\n|   |model-free for external controls | robust to outliers  |\n|---|------|--|\n| the adaptive lasso-based method[1] | ×   | ×    |\n| the influence-based method      | √     | √    |\n\n**In addition, in lines 61--68 (the end of Introduction), we did not claim any contribution to semiparametric efficiency. We only claimed the following main contributions:** \n- We reveal the limitations of existing approaches for estimating treatment effects by combining RCT data with external controls. \n- We propose an influence-based sample borrowing approach, which can effectively quantify the comparability of each sample in the external controls.\n-  We develop a data-driven approach to select the optimal subset of external control samples based on the MSE of the proposed estimator. \n- We conduct extensive experiments on both simulated and real-world datasets, demonstrating that the proposed approach outperforms the existing baseline approaches.\n\n> **W2:** The authors could consider providing some theoretical results that specify the conditions under which their method has a smaller MSE.\n\n**Response to W2:**  Thanks for your suggestions. **We are not aware of any settings where the lasso-based method is better than ours. If you have further comments on this, we welcome further discussion.**\n\n**In addition, we would like to clarify that our method has two key merits compared with [1]: Better comparability and insensitivity to outliers, please refer to Section 4 for more details. These two salient merits stem from the fact that ''our method is model-free for outcome model in the external controls.'**' In contrast, the lasso-based method [1] is model-dependent and relies on the outcome model of the external control data, and the simple difference between the two outcome models cannot fully capture the comparability of the external data, as discussed in Section 4. \n\n> **W3:** How to estimate $\\hat \\pi$. In addition, I think $\\hat e_1$ is known by the RCT design.   \n\n**Response to W3:** We appreciate your suggestions. In our experiments, we adopt a logistic regression model for estimating $\\hat \\pi$. **We acknowledge that $\\hat{e}_1(X)$ is sometimes known in the context of RCTs. However, we would like to clarify that our proposed method is directly applicable in such cases (as all theoretical results also hold).** We will make this point clearer in the revised version.\n\n> **W4:** I am confused about Section 6.2 -- find the \"optimal\" borrowed set. Could the authors elaborate on it? \n\n**Response to W4:** Thanks for your comments, and we apologize for the lack of clarity on selecting the \"optimal\" borrowed set. Below, we provide a detailed description.\n**The proposed method for selecting the \"optimal\" borrowed set consists of two steps:**\n- Step 1:  Based on the ranking of influence scores, we construct a series of nested subsets of external controls, where each subset comprises the top-K most comparable samples (i.e., the top-K samples with the smallest influence scores), see lines 183--185 of the manuscript.  \n- Step 2: Find the optimal $K$ that minimizes the MSE based on Theorem 1, see Section 6.2 of the manuscript.\n\n**From Figure 2 of the manuscript, we can see that the MSE curve will show a trend of first decreasing and then increasing. Then the lowest point of the curve corresponds to the optimal $K$ with the smallest empirical MSE.** \n\n> **W5:** The authors may also compare their method with other classical methods, such as matching.\n\n**Response to W5:** Thank you for raising this interesting question. To the best of our knowledge, there is currently no matching method specifically designed for adaptive selecting external samples. \n\n**We assume the reviewer may be suggesting a strategy in which external control samples that are close to at least one sample in the RCT are selected, while those far from the RCT data are excluded. If this is the case, we would like to clarify that such a matching approach cannot effectively distinguish between 'comparable' and 'non-comparable' samples.** \n\n**The reason is as follows: typical matching methods rely on covariate distance, based on the assumption that \n''units with similar $x$ also have similar potential outcomes.'' \nHowever, such methods ignore outcome information and may inadvertently match 'non-comparable' samples--particularly when some external control units exhibit conditional outcome patterns (i.e., $E[Y|X]$) that differ from those in the RCT.**\n\n For example, from Figure 2 in the manuscript,  we could see that all the range of $x$ in the external controls and RCTs are the same. If we conduct matching based on $x$, then we will treat all external controls as \"comparable\" samples. Clearly, it is not right.   \n\n> **W6:** I think the authors should mention that they assume that the unit data are i.i.d. samples. \n\n**Response to W6:** We apologize for the lack of details. We will revise the manuscript to present it clearly. \n\n> **Q1:** I am confused about Proposition 1. It seems [1] also provides a formula for the efficient influence function under the same assumptions if one replaces $P$ with $P_S$. In addition, what is the parameter $q$ in Proposition 1?\n\n**Response to Q1:**  Thank you for raising this issue. The efficient influence function is similar to that of [1] if $P$ is replaced by $P_{S}$, and $q=P_{S}(R=1)$ represents the proportion of RCT data (i.e., R=1) within the combined population $P_{S}$. We will clarify it in the revised version. Thanks again. \n\n**In addition, as our response to W1, our primary contributions are not in the estimation itself, but in developing a novel approach for selecting external comparable samples and determining the optimal selection set\nTherefore, Proposition 1 is just the beginning of our estimation method.**\n\n> **Q2:** In Section 5, when using the influence function to approximate the $\\hat \\theta_{\\epsilon,z}$, there is no regularizer term.\n\n**Response to Q2:** Thanks for raising this point. **We would like to clarify that, theoretically, the influence function approximation of the model change $\\hat \\theta_{\\epsilon,z}$ does not require a regularizer.** Below, we provide the formal derivation:\n\\\\\\[\\\n    \\\\hat \\\\theta\\_{\\\\epsilon,z}\\\\triangleq\n    \\\\arg\\\\min\\_{\\\\theta}\\\\frac{1}{N\\_{C}}\\\\sum\\_{Z_i\\\\in C}L(Z_i,\\\\theta)+\\\\epsilon\n    L (z;\\\\theta)+\\\\frac{\\\\lambda}{2}\\\\Vert\\\\theta\\\\Vert_2\\^2\\\n    \\\\\\]\\\nBy definition,\\\n\\\\\\[\\\n    0=\\\\frac{1}{N\\_{C}}\\\\sum\\_{Z_i\\\\in C}\\\\frac{\\\\partial\n      L (Z_i,\\\\hat{\\\\theta}\\_{\\\\epsilon,z})}{\\\\partial\n        \\\\theta}+\\\\epsilon\\\\cdot\\\\frac{\\\\partial\n          L (z,\\\\hat{\\\\theta}\\_{\\\\epsilon,z})}{\\\\partial\n            \\\\theta}+\\\\frac{\\\\lambda}{2}\\\\hat{\\\\theta}\\_{\\\\epsilon,z}.\\\n    \\\\\\]\\\nBy Taylor expansion with respect to \\$\\\\theta\\$ at \\$\\\\hat{\\\\theta}\\$,\nwe have that,\\\n\\\\\\[\\\n    0 \\\\approx\n    \\\\frac{1}{N\\_{C}}\\\\sum\\_{Z_i\\\\in C}\\\\frac{\\\\partial\n      L (Z_i,\\\\hat{\\\\theta}\\_{\\\\epsilon,z})}{\\\\partial\n        \\\\theta}+\\\\epsilon\\\\cdot\\\\frac{\\\\partial\n          L (z,\\\\hat{\\\\theta}\\_{\\\\epsilon,z})}{\\\\partial\n            \\\\theta}+\\\\frac{\\\\lambda}{2}\\\\hat{\\\\theta}\\_{\\\\epsilon,z}+(\\\\frac{1}{N\\_{C}}\\\\sum\\_{Z_i\\\\in C}\\\\frac{\\\\partial\\^2\n              L (Z_i,\\\\hat{\\\\theta})}{\\\\partial\n                \\\\theta\\^2}+\\\\epsilon\\\\cdot\\\\frac{\\\\partial\\^2\n                  L (z,\\\\hat{\\\\theta})}{\\\\partial\n                    \\\\theta\\^2}+\\\\frac{\\\\lambda}{2})\\\\cdot(\\\\hat{\\\\theta}\\_{\\\\epsilon,z}-\\\\hat{\\\\theta}).\\\n    \\\\\\]\\\nThen we have,\\\n\\\\\\[\\\n    \\\\hat{\\\\theta}\\_{\\\\epsilon,z}-\\\\hat{\\\\theta}\\\\approx\n    \\[\\\\frac{1}{N\\_{C}}\\\\sum\\_{Z_i\\\\in C}\\\\frac{\\\\partial\\^2\n      L (Z_i,\\\\hat{\\\\theta})}{\\\\partial\n        \\\\theta\\^2}+\\\\epsilon\\\\cdot\\\\frac{\\\\partial\\^2\n          L (z,\\\\hat{\\\\theta})}{\\\\partial\n            \\\\theta\\^2}+\\\\frac{\\\\lambda}{2}\\]\\^{-1}\\\\cdot \\\\epsilon\n    \\\\cdot\\\\frac{\\\\partial L (z,\\\\hat{\\\\theta})}{\\\\partial \\\\theta}.\\\n    \\\\\\]\nThus, when $\\lambda\\to 0$,\\\n\\\\\\[\\\n    \\\\left.\\\\frac{d \\\\hat{\\\\theta}\\_{\\\\epsilon,z}}{d \\\\epsilon}\n    \\\\right\\|\\_{\\\\epsilon = 0}\\\\approx\n    \\[\\\\frac{1}{N\\_{C}}\\\\sum\\_{Z_i\\\\in C}\\\\frac{\\\\partial\\^2\n      L (Z_i,\\\\hat{\\\\theta})}{\\\\partial \\\\theta\\^2}\\]\\^{-1}\\\\cdot\n    \\\\frac{\\\\partial L (z,\\\\hat{\\\\theta})}{\\\\partial \\\\theta}.\\\n    \\\\\\]\n\n> **Q3:** For the NSW data, could the authors explain how the unobserved outcomes are generated? \n\n**Response to Q3:** Thanks for your comments. **We would like to clarify that we did not generate the unobserved outcomes for the NSW data. Below, we give a detailed description of how to calculate the MSE with the observed data only.**\n- Calculate $\\hat\\tau_{S}$ and $\\hat\\tau_{aipw}$;\n- Calculate $\\textup{bias}(\\hat \\tau_{S}) = \\hat \\tau_{S}-\\hat \\tau_{aipw}$ and $\\textup{var}(\\hat \\tau_{S}) = \\textup{var}(\\hat \\phi)$\n- Calculate MSE: $\\textup{MSE}(\\hat \\tau_S) = \\textup{bias}^2(\\hat \\tau_S) + \\textup{var}(\\hat \\tau_S)$. \n\n**Response to Q4:** Thanks for pointing this out. Yes, we will revise it."}]

Author rebuttal:

Meta-review: This paper proposes an influence-function-based framework for adaptively borrowing external control samples to improve treatment effect estimation in RCTs. The method assigns influence scores to external controls, develops a semiparametric efficient estimator, and uses a bias-variance tradeoff to select optimal subsets. Reviewers found the work theoretically grounded and practically relevant, noting its novelty in moving beyond global exchangeability assumptions and demonstrating robustness across scenarios. I encourage the authors to include a candid discussion of the concerns raised in the review process, including computational scalability, limited empirical scope, and the lack of formal treatment of post-selection inference variability. This will strengthen the paper and clarify the practical boundaries of the proposed method.
Decision: Accept (poster)