FAST: a Fused and Accurate Shrinkage Tree for Heterogeneous Treatment Effects Estimation
Authors: Jia Gu
Caizhi Tang
Han Yan
Qing Cui
Longfei Li
JUN ZHOU
Venue: NeurIPS 2023 poster (2023)
Keywords: Data fusion
heterogeneous treatment effects estimation
shrinkage estimation
tree-based method

Abstract: This paper proposes a novel strategy for estimating the heterogeneous treatment effect  called the Fused and Accurate Shrinkage Tree ($\mathrm{FAST}$). Our approach utilizes both trial and observational data to improve the accuracy and robustness of the estimator. Inspired by the concept of shrinkage estimation in statistics, we develop an optimal weighting scheme and a corresponding estimator that balances the unbiased estimator based on the trial data with the potentially biased estimator based on the observational data. Specifically, combined with tree-based techniques, we introduce a new split criterion that utilizes both trial data and observational data to more accurately estimate the treatment effect. Furthermore, we confirm the consistency of our proposed tree-based estimator and demonstrate the effectiveness of our criterion in reducing prediction error through theoretical analysis.  The advantageous  finite sample performance of the $\mathrm{FAST}$ and its ensemble version over existing methods is demonstrated via  simulations and real data analysis.

FAST: a Fused and Accurate Shrinkage Tree for Heterogeneous Treatment Effects Estimation
Jia G22 Caizhi Tang*?, Han Yan', Qing Cui?, Longfei Lif, and Jun Zhout2
'Center for Statistical Science, Peking University
Ant Group
GGungghua School of Management, Peking University
gujia@pku.edu.om caizhi.tez @rizzaanti watiz/aannntt..oo hanyan@stu.pal
@ www.essstttuu...ommm fcuiqing.cg. longyao.lff,
jun.zhoujun) @ntgrroup.com
Abstract
This paper proposes novel strategy for estimating the heterogeneous treatment effect called the Fused and Accurate Shrinkage Tree (FAST). Our approach utilizes both trial and observational data to improve the accuracy and robustness of the estimator. Inspired by the concept of shrinkage estimation in statistics, we develop optimal weighting scheme and corresponding estimator that balances the unbiased estimator based on the trial data with the potentially biased estimator based the observational data. Specifically, combined with tree-based techniques, we introduce a new split criterion that utilizes both trial data and observational data accurately estimate the treatment effect. Furthermore, consistency of our proposed tree- rre-based estimator and demonstrate the effectiveness of our criterion in reducing prediction error through theoretical analysis. The ad advantageous finite sample performance of the FAST and its ensemble version over existing methods is demonstrated via simulations and real data analysis.
confirm the
Introduction
Causal effects the magnitude of the response of an effect variable (also called outcome) caused by the effect variable (also called treatment), which is a fundamental and essential issue in the field of casual inference (Imbens and Rubin 2016). And the heterogeneous treatment effect (abbr. HTE) is usually used to characterize the heterogeneity of causal effects across different subgroups of the population. In recent years, heterogeneous treatment effect estimation has been successfully applied in various fields such as epidemiology, medicine, and social sciences (Glass et al. 2013 Kosorok and
Laber 2019 Turney and Wildeman 2015 Taddy et 2016).
In general, the causal problems can studied through both experimental studies (also known randomized control trials, RCTs) and observational studies. Experimental studies widely regarded as the gold standard for assessing causal effects since the randomization process eliminates the possibility of confounding bias. However, large-scals scale RCTs can be challenging due issues related to cost, time, and ethics (Edwards et al.] 1999). On the other hand, observational data often readily available with adequate sample size. Under certain fairly strong assumptions, such as unconfoundedness assumption, there a rich literature regarding the estimation of HTE in observational studies, such tree-based methods (Athey and Imbens] 2016 Wager and Athey 2018 Athey et al. 2019 Hahn et al. 2020), boosting (Powers et al. 2017) meta learners (Künzel et al.
Equual Contribution CCorresponding Author
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

2019) and R-learner (Nie and Wager 2020). However, the unconfoundedness assumption, which requires measuring all confounders, is in general untestable unless extra information such as the existing of instrumental variables is available (de Luna and Johansson 2014). And any violations of this assumption may result in seriously invalid causal statements. Various methods have been proposed to mitigate the unmeasured confounding in observational studies, such as the sensitivity analysis (Rosenbaum and Rubin| 1983; Zhang and Tchetgen Tchetgen (022), the instrumental variables (IV) approach (Angrist et al. 1996) and the proximal causal inference (Kuroki and Pearl 2014; Miao et al. 2018 Shi et al. 2020 Cui et al, 2023). However, the validity of these procedures also relies crucially on assumptions that are often difficult to verify in practice.
Given the limitations of relying on individual data sources, data fusion, as a branch of causal inference strategies that integrates both the trial and the observational data, has gained significant interest in the literature (Bareinboim and Pearl 2016] Colnet et al. 2020; Shi et al. 2022). Existing data fusion methods for estimating the HTE include the KPS estimator obtained by modeling the confounding function parametrically (Kallus et al.] 2018). the
integrative estimator under the
parametric structural models (Yang et al. 2020) and the integrative R-learner (Wu and Yang 2022). Besides, (Tang et al. 2022) proposed the Gradient Boosting Causal Tree (GBCT), which integrated the current observational data and their historical controls for estimating the conditional average treatment effect on the treated group (CATT).
This paper presents novel approach for estimating heterogeneous treatment effects (HTE) in the context of causal data fusion. The proposed method, named Fused and Accurate Shrinkage Tree (FAST), avoids the need for a two-stage estimation process required in conventional data fusion strategies, which involves modeling and estimating the nuisance confounding bias function. The main contributions of this work can be summarized as follows (i) The authors propose novel shrinkage method for combining an unbiased and biased estimator, which effectively reduces the mean square error of the unbiased estimator, and provides an easy implementation of the method tailored for the HTE estimation; (ii) The authors extend the conventional node split criterion via a re-scaling technique, which automatically penalizes the use of the observational data with low quality (namely large confounding bias); (ii) The authors also provide a theoretical analysis to explain the advantages of our splitting criterion.
2 Background and motivation
2.1 Notations
Let X E X [-1, be a p-dimensional vector of pre-treatment covariates, U E RR9 (g 2 0) be possibly unmeasured random vector consisting of the confounding variables, D be binary treatment variable (D- = O denotes the control and D = 1 denotes the treated) and let ((d) be the potential outcome that would be observed when the treatment had been set to d E {0, 1}. We follow the potential outcome framework (Rubin, 1974) to define the heterogeneous treatment effect r(x) = (L) Y() w).
Suppose that we can collect two kinds of data: trial data and observational data, and they are described by + m quadruples. (Y,, D.. X, St 1 where S; indicates if the i-th individual would have been recruited (S = 1) or not (S = 0) in the trial. We also denote R = {1, 2, ... ... the set of indices of observations in the RCT study, and O {n 1, 2, ,n m) the set of indices of observations in the observational study. We define (X.UUS) = P(D = =1 U, as the propensity score of the trial and observational population, respectively. In practice, due U being unknown, we usually use é(X, S) to estimate e(X, U, S). In addition, é(X,1) is unbiased, but é(X, 0) is biased because the unmeasured confounder U in the observational study can be related to the assignment of treatment D. Let 7i (x) = IE (1) - Y (0) X = x, S = 1) be the HTE on the trial population. We then make the following fundamental assumption on the trial and observational studies, which facilitates the potential for causal data fusion:
Assumption 1. (i) For any x E X, Ti (x) = = (i) (d) 1 DI(X, S = 1) for d E {0, 1) and (iii) the propensity score ō < el X, S)
8 almost surely for some constart 0 < 8 << 1/2.
Assumption D (i) states that the HTE function is transportable from the trial population to the Conipitioooonncuude include the ignorability of study participation (Buchanan et al. 2018) and the mean exchangeability (Dahabreh et al. 2019). In the following of this
target population. Stronger versions of Assumption

---

paper, we use |A] to denote the number of elements for any set A, |c] to denote the biggest integer less than or equal to the constant e, and [p] to denote the index set f1, 2.. [p]] }. For two positive sequences {an J221 and {bn, 1221, we write on = 0(b,) if |0,/bo, / is bounded.
2.2 Tree-based methods
To estimate the HTE, it is reasonable to perform subgroup analysis by appropriately stratifying or matching (Frangakis and Rubin (2002) the samples into multiple subgroups that differ in the magnitude of treatment eftects. machine learning, tree-based methods (Breiman et al. 1984] Breiman [200T] Friedman 2001) are usually used for such stratification tasks, which greedily optimize the loss function, also called splitting criterion, via recursively partitioning feature space. fact, many tree- based causal methods designed for the HTE estimation were also proposed (Radeliffe and Surry 2012 Athey and Imbens 2016 Athey et al. 2019). Along with the development of the tree-based methods, various regulariin ragularization strategies, either implicit explicit, have been proposed to mitigate overfitting (Mentch and Zhou 2020 Agarwal et al. (022). Recently, tree-based methods have been generalized address the heterogeneous data from diverse data sources(Nasseri al. 2022). For convenience, in the following we define a regression tree by two components: a set of leaves Q = {Q1))=1 and associated parameter т. We can denote a causal tree by T(X; Q, Σÿ= (Q)) Q₁}. where 1(-} denotes the indicator function and T(2,) denotes the casual effect of aubaarea indexed by Qj-
E(O -0)² 2.37 min(E(O.. - -0),-00), 0)²;
Var(a,)
(5,) )
Var(O,,> 1.26 B(B.-) 1.05
Probabilng denatyy
0.4
Probability danaity
0.4
Shrinkage
0.2
Estimation
0.2
(ar().) 5
Optimal weight w' 0.53
0.0
0.0
-5.0
-2.5
0.0
2.5
5.0
-5.0
-2.5
0.0
2.5
5.0
Figure 1: The probability density functions (pdfs) of the unbiased estimator (pink) and the biased estimator (blue) in the left panel and the pdf of the shrinkage (fused) estimator under the optimal weight W" (green) in the right panel. The vertical dashed line represents the true parameter value 8" 0.
2.3 Shrinkage estimation
is important to note that applying conventional methods, such as the generalized random forest (Athey et al.) 2019),, separately trial data and observational data can readily lead two estimators: the first is unbiased but may exhibit large variability, while the second is potentially biased but usually has smaller variance due to the much larger amount of observational data. Therefore, the challenge becomes finding the optimal combination of an unbiased estimator and a biased estimator the data fusion problem. To see this, suppose we have parameter of interest estimator d_, and a (potentially) biased estimator Ô, of 0, such that E(QO.) = 0, TE(A))
R. an unbiased b(O), Var(o.,) = oi Var(@,) = of and Cov(e,, 0.) = Consider the family of estimators A., {Ø., wÓ, + (1 w) w).., the following expansion:
- 1}. then the mean square error (MSE) of its elements admits
E(O,. - 0) (oī + b0) yz 20 o
(I)
3

---

= Minimizing ① with respect to w, we can obtain the unique minimizer w" = / + (00) and the gain of the optimal weighting over the single estimators 0u and 0g can be characterized by the following formula:
R - -1-
(2)))).
Comment The weighting strategy is akin to the classical fames-Stein shrinkage estimation (Efron and Morris [973 Green and Strawderman 1991) method, in which it is shown that multivariate normal vector Z (p 2 3), as a maximum likelihood estimator (MLE) of its population mean μ = E Z). is not minimax optimal, and the MSE of the estimator Z can be reduced by shrinking towards the zero vector 0 by some factor O < w < 1. The zero vector can be viewed as a biased estimator of y with zero variance in their setting. In comparison, (potentially) biased estimator 03: The larger the variance of of the unbiased estimator is compared to (0) + of the more the fused estimator 0... will be shrunk towards the biased estimator that is less fluctuating. By doing so, one can efficiently mitigate the occurrence of significant estimation error in the unbiased estimator caused by its high variance, as unbiasedness alone does not guarantee reliable estimation performance with a limited sample size. Figure iillustaates concrete example of the benefit provided by the shrinkage estimation, where 0, B, N(0, 5) and fg NN 2, 0.5). The fused estimator d reduces over 50% of the MSE compared with the unbiased estimator 6..
replace the deterministic estimator with
3 Methodology
In this section, we propose new data fusion strategy, referred to the Fused and Accurate Shrinkage Tree (FAST). We proceed in a bottom-up manner to provide a clear and intuitive illustration of the entire estimation: we will begin by applying the shrinkage estimation strategy for local data fusion within each sub-region the feature space given by pre-specified partition. Then, we propose a fused criterion that incorporates the information contained in the observational data via a simple re- re-scaling of the conventional criterion. Theoretical guarantees are established in Section
3.1 Local fusion for the HTE estimation
Under pre-specified partition Q = {Qj}/=1 of the feature space, let R, R and O, {ili E O, X; E Q represent the sets of indices of the trial and observational sub-samples, respectively, that fall within the region Qj- Let
YD
Y(1-D) D)
Y
eX
1 1-e(X,5)
be transformed outcomes of all data, e.g., the transformed outcomes of i-th sample can be denoted by
Y, Then under Assumption
| one can immediately show for the trial population with S 1:
E EY 5= ==
E (Y X, S = 1) E (D)X, 1) and (Y X, 5 () =)), leading to
E(Y(1 - X)
E (01XX= = S =1) (m) = T(x).
Thus, (0,) (1/RR) EIER, is unbiased estimator of E(Y Y(J E EQ,
which can be seen as reasonable approximation of 7(2,) Q segments the feature space properly such that T(x) varies slowly in each sub-region Qj. An estimator of Var(f.(Q;)) is given by a (Q₂) (1/1(R, 1))) ΣiER, (Y, &u (Q))) In contrast, for the observational population, the conditional independence no longer holds and A (Q₂) = (1/10,1) Y, is biased estimator concerning r(2,). due to the presence of unmeasured confounding (U) on the observational data.
It remains to estimate the weight w" (Q₂) composed of the tuple (o) (Q)), (Q,)). The first term (Q)) can be estimated by (Q,). To bypass the unmeasured confounding issue of the observational population, re-sampling techniques, such as the Bootstrap (Efron) 1979) Hall 1992), can be applied to consistently estimate of (Q2). However, in the causal data fusion setting, (Q,)) D((O,1-1) is expected to be of smaller order term compared to a (Q,) O(RR,1-)).

---

which is consequence of the relative sample size between the trial and the observational data. Thus,
one can just avoid estimating the negligible term (Q))). For the last term, b(2,) = ( (Q))
serves natural estimator of the bias b(Q,), This leads to the following estimator of (Q,) and the corresponding fused estimator
(Q,) = (b(()))=0((0//660+++)))))) and
(5)
fos (Q₂) waf A fused estimator of the HTE function 7(-) under the partition Q can thus be defined as fz(x)
(6)
Ej=1 (Q Q2}.
3.2 Adaptive fusion for segmentation
In order obtain a tree-based partition Q designed for the fusion strategy (6). split criterion is required, which is sufficient to be defined only at the root node given the recursive nature of the partitioning. We follow the honest estimation approach (Athey and Imbens, 2016) prevent overfitting. Specifically, given fraction 0 < < 1 (typically = 0.5), [rm] observations are sampled without replacement from the trial data of sample size n for the tree structure estimation, while the rest of observations are used for local estimation of the HTE in each leaf node. Let the index sets of the trial data used for the partition and the HTE estimation be R. and R respectively, We do not further split the observational data to reduce uncertainty, since we have already partitioned the trial data to avoid overfitting.
The conventional criterion for growing a regression tree chooses the index of the split variable and its split value the root node by minimizing the following goodness-of-fi criterion
(q, arg min GElp),CER
Σ i
Y tu (u((),))))
Σ TE 12
(7)
where Qr = { S E}, R2 = fili E E Qr) and f. (Q2, R*) (1/liili R', X; e
(r.)1) EiE(HEER' E, xxE V and R% and (u R') can be defined correspondingly. Given a tree grown under 7 we fuse the trial data indexed by R and the observational data indexed by O at each leaf node according to (6) and refer to the resulting tree estimator as Shrinkage Tree (ST). direct modification of (7). which aligns more with the fused estimator at the leaf nodes, should be
Σ Y - Fas (Q.))² R
+ Σ (vi-Fo((RR)) EE
(q, = arg min
AEER
where for (Q) = (. + (1- (.) R¹) and f. R)) is defined corre-
( spondingly. The replacement the unbiased estimators in 7 with the fused estimators facilitates goodness-of-fit criterion of the proposed fusion strategy.
Altemattiely. (7) can be interpreted as minimizing the sum of the estimated MSEs of the unbiased estimators at the child nodes, if the two terms on the right-hand side of 7 are divided by the square of their respective sample sizes. By contrast, since the fused estimator for reduces variance by shrinking the original unbiased estimator to a potentially biased estimator, simply comparing the fused estimators with the outcomes of the trial data as in (8) fails to capture the variability the child nodes. Instead, an appropriate criterion shall respect the MSE of the fused estimator. To this end, we introduce the following split criterion
(11 (1--ūs - (2))) RR) + (1 - Dos (2)
(g.c) = arg min qE PE,ER
(9)
where (1-0,
R') and (1- (I- (2))) Rt) estimate the MSE of for for (QL) and
for @R). respectively, according to formula 2 Compared to the proposed criterion incorporates the additional information from the observational data into each node split an adaptive manner by simply re-scaling the estimated MSE of the unbiased estimator.
5

---

Comment The criterion (9) offers the benefit of local adjustment, which can be intuitively justified. In sub-regions where the observational data exhibit moderate confounding biases, this criterion improves tree building by providing sharper assessment of the variability of the fused estimator. On the other hand, for sub-regions where the observational data exhibit substantial confounding biases, the estimated weights of those sub-regions approach zero according to ( In such cases, the criterion reduces to the conventional criterion (7). except for the standardization of the square of the sample size. It is worth mentioning that all the local adjustments achieved by applying this adaptive fusion strategy are data-driven, namely one can just avoid global modeling of the confounding bias function, which requires the exclusion of the global impact of extremely large confounding biases of the observational data that only exist in certain sub-regions of the feature space.
knowledge of the observational studies. Additionally, it also enables
We denote the partition obtained under criterion (9 as Qor -Q0o Cot., =Q,,,,,00,2, aos Y. and the corresponding tree-based estimator of the HTE defined as
Casl
fast (x)
4or (ot. E Qossh L
(10)
where the superscript "c" is to show that the RCT data used to construct the fused estimator at the leaf node is indexed by R and "fast" is acronym for the name Fused and Accurate Shrinkage Tree, which is due to the data fusion nature of the criterion (9), the shrinkage-type leaf node estimator (6) and its accuracy in terms of the MSE
3.3 Ensemble fusion
To reduce overfitting, improve robustness against outliers, and enhance generalization, we introduce the bagged version (Hastie et al.) 2009) of the PAST, referred to as the rIFAST, as follows: We randomly draw index sets RD of size n and O of size m, separately from and with replacement. **.(6)
We repeat the process B times, resulting in (R),()) (+(()) ) Then, B estimators fast (xx can be calculated based on the trial data indexed by R²,,0) and the observational data index by O+.())
fost (x). For the construction of the prediction intervals, We then define fr farf (x) = (1/B) E
(b)
see Zhhag et al. 2020).
4 Theoretical guarantee
In this section, we formally establish the benefits of the proposed split criterion (9) compared with the conventional criterion 77). To present the theoretical result, we first pose the following regularity conditions that are standard in literature (see e.g., Györfi et al. 2002 and SScrnet et al. 2015).
Assumption 2. (i) There exists positive constant X < ao such that B(exp(Arr)] i) co for i i= 0, 1. (ii) There exists positive constants Juin such that oi Var Y X S= 0) for any x E X.
Theorem 1 (MSE reduction of the proposed split criterion), Let (g. and θ [p] XR. Suppose the node that needs to be partitioned is Qj, under which the sample sizes of the trial data and observational data are nj and m,. respectively. Let (f) and Mof(B ) be the sum of MSEs of the conventional HTE estimator and the fused HTE estimator on the two child nodes of Q, split by O.
respectively. Denole bmax SuPzEQ, [(E())
S 0) E X x S 1)}l- Let Ô 77 and θ., be the solution of the proposed split
be the solution of the conventional split criterion criterion (9. Under Assumptions 1-2, we have
(i) For any 0 E O
Mor(O) M(8)
-
(11)
T2
min + n, 82
max

---

(ii) With probability at least 1 Oze for some positive constant C₁ < 00, it holds that
M(6) - M(00) I t + log(pn,) log"(n))
(12)
R2
(og(png) log (ng) I. log(pn,) log (ns)
and Moj (0.) (o,) S C3 for some positive constant C2. C3 < oo, where Or and Oog are oracle splits defined
(13)
mj
0 = arg min M (0) and 03 min Mor(O). dg
dE8
In the above theorem, the (i) part establishes a uniform MSE reduction result for any split choice 0 ④ of the proposed split criterion ( As revealed in (II), the criterion (9) leads to larger MSE reduction on the nodes with larger variance of Y and less bias of the observational data. In addition, the upper bound in [1T decreases as the node sample size n, decreases, implying that our proposed criterion leads to increasing relative benefits as the tree grows deeper. Besides, in the (ii) part we present non-asymptotic bounds for the discrepancies between the MSEs under the empirically estimated splits and the oracle splits, showing that the MSEs under the estimated splits can achieve a fast convergence rate. As a direct consequence of Theorem 1, the consistency of our final HTE estimator (10) can be established, since it is known from Scornet al. (2015) and Athey al. (2019) that the conventional tree-based estimator using only the trial data is mnennsquared consistent, and our proposed method leads a reduced MSE.
Proposition 1 (Consistency of ffast) For almost every x E [-1,1]], we have ffast(x)) + ((x) in probability as n, m - DO.
Experiments
In this section, we demonstrate the results of a series of experiments to answer the following two questions: (i) Whether the proposed method can effectively alleviate the impact of confounding bias of observational data and limited sample size of trial data; (ii) Whether the techniques we proposed including local fusion in tree leaves and adaptive fusion in partitioning are valid, respectively.
In consequence, we conducted experiments on both simulated and real-world datasets to verify the effectiveness of our method. We evaluated our method against both traditional tree-based and data fusion-based casual methods. The former includes the classical Transformed Outcome Honest Tree (HT) Athey and Imbens (2016) and its ensemble version Generalized Random Forest (GRF) Athey et al. (2019). The latter includes the simplest fusion estimator (SF) training both trial data and observational data together without distinction and the KPS estimators Kallus et al. (2018). In order to facilitate better comparison and understanding of our proposed method, we demonstrate three versions: the simple implementation, Shrinkage Tree (ST), described in Section 3.1 the improved version, Fused and Accurate Shrinkage Tree (FAST), described in Section 3.2) and its final ensemble version rfFAST described in Section 3.3 The results of each simulation experiment were based on B = 100 replications. The ensemble size for all the ensemble estimators was set to 100. For the tree estimators, the minimum number of observations required to be at a leaf node was set to and the maximum depth of the tree set 10.
5.1 Simulation
We conducted two sets of simulation experiments to evaluate the finite sample performance of the fused estimator and various baseline estimators. In both experiments, we first generated the pre- treatment covariates X = (X1, M11,4 X2, ... X,,) from Uniform[-] 1|2 and the unobserved variable U from NO, 1). Then, we generated the potential outcomes by (d) = dr dr(X)+ d X, + +.50 c(d), where 7 T(X)
0, Thus The treatment assignments for the trial sample of size and the observational sample of size m were generated as ))) Thus, the magnitude of B controls the strength of the unmeasured confounding: a larger ,3 leads to larger confounding bias. The test data X Xtest.j for S j S p were generated from Uniform(-1,1 with sample size 1000.
+ X2 + X + X² + X3 and c(d) N(0, 1) for
follows: ( S 1)- ~ (05() and (X U,S = 0) ~ Ber(1/(1 + exxp(=
7

---

In the first experiment, we aim to verify the effectiveness of the proposed data fusion strategy via an ablation study. We compared the robustness of the ST and the FAST against different levels of confounding bias parameter 3. Two baselines were considered: (i) the HT using only the trial data and (ii) the SF estimator obtained by directly merging all the available data and constructing Fit-Based Causal Tree (Athey and Imbens, 2016). We set the sample sizes of the trial data and the observational data be 7 200 and m 2000, respectively, the dimension of covariates = and / {0.1clc N,e < 19). The following three conclusions could be drawn from Figure 2 (I) When confounding bias in observational data was small, the simple fusion (SF) strategy can effectively improve the model performance. But when it became large, the SF was very vulnerable to confounding bias in observational data; (2) Even with the increase of 3, both ST and FAST consistently showed resistance to confounding bias; (3) FAST was significantly better than other methods including ST, which verified the effectiveness of our proposed split criterion (9) numerically.
Estimators
HT SF
ST
FAST
2.2
AMSE
1.8
Figure 2: The averaged root square error (RMSE) (mean with error bars) of the estimators on simulation datasets with different levels of the confounding bias parameter B.
In the second experiment, we evaluated the RMSES with respect to different n and B. We set m 2000 and p We included seven estimators in the analysis: The first two estimators were calculated purely based on the trial data: (i) the Transformed Outcome Honest Tree (HT) (Athey and Imbens. 2016) and (ii) the Generalized Random Forest (GRF) (Athey et al., 2019). The rest estimators were calculated using different data fusion strategies: (iii) the Shrinkage Tree (ST) estimator, the Fused and Accurate Shrinkage Tree (FAST) estimator, (v& vi) the KPS estimators (Kallus et al.) 2018) with a parametric (OLS) estimator and non non- non-paramettic (Random Forest) specification of the confounding function, respectively and (vii) the bagged FAST estimator (rfFAST).
Table T reports the RMSES of the estimators, conveying good estimation accuracy of both the FAST and its ensemble version rIFAST. Among the three individual estimators, the ST and FAST, exhibited superior performance compared the HT, and the FAST outperformed the ST. These relative performances provided support for the FAST approach compared to the classical honest regression tree, the proposed split criterion 9 and the shrinkage estimation strategy (6). which are implemented progressively. Among the three ensemble estimators, the rfFAST estimator demonstrated the best performance among all the six combinations of the trial sample size n and the confounding bias parameter 3. On the other hand, the performance of the KPS estimators appeared to be less stable. The KPS outperformed large (n 200). Under the non-parametric specification of the confounding function, the KPS nF did not gain benefit from incorporating the observational data and was consistently inferior to baseline estimator GRF.
GRF only when the trial sample size was relatively

---

Table I: The averaged RMSE (standard error in parentheses) of the estimators with respect to the trial sample size n and the confounding bias parameter B3. The best performance marked in bold.
n
HT
ST
FAST
GRF
KPS
KPS RF
rIFAST
1.89
1.84
1.33
1.73
0.84
0.5
(0.06)
(0.06)
(0.04)
(0.03)
(0.02)
2.28
1,90
1.85
1.19
1.29
1.65
0.89
100
1.0
(0.06)
(0.05)
(0.05)
(0.02)
(0.04)
(0.03)
(0.02)
2.05
2.02
1.28
1.71
0.98
2.0
(0.05)
(0.04)
(0.04)
(0.03)
(0.02)
1.87
1.71
0.96
1.56
0.73
0.5
(0.04)
(0.04)
(0.02)
(0.02)
(0.01)
2.20
1.98
1.83
1.12
0.97
1.59
0.84
200
1.0
(0.04)
(0.04)
(0.04)
(0.01)
(0.03)
(0.02)
(0.02)
2.08
1.97
1.01
1.57
0.92
2.0
(0.03)
(0.03)
(0.02)
(0.03)
(0.02)
5.2 Real-world data
In this sub-section, we report an analysis of the Tennessee Student/Teacher Achievement Ratio (STAR) Experiment (Krueger 1999) to demonstrate the proposed FAST for the HTE estimation. We aim at quantifying the treatment effect of the class size on the student's ' academic achievement.
Data description The STAR Experiment was randomized controlled trial conducted in the late 1980s. Students were randomly assigned to one of the two types of classes during the first school year: D 1 for small classes containing 13 17 pupils and f) = 0 for regular classes containing 22 25 pupils. The outcome Y is the average of the listening, reading, and math standardized tests at the end of first grade. The vector of covariates X includes gender, race, birth month, birthday, birth year, free lunch given or not, and teacher id. This made a universal sample of 4218 students, among which 2413 were randomly assigned to regular-size classes (D 0) and 1805 to small classes (D 1).
Ground-truth practice, the ground-truth truth 7(.) is not accessible, so we replaced it with an estimate calculated by a generalized random forest (Athey et al. 2019) based on all the 4218 observations.
Construction of the trial, observational and test data Following Kallus et al. (2018). introduced confounding bias by splitting the population over a variable which is known to strongly affect the observed outcome Y (Krueger] [999]: rural or inner-city (U
2811 students) and urban suburban (U' = O, 1407 students). The trial data were generated by randomly sampling a fraction of the students with U 1, where ranges from 0.1 0.5. The observational data were constructed as follows: From students with U = 1, we took the controls (D = 0) that were not sampled in trial data, and the treated (D 1) whose outcomes were in the lower half of outcomes among students with D = 1 and U 1; From students with U 0, we took all of the controls (D = 0). and the treated (D- 1) whose outcomes were in the lower half of outcomes among students with D 1 and U = 0. The test data consisted of held-out sub-sample of all the observations in the universal sample excluding the trial data.
Results We compared the performance of the rfFAST with various baseline estimators. In particular, the NF and the SF estimators were constructed using the Random Forest regressor. The NF' estimator utilized only trial data, while the SF estimator utilized both trial data and observational data together without distinction. As shown in Figure the proposed rfFAST method consistently outperformed other estimators.
6 Discussion
This paper explores the estimation of heterogeneous treatment effects (HTE) within the framework of causal data fusion. Drawing inspiration from the classical James-Stein shrinkage estimation (Green and Strawderman, 199T) approach, the authors introduce new method called Fused and Accurate Shrinkage Tree (FAST) that effectively incorporates observational data in both feature

---

Estimators NF SF
KPS(ridge)
KPS(RF)
riFAST
40
30
RMSE
20
10
0.1
0.2
0.3
0.4
0.5
h
Figure 3: The RMSEs of the five estimators with respect to different sample sizes the trial data, reflected by the fraction parameter h. A large means a large trial sample size.
space segmentation and leaf node value estimation. This new approach is shown to outperform existing data fusion methods via numerical experiments.
The above estimation framework can be generalized to any data fusion problem if there exists an unbiased estimator and a biased estimator of some functions of interest. would be worthwhile to explore the combination of the FAST method with other ensemble methods, such as the boosting and the grf-style (Athey et al., 2019) bagging, in addition Breiman-style (Breiman) (00T) bagging used in rfFAST. Moreover, extending the framework to handle time-series observational data would be an interesting direction for future research. Additionally, investigating statistical inference under the proposed fusion framework would also be valuable.
Acknowledgements
This work is supported by Ant Group through Ant Research Intern Program, and funded by National Natural Science Foundation of China Grants 92046021, 1207 and 12026607. We thank the editors and five anonymous referees their constructive suggestions and comments.
References
Agarwal, A., Tan, Y. S., Ronen, O., Singh, C., and Yu, B. (2022). Hierarchical Shrinkage: Improving the accuracy and interpretability of tree-based models. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato. S., editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 111-135. PMLR.
Angrist, D., Imbens, G. W. and Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 94444)):4445555
Athey, S. and Imbens, G. (2016). Recursive partitioning for heterogeneous causal effects: Table 1. Proceedings of the National Academy of Sciences, 113:7353-7360
Athey. S., Tibshirani, J., and Wager, S. (2019). Generalized random forests. The Annals of Statistics, 47(2):1148 1178.
Bareinboim, E. and Pearl, (2016). Causal inference and the data-fusion problem. Proceedings of the National Academy Sciences, 113:7345-7355
Breiman, L. (2001). Random Forests. Machine Learning, 45:5-32.
10

---

Breiman, L.,, Friedman, J., Olshen, R., and Stone, C. (1984). Classification And Regression Trees.
Chapman & Hall/CRC.
Buchanan, A. L., Hudgens, M. G., Cole, S. R., Mollan, K. R., Sax, P. E., Daar, E. S., Adimora, A. A., Eron, J. J., and Mugavero, M. J. (2018). Generalizing Evidence from Randomized Trials Using Inverse Probability of Sampling Weights. Journal of the Royal Statistical Society Series A: Statistics in Society, (91(4):1193-100)
Colnet, B., Mayer, I., Chen, G., Dieng, A., Li, R., Varoquaux, G., Vert, J.-P., Josse, J., and Yang, (2020). Causal inference methods for combining randomized trials and observational studies: review. arXiv preprint arXiv:201 1.00047.
Cui, Y., Pu, H., Shi, X., Miao, W., and Tchetgen, E. T. (2023). Semiparametric Proximal Causal Inference. Journal of the American Statistical Association, 0(0):1-12.
Dahabreh, I. J., Robertson, S. E., Tchetgen, E. J., Stuart, E. A., and Hernán, M. A. (2019). Gen- eralizing causal inferences from individuals in randomized trials to all trial-eligible individuals. Biometrics, 75(2):685-69 694.
de Luna, X. and Johansson, (2014). Testing for the Unconfoundedness Assumption Using an Instrumental Assumption. Journal of Causal Inference, 17-1199
Edwards, S., Lilford, R., Braunholtz, D., Jackson, J., Hewison, J., and Thortoo,, (1999). Ethical issues in the design and conduct of Randomised Controlled Trials. Health technology assessment (Winchester, England). 2:i-vi, 1.
Efron, B. (1979). Bootstrap Methods: Another Look at the Jackknife. The Annals of Statistics, 7(1):1 266
Efron, B. and Morris, C. (1973). Stein's ' estimation rule and its competitors an empirical bayes approach. Journal the American Statistical Association, 687341):117-13.
Frangakis, C. and Rubin, D. B. (2002). Principal stratification in causal inference. Biometrics, 58.
Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29(5);1 1232.
Glass, T. A., Goodman, S. N., Hernán, M. A., and Samet, J. M. (2013). Causal inference in public health. Annual review of public health, 34:61-75.
Green, E. and Strawderman, W. (1991). A James-Stein type estimator for combining unbiased and possibly biased estimators. Journal of The American Statistical Association AMER STATIST ASSN, 16:1001-1006.
Györfi, L., Köhler, M., Krzyžak, A., and Walk, H. (2002). A distribution-free theory of nonparametric regression, volume 1. Springer.
Hahn, P. R., Murray, J. S., and Carvalho, C. M. (2020). Bayesian Regression Tree Models for Causal Inference: Regularization, Confounding, and Heterogeneous Effects (with Discussion). Bayesian Analysis, 15(3):965 2020.
Hall, (1992). The bootstrap and Edgeworth expansion. Springer- Verlag New York.
Hastie, T., Tibshirani, R., and Friedman, J. (2009). The Elements of Statistical Learning : Data Mining, Inference, and Prediction. Springer series in statisties. Springer, 2nd edition.
Imbens, G. W. and Rubin, D. B. (2016). Causal inference for statistics, social, and biomedical sciences: An introduction. Taylor & Francis.
Kallus, N., Puli, A. M., and Shalit, U. (2018). Removing hidden confounding by experimental grounding. In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R., editors, Advances Neural Information Processing Systems, volume 31. Curran Associates, Inc.

---

Kosorok, M. and Laber, E. (2019). Precision medicine. Annual Review of Statistics and Its Application, 6:263-288
Krueger, A. (1999). Experimental Estimates Of Education Production Functions. The Quarterly Journal of Economics, 11::497-532.
Kuroki, M. and Pearl, J. (2014). Measurement bias and effect restoration in causal inference. Biometrika, 101(2):423-45 437.
Künzel, S., Sekhon, J., Bickel, P., and Yu, B. (2019). Meta-learners for estimating heterogeneous treatment effects using machine learning. Proceedings of the National Academy of Sciences, 116:4156-4163
Mentch, L. and Zhou, S. (2020). Randomization as regularization: A degrees of freedom explanation for random forest success. Journal of Machine Learning Research, 21(171):1-36.
Miao, W., Geng, Z., and Tchetgen Tchetgen, E. J. (2018). Identifying causal effects with proxy variables of an unmeasured confounder. Biometrika, 105(4):987-99)
Nasseri, K., Singh, C., Duncan, J., Korbblith, A., and Yu, B. (2022). Group wreeahel tree sums for interpretable modeling of heterogeneous data.
Nie, X. and Wager. S. (2020). Quasi-oracle estimation of heterogeneous treatment effects. Biometrika, 108(2):299-31)
Powers, S., Qian, J., Jung, K., Schuler, A., Shah, N., Hastie, T., and Tibshirani, (2017). Some methods for heterogeneous treatment effect estimation in high-dimensions, Statistics in Medicine, 37.
Radcliffe, J. and Surry, P. D. (2012). Real-world uplift modelling with significance-based uplift trees.
Rosenbaum, P. and Rubin, D. (1983). Assessing Sensitivity to an Unobserved Binary Covariate in an Observational Study with Binary Outcome. Journal of the Royal Statistical Society. Series (Methodological), 45(2):212-21
Rubin, D. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66.
Scoree,, E.,, Biau, G., and Vert, J.-P. (2015). Consistency of random forests. The Annals of Statistics. 43(4):1716 1741.
Shi, X., Miao, W., Nelson, J., and Tchetgen, F. (2020). Multiply robust causal inference with double-negative control adjustment for categorical unmeasured confounding. Journal of the Royal Statistical Society: Series B (Statistical Methodology). 82.
Shi, X., Pan, Z., and Miao, W. (2022). Data integration in causal inference. WIREs Computational Statistics, 15(1).
Taddy, M., Gardner, M., Chen, L.,, and Draper, D. (2016). A nonparametric bayesian analysis of heterogenous treatment effects in digital experimentation. Journal of Business & Economic Statistics, 34.
Tang, C., Wang, H., Li, X., Cui, Q., Zhang, Y-I.., Zhu, F., Li, I.., Zhou, J., and Jiang, L. (2022). Debiased causal tree: Heterogeneous treatment effects estimation with unmeasured confounding. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A., editors, Advances in Neural Information Processing Systems, volume 35, pages 5628-5640. Curran Associates, Inc.
Turney, K. and Wildeman, C. (2015). Detrimental for some? heterogeneous effects of maternal incarceration child wellbeing. Criminology Public Policy, 14.
Wager, and Athey, S. (2018). Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, :13(52)):1228-122 1228-1
12

---

Wu, L. and Yang, S. (2022). Transfer learning of individualized treatment rules from experimental to real-world data, Journal of Computational and Graphical Statistics, 0(0):1-10.
Yang. S., Zeng, D., and Wang, X. (2020). Improved inference for heterogeneous treatment effects using real-world data subject to hidden confounding. arXiv preprint arXiv:2007.12922.
Zhang, B. and Tchetgen Tchetgen, E. J. (2022). A Semi-Parametric Approach Model-Based Sensitivity Analysis in Observational Studies. Journal of the Royal Statistical Sociery Series A: Statistics in Society, 185:S668- 1691.
Zhang, H., Zimmerman, J., Nettleton, D., and Nordman, D. J. (2020). Random Forest Prediction Intervals. The American Statistician, 74(44:392-406.
13

---

APPENDIX
A Additional figures
Estimators HT SF ST FAST
4.0
3.5
RMSE
3.0
2.5
2.0
0.0
0.5
1.0
1.5
2.0
Figure 4: The averaged root mean square error (RMSE) (mean with 2xs.d. error bars) of each algorithm on multiple simulation datasets with different levels of the confounding bias parameter B.
B Pre-processing of the real-world data
In the STAR dataset, each of the pre-treatment covariate X; (1 و j p) was standardized to range of -1 to 1, and the outcome variable Y was standardized to a range of to 100.
C Proof of Theorem
The proof follows the similar arguments as in Györfi et al. 2002) and Scornet al. (2015). It
sufficient to show the result at the root node given the recursive nature of the partitioning. We
will use the following notations in the sequel. We denote Er, Pr and Eo, PP as the expectation and probability under trial data and observational data, respectively. We let Z (X,Y), Y). For
any E [p] and с E IR, let 0 (q,c) and the corresponding two partitioned notes are denoted as
Qr. {ar|x x. S c) and (0) = {I/ > c). The parameter space of is denoted O [p] x R.
Let jur, and un be the predictions for on (0) and ( respectively and denote 7 = (Tz. TR).
Let (0) and (0) be the MSEs of the conventional HTE estimator and the fused HTE estimator on the child nodes of (0), respectively, for E (R, L).
(i). For any θ, according to Equation (2) the main paper we have
(0 w. (O))
for E (R, L), where the weight (0) satisfies
(2.(nn) (+) of (Q.)nn (2)
by Equation (5) in the main paper, which lower bounded by amin
where o2 <
minn
min
X X S S 0) and bmax supæeQ, [E x. S EY X S 1)}
14

---

Therefore, we conclude that
Maf (6)
o2 min nb max
-15
T min
which reveals the MSE reduction effect of the proposed split criterion.
(ii). The proof includes two parts. In Part I, we will derive the bounds for the discrepancies between the MSEs under the empirically estimated split and the oracle split under the conventional criterion, and in Part 2 the similar results under the proposed split criterion.
Part 1. We define the following criterion function:
1 - Σ ( (Vo.. - Txo) E (Q((0) (Yo,- - T E Qn(0))}
e, (0, (O,,,,
12 i=1
=: c/ (0, TL, Ri,)) + (0, Tn. R())
For E {L, R), let
L (0, T;) = Er { (0, T2 R(}} and [(),T) C TL. + (0,
(14)
Then ('(0, T) represents the MSE of on the region (7))). For given split 0 = (q, it is straightforward to see that the optimal (0) (0z Tn is given
T (0) = arg min e, (0, Tu. R = En fol 2,00)
RER
for i E {L, R). which is the sample mean of Y on the region (.) Therefore, by the definition of (0). holds that (0, (0)) (0) for E {L, R). The optimal split 0g (go, co) on the population level is defined via minimizing the profiled criterion function:
(go, co) = arg min (M- + (R) ,E(r),eRR
arg min (0).
gE|p),eR
Define (0) e, (0. (0). R for E {L, R) and the empirical optimal split Ø () is defined via minimizing the sample criterion function:
arg min (0) + (2)(0)} =: arg min M, (0).
qEip),CER
gE(p),cER
Step 1 (Main error decomposition).
Now we will bound M0) (fo), which represents the discrepancy of the MSEs of the oracle
and empirical split. To apply empirical process theories for stochastic error analysis, we will
use a truncation argument. We let Mi, (0, n., R³) E, (T Y Ta. ( E ())
and M2, (0) = Er Mr MM, (0, (0), R₂) where TB. x =: (12l A (x)sign(z) for any 8n > 0. Correspondingly. let Ms, (0) M4, (0) + M (0) and Mn,9, (0) = M a. (0) + MR n, (0). Then we have the following error decomposition:
0 << M (0p))
-M(0) - Ma. (0) - M(0b) + MA, (0o)
R, (0) + 2M, R,, (0o) + MR, (0) MMa, (06) - 22,,,
+ 2M (0) - (2), - -2Mm,.. (0o) + 2M, (0n)
+ 2M,, (0) 2M,
= S1, + S2,, + S3,n + S4,a-
By the definition of we have S4,, S O. In following steps, we will bound St,., S2,n and Sa respectively. The truncation level 8, chosen Bn 30 log(n) for 3o 20y.
15

---

Step 2 (Bounding S1,x). For any 0, it holds that
M (0) M3, (0) =Ir { - -;())) (Te, v - T3. ((0)) - E 2.(01)}
Er { (ir - T3 8) +T,,
2(01))
+ Br (T) - (.) + ()) E Q.(0))
5, + S6,
For Ti by Sehway-Schwarz inequality have
[8tml S V VT Ts, Y>, BY + +T, Y 27) I VET(Y Tp, Y)2 where the second inequality is because ET ET(P2) S and Er { < (T((Y)/1Q Since
exp(or 129/2)
I(| > Bx) v
0r/2122
therefore,
|Tial $ By -Tp, vy < v Br Mrp Sypeeppor//00 (
2
Tr
Ey exp(or|l[() exp(
oy 1y/2//2
oy
4
Since Ey exp(or/MM)) < and 8n 8o log(n,, we conclude that |St,a| S 4. With the same argument, we have So,n implying that
M(0) Ms, (0) x
(15)
for any 0 E θ. Therefore, the truncation error S1, S 1 n.
Step 3 (Bounding 22
Let MN.o/ = {f (Ta. - E Q.(0)) :0 = (g, [p] x R). By applying Lemma we obtain
cBn
(6 MN S (pn)²
where 2/ any set (21,"",,,) E [0, 1|2 x y and > is a universal constant. follows from
Lemma 1 that
30 E : MM, (0) Mn,e. (0) 2 aa + Ma. (0))
Pr
/80c8h
<28(pn)²
exp
128484
(- (-0p)
S exp
an
+ -log(pp)) -llogl)))))))))))(())))
B
Taking 7 = 1/n and a = (t + log(pn)) implies that with probability at least Cie for
some universal constant C1 0.
v0 E θ, MM (0) 2M,,s, (0) $ I + log(pn) logg(n)
(16)
n
Therefore, we conclude that with probability at least 1 - Cie the stochastic error S2,1 S {t log(pn) log (n)) /n.
16

---

Step 4 (Bounding S, According to 15
we have
v0 E + Er (M (0) (,)( 2
Since Y is sub- Gaus-an by assumption, it straightforward to see that (Ta, y - T3. (0)) E Q:(+)) and (Y - (0)) E Q:(+)) are sub-exponential for i E (L. R] Suppose
S oo for all 0 E 6, where I1l,,,, is the sub- cubbeexonential norm operator. By applying Bernstein's 's inequality, for any >> O, we have
(Tg, Y - Ta. (X E (()))) I do and (8 (0)) E Qi(0))
Pr { MM (0) (0)- - Er (Mas (0) (,) 2 ${}
"ns²
<2exp (-emin
o de
for i E {R, L), where c > 0 is a universal constant. Taking 8 = out C2t, for any 2 0 we obtain
Pr M| (0) M Er (M (0) M,) 2 C2tl} exp(
(17)
for any n > l/c. Since the above result holds for any E O, we conclude that for any >> 0, with probability least 1 4e- + have Sa,n X (t 1)/n.
Combining the results on SI.. S2nn and S3,n. we conclude that for any > O, with probability at least 1 - Cze it holds that
t log(pm) log (n)
(0, (0) C; x(0))) x
(18)
n1
for some universal constants C3, C4 > 0.
Part The proposed scale criterion can reformulated as follows. For E {L, R}, let
(0) = {1 - w. (0)) (1% (0)) Q:(0)) (0) =w: -
e (,))) and
where (0) E, (Yo X. (0)) and r1, E.. (.)), and =,(0)= = 1:000-0000(00000/000000000000000
where o²)):(0)) Var, (TO.: (0)) (f)),
(TI. (0)) and ((Q) = (0) To.
Let Fe. (F,, for (0,1) and =.
(0),, the population criterion
is defined as Mo)(0) Fol0) + (O). For the empirical criterion, we first define F0,
E, (F.R()) and F (0) = (P Let (O) = (0) + (0)) and F (0) = (m) + Fm, the empirical criterion is the denoted as MN.or(0) = MN (0) + Fm (0). The
population and empirical optimal splits are defined by
Oof = arg min Mof and 0y arg min (N.of(0). Dee
dee
We first have the following error decomposition:
Moj (0) (0,) = (0) Mor.a, (0.r) + Mor(0o) Mos.s, (0o1)
+ Mor.o, (Oor) + Mor.a. (Dor) 2M..of. (Bor) + 2M oo (0os)
2M of.. (0or) 2M (B.)) 2M 2M, + (M.,)
+ 2M oo (00)) 2MN
TT, + T2,n Ts,n + Ta.n-
By the definition of 0oy, we have Trn < 0. In the following steps, we will bound Ti T2, and Ta respectively. Following the same argument for S1,. it can be obtained that T X 1. n.
17

---

We now bound T2.n Let On = {g s(v.z) = w()w( - T)[( E Q e θ,)} and
Hr, (h h(y, x) vw(O)(ū 7)((x Q 8 E Q.)). then via Lemma 2 we have
cBn
(n) S (mm)
and (6, Hn, =f) < (pm)²
for any where 21 is any set f21, ,2, E [0, 1_ y and c > 0 is universal constant. follows from Lemma that for any 0n, 0
Pr {
Fu, (0) 2 (a + + Moraa, (0))
30 E On. |Fo,s, (0)
<28(pn)
880.h
an
exp
(1128483
71
.1n
eexp
log(pn) -log(11)
84
Taking 71 1/n and 01 (t + log(pn)) implies that with probability at least C4e for
some universal constant
C4 > 0,
40 E .n [ M. (0) nF,n (0)1 t log(pn) log
(19)
n
Similary, for any 02 122 > 0,
1 On 1F1.8n (0) - (0))
Fo 30
2 (a + FI.
(- 0.m (0a1m + log(mn) - log(?1)
S exp
31
Taking 72 = 1/n and a2 = (t + log(pn)) / //mm implies that with probability at least Cse for
some universal constant Cg > 0,
v0 E On, Fi, (0) - mFm.... (0)1 S + log(pn) log (n)
(20)
Combining
19 and 20 delivers that with probability at least 1 - 20ie log(pn) log (n) + log(pn)) log (n)
Te.
(21)
for ant t O, since Pof 0, E θ, and Mos.o, Fo,g, (0) + F1,0, (0) and MN.of (0) Fr,s,. (0) + Fm, B.. (0) for any 0 E On.
Now turn to T3.n, the truncation error for the empirical loss. With the similar argument in (16 we have with probability at least 1 4e- it holds that T3, S (t + 1)/n + (t 1)/m for any
Combining the results for Ti,,, T2. and Ta.,, conclude that for any ł > 0, with probability at least 1 C.
log(pm) log" (n) +
+ log(pn) log"(1)
(o, (os(O S
(22)
m
which completes our proof.
D Supporting lemmas
The following to lemmas are from Section 11.3 and Section 13.1 of Györfi et al. (002). which are useful for our proofs.
Lemma 1. (Deviation inequality of quadratic process). Suppose that G is class of uniformly bounded functions {g: Rd R. holds that
|l S M). Let F {g² E g). Then for any n 2
[3f F IE [f(z)} E,, (f(2))] s(a B(f()))
g, x exp (- 214(1 + +
s2(1 e)om
78
28 Ni
z/¹ 20.M'
where z/ (2), 2x) E R a,7 7 0 and 0 < < 1/2.
18

---

Lemma 2. (Covering number of piece-wise constant fumctions). Let II be the family of partitions of [0, 1P. For any set 2 {.,.-.. In of x induced by elements of. Let (IT) be the maximal number of sets contained in partition
1 [0, let (II) be the maximal number of partitions
IT. Denote the piece-wise constant functions on 1 be F(IT) with |[ S Bn for any
f E F(T). Then using Lemma 13.31 and Theorem 9.4 of Györff et al.
2002 we have
(c1
(1
8
for any ô > O, where C1 > 0 is some univiersal constant. Specifically, in each partition for node Cj of tree, have M (1I) and (II) S (pon)² where an is the sample size of Ct.
19


Reviews:
[{"review_id": "RbFaqNBsF6", "reviewer": "Reviewer_YpW7", "summary": "This paper considers the problem of estimating (heterogeneous) treatment effects via both interventional and observational data. The authors proposed a new estimation, namely the Fused and Accurate Shrinkage Tree (FAST), which optimally weights the interventional and observational estimator, and combines with a new spilt criteria for tree-based heterogeneous treatment effect estimation. The authors further conducted experiments to compare FAST against existing methods.", "questions": "See above.", "limitations": "The authors has somewhat discussed the limitations of the method.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Apart from a few typos, the paper is well written and ideas are presented in a rigorous but clear also way. \n- The idea of applying shrinkage method to combine interventional and observational data for better estimator is novel and could be a nice addition to the literature.\n- Note that I have not gone through all the proofs in the appendix, the mathematical correctness might need further input from other reviewers. ", "weaknesses": "\n\nGenerally I like this paper, but there are still a few weaknesses.\n\n- The main issue with shrinkage method is interpretability: we need to understand better how the variance-bias trade-off behaves in different regimes, especially the authors takes a more analytic way to first estimate the required quantities, then solve the optimal weights. More specifically for example, it would be beneficial to at least see different how trial mechanisms affects the estimation. For example, in a more realistic setting, one may consider non-randomized trials rather than RCT, in which treatments are assigned by a *known* true model. By adjusting the parameters of such true assignment model, the estimator variance for the trial population HTE estimator can be controlled (even with fixed N). Then the performance of FAST can be evaluated against different variance regimes of trial HTE estimator, which will help us understand the sweat spot of the method.\n\n- Regarding experimental settings. It is indeed quite standard for these type of papers to have 1 or 2 synthetic experiments and 1 real data experiment. However, in the case of this paper I found the simulation setting is a bit weak. It would be great to perform experiments on multiple data generating mechanisms with randomly sampled parameters and coefficients, allowing us to evaluate the marginal performance of the method. Otherwise the authors at most demonstrated the capability of the method on only one single data generation mechanism (which arguably is much easier to hack/cherry-pick).\n\n\n- The other potential room for improvement is the baseline. I understand that the paper mainly only compares to data fusion methods. However, due to the variance-bias trade-off of the shrinkage method, it would be natural to also expect some comparisons to variance reduction methods for trial estimators as well. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your insightful and comprehensive review. We are encouraged by the overall positive responses from the comments and suggestions. Our point-to-point responses to your comments are itemized below and we would add the discussions and numerical experiments in the revised manuscript if given a chance.\n\n**W1:** Our apologies for any confusion that might have arisen due to the unclear illustration. It is worthwhile to first  note that for any regular unbiased estimator $\\hat{\\tau}_u$ based on the trial data and (potentially) biased estimator $\\hat{\\tau}_b$ based on the observational data, the variance of both estimators should admit the following expressions:\n\\begin{equation}\n\\sigma_u^2 = \\mathrm{Var}(\\hat{\\tau}_u) =  \\frac{C_u}{n} \\ \\hbox { and } \\ \\sigma_b^2 = \\mathrm{Var}(\\hat{\\tau}_b) = \\frac{C_b}{m},\\nonumber\n\\end{equation}\nwhere $C_u, C_b$ are positive constants, and $n$ and $m$ are the sample sizes of the trial data and the observational data, respectively. \n\nIndeed, as mentioned in your comment, by applying various methods including adjusting the trial mechanisms (either randomized or non-randomized), one is able to control the variance for the trial population estimator $\\sigma_u^2$ only through the constant factor $C_u$ without changing the order $n^{-1}$. But the implementation of our shrinkage method is independent of the trial mechanism. To see this, for a given trial mechanism and estimation method (mapping to a $C_u$), one can readily apply the shrinkage method to obtain the optimal fused estimator $\\hat{\\tau}\\_{w^*}$, whose mean square error (MSE) is $(1 - w^*)\\sigma_u^2$. Now,  the relative improvement of our proposed estimator $\\hat{\\tau}\\_{w^*}$ over the original trial estimator in terms of MSE is as follows:\n\\begin{equation}\n1 - \\frac{\\mathrm{MSE}(\\hat{\\tau}_{w^*})}{\\mathrm{MSE}(\\hat{\\tau}_u)} = w^*, \\hbox{ where } w^* = \\frac{\\frac{C_u}{n}}{b^2 + \\frac{C_u}{n} + \\frac{C_b}{m}}.\\nonumber\n\\end{equation}\nFrom the above equation, one can find that $w^*$ is not only an optimal weight facilitating the shrinkage estimation, but it itself also characterizes the extent of improvement in terms of MSE one can expect via incorporating the observational data. Thus, in real-world applications, one is able to get an estimate of the improvement of the procedure once $w^*$ is estimated. \n\n**W2:**  In response to this suggestion, we designed the following two data-generating processes (DGPs). In both DGPs,  we generated the pre-treatment covariates $X = (X_1, X_2, \\cdots, X_p)^T$ from $\\mathrm{Uniform}[-1,1]^p$, $U$ from $N(0,1)$, $D|(X,U, S=1) \\sim Ber(0.5) $ and  $\\epsilon(d) \\sim N(0,1)$.\n\n-  (I) \n$$    Y(d) = d\\tau(X,\\beta_{\\tau}) + X^T\\beta_{\\ell} + \\beta_U U + \\epsilon(d), \\beta_{\\tau} = (\\beta_{\\tau,1},\\beta_{\\tau,2},\\cdots,\\beta_{\\tau,5})^T \\sim N(1_{5\\times 1}, 0.5^2I_5) , \\beta_{\\ell} = (\\beta_{\\ell,1},\\beta_{\\ell,2},\\cdots,\\beta_{\\ell,p}) \\sim N(1.5_{p\\times 1}, 0.5^2I_5),\\beta_U \\sim N(1.5,0.5^2),\n $$\n$$\n \\tau(X,\\beta_{\\tau}) = \\beta_{\\tau,1} + \\beta_{\\tau,2}X_1 + \\beta_{\\tau,3}X_1^2 + \\beta_{\\tau,4}X_2 + \\beta_{\\tau,5}X_2^2, D|(X,U,S=0) \\sim Ber(1/(1+\\exp(\\beta U + \\beta_oX_1))), \\beta_{o} \\sim N(1.5, 0.5^2)\n  $$\n- (II) \n$$Y(d) = d\\tau(X, \\beta_\\tau) + \\beta_{\\ell,1}\\sin{X_1} + \\beta_{\\ell, 2}\\cos{X_2} + \\beta U + \\epsilon(d), \\beta_{\\tau} \\sim Uniform(0.5,1.5)^p, \\beta_{\\ell} \\sim N(1_{2\\times 1}, 0.5^2I_2), $$\n$$\\tau(X,\\beta_{\\tau}) = 1 + \\sum_{s=1}^p\\beta_{\\tau,s}\\left(X_s+X_s^2\\right), D|(X,U,S=0) \\sim Ber(1/ (1+ \\exp(\\beta U + \\beta_{o,1}X_1 + \\beta_{o,2}X_2))), \\beta_{o,i}\\sim N(1.5,0.5^2).$$\n\nIt is noted that in (II), the factor $\\beta$  appears in both  $Y(d)$  and $D|X, U, S=0$. We repeated each DGP 100 times and the results are presented in Table 2 in the additional PDF. In both DGPs, the proposed $\\mathrm{rfFAST}$ estimator performed the best in most scenarios. Furthermore, its advantage over other data fusion methods was more obvious when the trial sample size was small ($n=100$),  which is exactly when data fusion is necessary to improve the causal effect estimation in real-world applications.\n\n\n**W3**.  Thanks for your understanding of our focus on comparing with data fusion methods. Following your suggestion, we have done another round of literature review regarding variance reduction methods for trial estimators:\n  \n- Li, F., Morgan, K.L. and Zaslavsky, A.M. (2018) Balancing covariates\nvia propensity score weighting. Journal of the American Statistical\nAssociation, 113(521), 390–400. \n -  Sturmer T. et al., (2021). Propensity score weighting and trimming strategies for reducing variance and bias\nof treatment effect estimates: a simulation study. American journal of epidemiology, 190(8):1659–1670. \n- Liao, J. and Rohde. C. (2022). Variance reduction in the inverse probability weighted estimators for the\naverage treatment effect using the propensity score. Biometrics, 78(2):660–667. \n    \n\nWe believe that these references will definitely enrich our literature review and make it more comprehensive. However, two points are also worth mentioning: First,  while indeed the $\\mathrm{HT}$ estimator based on the inverse probability weighting (IPW) is sometimes considered to be unstable,  the $\\mathrm{GRF}$ estimator namely the generalized random forest estimator already have certain satisfying statistical properties verified both theoretically and empirically (Athey et al., 2019), where implicit variance reduction techniques have been applied. The other one is closely related to the core feature of our shrinkage method. That is, the proposed framework is essentially built upon the trial and observational estimators, namely the implementation of the procedure is independent of the construction of the trial estimator. In real applications, the practitioners can first construct a variance-reduced trial estimator designed for their particular scenarios, then estimate the optimal weight $w^*$ to facilitate the shrinkage estimation. \n"}, {"review_id": "qIPjXFYrLb", "reviewer": "Reviewer_tnj1", "summary": "The paper introduces a Fused and Accurate Shrinkage Tree (FAST) algorithm for heterogenous treatment effect estimation given trial and observational data. The FAST algorithm introduces (i) a shrinkage based approach that combines trial and observational data for MSE reduction in treatment effect estimation, and (ii) a split criteria which down-weights observational data with high confounding bias. Further, the paper provides theoretical analysis demonstrating the benefits of the proposed split criteria. Experimental results on synthetic and real-world data demonstrate that the proposed approach outperforms baselines per metric MSE.", "questions": "**Experiments**\n- Table 1: Could you include results for the NF estimator (using trial data only) and SF estimator (both trial and observation data)\n- Figure 3: Could you comment on why NF is not monotonically decreasing with sample size? Shouldn't we expect the rFAST to NF estimators to cross when $n$ is large enough?\n\n**There are some limitations with tree-based methods that the paper does not adequately address:**\n- How sensitive is the FAST algorithm to the dimension $p$ and heterogeneity of the covariates? The experiments set $p=5$, which is unrealistic in real-world scenarios.\n- What is the complexity, e.g., training time of the rFAST algorithm compared to baselines, and how easy is it to scale given high-dimensional heterogeneous covariates?\n\n\nMinor:\n- Figure 1: Could you add more details summarising the plots in the caption\n", "limitations": "The limitation discussion is inadequate. I encourage the authors to add a paragraph discussing the limitations.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- The paper is well written and easy to follow. The reviewer enjoyed reading this paper.\n- The proposed FAST algorithm is well-justified and the theoretical analysis might be of interest to some readers. \n- The paper tackles an important problem (fusing small RCT with large readily available observational data ) with many applications. \n- The algorithm seems simple and easy to implement", "weaknesses": "-  Eqn. 5:  The paper seems to have glazed over the rationale for dropping $\\sigma_b$ in the shrinkage estimator. It's unclear in what scenarios, e.g., how large the observational sample size must be for $\\sigma_b$  to become negligible.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your inspiring review comments. We are greatly encouraged by the overall positive responses from the comments. Our point-to-point responses are itemized below and the references are listed in the end. We would add the discussions and numerical experiments to the revised version if given a chance.\n\n**W1:** Thanks for pointing out the missing details in dropping the $\\sigma_b$ term. In fact, as pointed out in Section 3.1, both  $\\hat{\\tau}_u$ and  $\\hat{\\tau}_b$ are approximately sample means, so the variances of both estimators should admit:\n$$\n\\sigma_u^2 = Var(\\hat{\\tau}_u) = \\frac{C_u}{n} \\ \\hbox { and } \\ \\sigma_b^2 = Var(\\hat{\\tau}_b) = \\frac{C_b}{m},\n$$\nwhere $C_u, C_b$ are positive constants, and $n$ and $m$ are the  sample sizes. Thus, if we denote the dropping-$\\sigma_b$ version of $w^*$ as $\\tilde{w}^*$, then\n$$\n|1 - \\frac{\\tilde{w}^*}{w^*}| = |1 -\\frac{b^2 +\\sigma_u^2}{b^2 + \\sigma_b^2 + \\sigma_u^2}| = |\\frac{1}{1 + m(\\frac{b^2 + \\frac{C_u}{n}}{C_b})}| = O(\\frac{1}{m(b^2 + n^{-1})}).\n$$\nThus for a given tolerance $\\epsilon > 0$,  the condition for the obs sample size $ m > \\frac{1}{\\epsilon (b^2 + n^{-1})}$ suffices for $\\sigma_b$ to become negligible. \n\n\n**Q1:** We have added the results of the SF estimators (the $HT\\_{SF}$ and the $GRF\\_{SF}$) in Table 1 of the additional PDF. Besides, both the $HT$ and $GRF$ estimators in the original Table 1 are in fact  NF estimators. We added the subscript ''NF'' to address this aspect. From the new Table 1, the estimator $GRF\\_{SF}$ performed inferior to the $GRF\\_{NF}$ in all scenarios. And the single-estimator method $HT_{SF}$ exhibited smaller MSEs compared to the $HT_{NF}$ only when $\\beta$ is small. Both SF estimators performed worse than the  $rfFAST$ estimator. \n\nSecond, unlike other fields of machine learning,  the actual causal effect is typically inaccessible in causal inference. So we used the generalized random forest estimator trained on the full dataset as a surrogate for the underlying inaccessible ground truth. And similar approaches have been used in literature (Kallus et al., 2018). Thus, the nonmonotonicity is largely due to the intrinsic randomness of the data. Besides, the $\\mathrm{NF}$ estimator in the Figure does show a downward trend, which is consistent with our simulation results if we check the column of the $\\mathrm{GRF}_{NF}$ estimator of Table 1 of the additional PDF. Besides, as the theoretical MSE of $\\mathrm{rfFAST}$ estimator is $(1-w^*)\\sigma_u^2$ and that of the $NF$ estimator is $\\sigma_u^2$, the proposed fused estimator $\\mathrm{rfFAST}$ should always perform better than its no fusion counterpart $NF$.\n\n**Q2:**.  First, since the $FAST$ is essentially a tree-based method, its sensitivity to the dimension $p$ and heterogeneity of the covariates largely resembles that of the classical decision tree. And one of the appealing advantages of tree-based methods compared with other nonparametric methods is that they can be flexibly adaptive to high-dimensional and complex features, as revealed both empirically (Archer et al., 2008) and theoretically (Chi et al., 2022).  To see this, we conducted an additional experiment  presented in Table 3 in the additional PDF. We did not include $SF$ methods to save space. And $rfFAST$ estimator was quite robust against increasing  $p$.\n\nSecond, we admit that our current code is only used to implement the method prototype in Python, and we have not done much optimization. And the actual running time of the algorithm depends on many factors, such as the different implementation languages (GRF is based on C++), histogram preprocessing acceleration like LightGBM, and parallelization. Therefore, here we analyze the theoretical time complexity.  For data with $n+m$ samples and $p$-dimensional features, the time complexity of building a tree by our method is $O(p\\cdot (m+n)\\cdot\\log((m+n))+p\\cdot n\\cdot\\log(n))$, where the extra overhead compared to the decision tree is mainly due to calculating $w^*$, that is, the second term. But according to equation (2), $w^*$ has a closed-form solution, and the computational complexity is $O(pn\\log n)$. Given that $n\\ll m$, the overall computational complexity $O(p\\cdot (m+n)\\cdot\\log((m+n))$ is approximately the same order as that of the traditional decision tree. Therefore, there is no significant difference between the two methods.\n\n**Q3**. We re-wrote the caption of Figure 1:  ''The probability density functions (pdfs) of the unbiased estimator (pink) and the biased estimator (blue) in the left panel and the pdf of the shrinkage estimator under the optimal weight $w^*$ (green) in the right panel. The vertical dashed line represents the true parameter value $\\theta^* =0$. '\n\n**Q4**. We added a paragraph discussing the limitations: Our work also has several limitations. First, as mentioned above, our method currently can not provide a confidence interval for the fused estimator.  Second,   we opt for the mean square error (MSE) criterion to attain an optimal balance between the variance of the trial estimator and the bias of the observational estimator. This choice is motivated by the existence of a readily obtainable closed-form expression for the optimal weight $w^*$ under this criterion, thereby enhancing its interpretability. The consideration of alternative criteria for optimization remains untouched.\n\n\n**Refs:**\n- Archer, K. J. and Kimes, R. V. (2008). Empirical characterization of random forest variable importance\nmeasures. Computational statistics & data analysis, 52(4):2249–2260.\n- Chi, C.-M., Vossler, P., Fan, Y., and Lv, J. (2022). Asymptotic properties of high-dimensional random\nforests.The Annals of Statistics, 50(6):3415–3438\n- Kallus, N., Puli, A. M., and Shalit, U. (2018). Removing hidden confounding by experimental grounding. In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc."}, {"review_id": "cZ4lKJeqOr", "reviewer": "Reviewer_7Nsn", "summary": "The authors propose a novel shrinkage method that fuses an unbiased estimator with a biased estimator. This method effectively reduces the MSE of the unbiased estimator. The approach offers a practical and straightforward implementation specifically tailored for estimating heterogeneous treatment effects. The authors extend the conventional node split criterion to align with the fused estimator and penalizes the use of observational data with substantial confounding bias. The authors also provide a theoretical analysis that explains the advantages of the modified splitting criterion.", "questions": "The sample sizes considered in the experiment are always large. If a sample from RCT of size 100 or 200 is avaliable, the baseline models would do the job. I'm more curious in the senario when we have a relatively small RCT sample and that we do need more data from obervational studies to enhance the estimation.", "limitations": "No other limitation.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The application of the weighting strategy from shrinkage estimation to fusing unbiased and biased estimators in order to reduce the MSE of the unbiased estimator is a great idea.\n- The modification of the node-splitting criterion that aligns with the fused estimator is an excellent enhancement to the methodology.\n- The paper is well-organized and thanks to the authors's thoughtful and consistent notation, the methodology is easy to follow.", "weaknesses": "I'm concerned with the omission of $\\sigma^2_b$ in practice. If we only look at the weight $w$, it make sense if $\\sigma^2_b$ is small comparing to $\\sigma^2_u$. However, in the tree building process, we need working estimates of the MSE of fused estimator, $\\frac{(\\sigma^2_b+b^2)\\sigma^2_u}{\\sigma^2_b+b^2+\\sigma^2_u}$, and I don't think omitting $\\sigma^2_b$ is justified by the same reason anymore.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your instructive review comments. We are greatly encouraged by the overall positive responses from the comments and suggestions. Our point-to-point responses to your comments are itemized below and we would add those discussions in a revised manuscript.\n\n**W1:** Thanks for pointing out the issue. The MSE of the fused estimator can be equivalently expressed as \n\\begin{equation}\n\\frac{(\\sigma_b^2 + b^2)\\sigma_u^2}{\\sigma_b^2 + b^2 + \\sigma_u^2} = (1 -\\frac{\\sigma_u^2}{\\sigma_b^2 + b^2 + \\sigma_u^2})\\sigma_u^2 = (1-w^*)\\sigma_u^2,\\nonumber\n\\end{equation}\nso working estimates of the MSE of the fused estimator amount to estimating the optimal weights $w^*$ and the variance of the trial estimator $\\sigma_u^2$. Thus, we would think that the same reasoning can be applied to both the estimation of $w^*$ alone and the MSE of the fused estimator. This also can be considered to be a significant benefit of applying the proposed shrinkage method: in real applications, once the optimal $w^*$ is estimated, one can immediately obtain an estimate of the MSE of the corresponding fused estimator. Furthermore, the relative performance improvement of the fused estimator over the original trial estimator is readily at hand -- it is expected to be exactly $w^*$.\n\n**Q1:**      Thanks for raising this issue. Indeed, as mentioned in your comment, one motivation for developing data fusion methods is to tackle the inadequacy of the RCT sample in real applications. Following your suggestions, we have enriched the original Table $1$ (see Table 1 in the global response) by adding a scenario when the trial sample size $n$ is reduced to $50$. We added the subscripts ''NF'' and ''SF'' to represent the trial-data-only and the simple fusion method (the two simple fusion methods were also included for completeness), which makes it easier to compare the performances of all the methods presented in the Table.\n    When $n=50$, the $\\mathrm{rfFAST}$ still dominantly outperformed other data fusion methods and led to at least a 20 percent improvement in performance compared with its no fusion counterpart $\\mathrm{GRF}_{NF}$. "}, {"review_id": "VFckoAXbr8", "reviewer": "Reviewer_jsqm", "summary": "The paper deals with the problem of estimating the heterogeneous treatment effects with multiple data sources. In particular, the paper aims to utilize the information from the observational data to better estimate the causal effects in the trial data. Inspired by the shrinkage estimation, a weighting scheme is developed to balance the unbiased estimator based on trial data and the potentially biased estimator based on observational data. Specifically, a tree-based algorithm with new split criterion is proposed based on above motivations. Some theoretical results about the causal effect estimation is derived. Finally, the author provides simulations and real data analysis to demonstrate the performance of the proposed method.\n", "questions": "1. When the bias of the causal effect estimation in observation studies is large, will the data fusion cause any harm to the estimation of the causal effects? Can it be interpreted as a trade-off between the bias and variance?\n2. In section 2.1, the author mentions $\\hat{e}(X,1)$ is unbiased while $\\hat{e}(X,0)$ is biased. Why is that? Didn't they come from the same data source with $\\hat{e}(X,1) + \\hat{e}(X,0) = 1$?\n3. One of the main questions is that, why using tree-based methods for data fusion?\n4. In real data analysis, the author mentions that the true causal effect is unknown and hence using the estimation of the generalized random forest as the ground truth. This is confusing. If the estimation using random forest is not accurate, then the results in the real data analysis is not reliable and convincing. How do the author correct this potential bias?", "limitations": "Please see in Weaknesses. ", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The papers deal with an interesting problem in practice, i.e., data fusion. In particular, we may have multiple data sources. However, some sources has limited observations with unbiased causal effects and other sources have sufficient observations with biased causal effects. The paper utilizes the tree-based algorithm with a new splitting criterion to tackle this issue. In addition, some theoretical analysis are provided to prove the advantages of the method.", "weaknesses": "1. Although the paper considers an important problem in practice, the reason why the method chooses tree-based algorithm is not convincing. In particular, other ML methods can also achieve data fusion. The advantages of using tree-based algorithm over other methods are confusing and are not clearly discussed. \n2. In the introduction and simulation studies, the author also mentions many other methods that deal with the data fusion problem. However, the reason why the proposed tree-based method can perform better than other methods are not interpreted. \n3. Although the paper provides the theoretical analysis for the causal effect estimation, the interpretation for the theorem are not convincing. For example, many other methods also have theoretical guarantee for the causal effect estimation. Which part in the theoretical analysis can illustrate the advantages of data fusion and the tree-based algorithm?\n4. In the general picture, the idea of data fusion is very similar to that of transfer learning, i.e., we want to transfer the information from the observational studies to help the estimation of causal effects in trial data. However, the paper does not mention any related literature in transfer learning. In particular, what is the advantages and differences of the method compared with transfer learning? A more comprehensive literature review is encouraged. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your careful review. Our point-to-point responses are as follows and we would add the additional discussions in the revised manuscript if given a chance.\n\n**W1&W2&W3&Q3**: \nThanks for raising the issues concerning tree-based methods. As mentioned in the abstract of the paper, one of the key contributions of the paper is the establishment of a new data fusion framework called shrinkage estimation. The second is the implementation of a tree-based method by developing an adaptive fusion criterion in Section 3.2. And we chose tree-based methods to show the core idea of the shrinkage estimation framework for two main reasons: \n- the advantages of tree models on tabular data, e.g., robustness to features and model interpretability;\n- The core of this fusion framework is to estimate the variance and bias, which further facilitates the estimation of $w^*$. Although the estimator mentioned here can be any ML model e.g., neural network (NN), many of them cannot conveniently calculate the estimator’s variance. For the tree model,  the variance $σ_u^2$ of the trial estimator can be easily estimated based on the observations that fall into the corresponding tree leaf. \n\nBesides, the rfFAST does not require any parametric models on the data generation mechanism, which ensures its robustness. In comparison, existing methods in the literature usually rely on parametric assumptions. This may explain the better performance of the rfFAST over other existing methods.\n\nFinally, for the theoretical analysis, as stated in Line 210 of the manuscript, \"we formally establish the benefits of the proposed split criterion (9) compared with the conventional criterion (7)\",  which can be specifically listed as follows:\n\n- Theorem 1 shows that our fused estimator with the novel proposed criteria enjoys a uniform MSE reduction property compared to the conventional method using only the trial data. \n- It further takes finite sample variations into account and establishes non-asymptotic bounds for the excess risks of the empirical solutions of the conventional and proposed tree split criteria, respectively, ensuring that the MSEs of the estimated tree are close to those of optimal solutions under the population level.\n\n**W4** Thanks for your suggestion regarding a comparison between transfer learning (abbr TL) and causal data fusion. Intuitively, if we simply consider observational data as the source domain and trial data as the target domain, the trial and observational data fusion scenario can be regarded as a domain adaption problem. \nHowever, there are a few significant differences between these two concepts.\n- Our purpose is to identify cause-and-effect relationships between different variables, while traditional TL focuses on the predictive task. However, The main motivation of TL is to leverage the labeled data of the target domain to improve the performance of the prediction task (Pan and Yang, 2010; Weiss et al., 2020).\n- TL aims to learn the shared knowledge of domains while distinguishing the specific knowledge. In comparison, the prerequisite for causal data fusion is that both the trial data and the observational data share the same causal effect function, as stated in Assumption 1. Besides,  in general, the ground truth is inaccessible for causal data fusion.\n\n**Q1** Indeed, the core idea underlying the shrinkage method is closely related to the classical \"bias-variance\" trade-off in ML. Recalling some notations:\n$$σ_u^2=Var(\\hat{τ}_u^2),b=\\mathbb{E}(\\hat{τ}_b-\\hat{τ}_u), w^*=\\frac{σ_u^2}{σ_u^2+b^2},$$\n and the MSE of  the optimal fused estimator $\\hat{τ}\\_{w^*}=w^*\\hat{τ}_b+(1-w^*)\\hat{τ}_u$ can be expressed as $MSE(\\hat{τ}\\_{w^*})=(1-w^*)σ_u^2$. From the expression of $w^*$, it becomes apparent that the optimal weight seeks an equilibrium between the squared bias $b^2$ of the observational estimator and the variance $σ_u^2$ of the trial estimator.   Also, it is worth mentioning that this interplay between bias and variance is easily achieved through our shrinkage method for real applications, as outlined in the paper.\n\n**Q2** The symbol $S$ (the second parameter of the function $\\hat{e}$) serves as an indicator, and $S=1$ means that the individual is sampled from the trial population otherwise from the observational population. And $e(X,U,S)=P(D=1|X,U,S)$ means the conditional probability of being selected to the treatment group of an individual.\n\nFirst, as outlined in Lines 72 to 74, \"In practice, Due to $U$ being unknown, we usually use $\\hat{e}(X,S)$ to estimate $e(X,U,S)$. In addition, $\\hat{e}(X,1)$ is unbiased for the randomization of trial data, but $\\hat{e}(X,0)$ is biased because the unmeasured confounder $U$ is related to the assignment of treatment $D$\". \n\nIn general, $\\hat{e}(X,1)+\\hat{e}(X,0)$ means $P(D=1|X,S=1)+P(D=1|X,S=0)$ which doesn't equal to $1$.\n\n**Q4**  We first trained a generalized random forest using the full STAR dataset. We then regarded the resulting estimator as a surrogate of the underlying ground truth. The reasons are listed below.\n- It is worth mentioning that a generalized random forest (GRF) estimator is consistent under mild conditions (Athey et al., 2019), meaning that as the sample size increases, it converges to the true causal effect. In our case, since the full dataset is collected from a randomized controlled experiment, then the estimation error of the GRF estimator is expected to be quite small given its sample size.\n- Unlike other fields of machine learning, in causal inference, the ground-truth causal effect is typically inaccessible.  From this perspective, a certain degree of approximation is required. And similar approaches have been used in the literature (Kallus et al., 2018).\n\n**Some Refs:**\n- Pan, S. and Yang, Q. (2010)  A survey on transfer learning. IEEE Transactions on knowledge and data\nengineering, 22(10):1345–1359. \n- Weiss, K., Khoshgoftaar, T. M. and Wang, D. (2016). A survey of transfer learning. Journal of Big Data, 3(1), 1-40."}, {"review_id": "MshCORxRBh", "reviewer": "Reviewer_7DZZ", "summary": "This paper proposes a novel strategy for estimating the heterogeneous treatment effect called the Fused and Accurate Shrinkage Tree (FAST). The authors confirm the consistency of the proposed tree-based estimator and demonstrate the effectiveness of their criterion in reducing prediction error through theoretical analysis. The advantages of the proposed method over existing methods are demonstrated via simulations and real data analysis. As I am not very familiar with this field, it might be better to consider my opinion less.", "questions": "None", "limitations": "None", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper is technically sound.\n2. The proposed method has better performance than the existing methods.", "weaknesses": "Imcomplete references: The tree-based method seems to not be a new method in this field. There might be some other references as follows.\n\nAgarwal, Abhineet, et al. \"Hierarchical Shrinkage: Improving the accuracy and interpretability of tree-based models.\" International Conference on Machine Learning. PMLR, 2022.\n\nNasseri, Keyan, et al. \"Group Probability-Weighted Tree Sums for Interpretable Modeling of Heterogeneous Data.\" arXiv preprint arXiv:2205.15135 (2022).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your instructive and detailed review comments. We are  encouraged by the overall positive responses from the comments and suggestions. Our point-to-point responses to your comments are itemized below.\n\n**Q1:**  Thank you for highlighting the potential absence of references. This suggestion greatly contributes to our effort to conduct a more comprehensive literature review. \n  \n  After a careful reading of the two papers, we find them indeed closely related to our work. Specifically, Agarwal et al. (2022) also developed a shrinkage method to improve the performance of the tree estimator. One of the main differences is that our procedure shrinks the original estimator to a (potentially) biased estimator with small variability obtained from extra data sources, namely the observational data, while they propose to shrink the estimates at each leaf node to the sample means of its ancestors.\nNasseri et al. (2022), on the other hand, generalized the tree-based methods to address the challenge of heterogeneous data from diverse data sources. It is worthwhile to note that both methods can be readily applied together with our shrinkage strategy, as long as a sufficient amount of (potentially biased) observational data is available. \n\nWe would discuss these aspects in the revised version of the paper if given a chance.\n\n"}]

Author rebuttal:
We would like to express our sincere thanks to all the reviewers for your insightful and constructive review comments and we are greatly encouraged by the overall positive responses. Based on your valuable suggestions, we have carefully done a round of revision of the manuscript. While detailed responses to individual points can be found in the subsequent rebuttals, we also seize this moment to summarize the progress achieved in improving this paper over the past week.

-  We improved the illustration of the core idea of the shrinkage estimation framework.
-  A more detailed explanation of the expression of the optimal weight was given and missing details of the derivations due to page limit were presented.
-  The algorithm complexity of the FAST estimator was analyzed.
-  The advantages of the proposed method over existing methods were interpreted and limitations were discussed.
-  Three additional numerical experiments were conducted (shown in the additional PDF file) to better validate the effectiveness of the proposed method. 

We would add all the revisions to the manuscript if given a chance. Thanks again for all your great efforts and contributions in jointly improving this work.

Meta-review: The reviewers appraised the paper is well written and easy to follow, that the proposed algorithm is well-justified and the theoretical analysis is interesting, that the underlying problem is important, and the idea of applying shrinkage method to combine interventional and observational data for better estimator is novel.

There is one low rating but in my opinion it is somewhat offset by other high ratings. Overall, I agree with the over majority of the reviewers and support acceptance of this paper.
Decision: Accept (poster)