Generalizing Causal Effects from Randomized Controlled Trials to Target Populations across Diverse Environments
Authors: Baohong Li
Yingrong Wang
Anpeng Wu
Ming Ma
Ruoxuan Xiong
Kun Kuang
Venue: ICML 2025 poster (2025)
Keywords: Causal Inference
Randomized Controlled Trial
Generalization
Treatment Effect Estimation
Data Fusion
Missing Covariates
Selection Bias
Primary area: general_machine_learning->causality
TL;DR: This paper proposes identifiable conditions and a novel method for generalizing causal effects across environments in the presence of distributional shift and missing variables.

Abstract: Generalizing causal effects from Randomized Controlled Trials (RCTs) to target populations across diverse environments is of significant practical importance, as RCTs are often costly and logistically complex to conduct. A key challenge is environmental shift, defined as changes in the distribution and availability of covariates between source and target environments. A common approach addressing this challenge is to identify a separating set--covariates that govern both treatment effect heterogeneity and environmental differences--and combine RCT samples with target populations matched on this set. However, this approach assumes that the separating set is fully observed and shared across datasets, an assumption often violated in practice. We propose a novel Two-Stage Doubly Robust (2SDR) method that relaxes this assumption by allowing the separating set to be observed in only one of the two datasets. 2SDR leverages shadow variables to impute missing components of the separating set and generalize treatment effects across environments in a two-stage procedure. We show the identification of causal effects in target environments under 2SDR and demonstrate its effectiveness through extensive experiments on both synthetic and real-world datasets.

Generalizing Causal Effects from Randomized Controlled Trials to Target Populations across Diverse Environments
Baohong Li
Yingrong Wang' Anpeng Wu
Ming Ma
Ruoxuan Xiong?
Kun Kuang'
Abstract
considered the gold standard for estimating causal effects of treatments (Imbens & Rubin, 2015; Hariton & Locascio, 2018). However, due to the long duration and high cost of RCTs, it is often desired to generalize the treatment effect estimates from existing RCTs to target populations across different environments.
Generalizing causal effects from Randomized Controlled Trials (RCTs) to target populations across diverse environments is of significant prac- tical importance, as RCTs are often costly and logistically complex to conduct. A key chal- lenge is environmental shift, defined as changes in the distribution and availability of covariates between source and target environments. A com- mon approach addressing this challenge is to iden- tify a separating set- covariaass that govern both treatment effect heterogeneity and environmen- tal differences-and combine RCT samples with target populations matched on this set. However, this approach assumes that the separating set is fully observed and shared across datasets, an as- sumption often violated in practice. We propose a novel Two-Stage Doubly Robust (2SDR) method that relaxes this assumption by allowing the sep- arating set to be observed in only one of the two datasets. 2SDR leverages shadow variables to im- pute missing components of the separating set and generalize treatment effects across environments in two-stage procedure. We show the identifica- tion of causal effects in target environments under 2SDR and demonstrate its effectiveness through extensive experiments on both synthetic and real- world datasets.
A major challenge in generalizing RCT findings across envi- ronments is environmental shifts, defined as changes in the distribution and availability of covariates between the RCT sample and the target population (Kuang et al., 2018; 2020). Such shifts cause RCT participants to no longer represent the target population, thereby limiting the external validity and generalizability of causal effect estimates from the RCT (Nguyen et al., 2018; Colnet et al., 2024).
A common solution to addressing the challenge of environ- mental shifts is to combine RCT data with observational data from the target population, using a separating set for adjustment. Compared to collecting new RCT data, obtain- ing observational data from the target population is typically more cost-effective and feasible. The separating set con- sists of covariates that simultaneously affect both treatment effect heterogeneity and environmental shifts. Prior work has shown that the Target population Average Treatment Effect (TATE) is identifiable with the separating set (Cole & Stuart, 2010; Tipton, 2013; Kern et al., 2016; Egami & Hart- man, 2021; Pearl & Bareinboim, 2022), and many methods have been proposed to leverage the separating set for gen- eralizing treatment effects from RCTs target populations (Stuart et al., 2011; Hartman et al., 2015; Lesko et al., 2017; Dahabreh et al., 2019; Lee et al., 2023).
1. Introduction
The limitation of this common approach is its reliance on the assumption that the covariates shared across both datasets fully contain the separating set-an assumption that is often violated in real-world settings with environmental shifts, When data from the RCT and the target population are col- lected in different environments, it is challenging to ensure complete alignment in the covariates measured, leading to a missing covariates problem: certain covariates present in one dataset may be entirely absent in the other. If these missing covariates include variables from the separating set, existing methods may fail generalize treatment effects accurately, as shown in prior studies (Nguyen al., 2017;
Estimating treatment effects is crucial for informing whether to apply new policy or treatment in a given population (the target population) (Hernán& & Robins, 2020; Luo et al., 2024). Randomized Controlled Trials (RCT) are generally
College of Computer Science and Technology, Zhejiang Uni- versity, Hangzhou, China Kuaishou, Beijing, China 'Deparmment of Quantitative Theory & Methods, Emory University, Atlanta, USA. Correspondence to: Kun Kuang <kunkuang @ zju.edu.cn>>.
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
Setting
Setting
X2
X1
RCT
Data
Obvervationa Data
RCT Data
Observational Data
V
8
x
4
x
x
x
R
v
V
x
x
4
x x
8
V
V
x
XG
X3
(a)
(b)
Figure 1. (a) The causal graph for the problem setup. For clarity, we only present the causal graph for the ohservational data, while for the RCT data, all edges pointing to T removed because the treatment is randomly assigned. Suppose that the heterogeneous treatment effect of T on Y is (1) (0) = 7(X1, X2, X2),),,, Xs). where T(X1, X2, Xs) is function of X1, X2, and Xs. {X1, X2, X33 } is the set of covariates affecting treatment effect heterogeneity. {X1, Xa is the of covariates affecting environmental shifts (pointing to R). X₂ is a covariate affecting neither treatment effect heterogeneity nor environmental shifts. Therefore, the separating set, which contains all variables that simultaneously affect treatment effect heterogeneity and environmental shifts, is W {X1, X3). . (b) Example data for the problem setup. where X [X1, Xa, Xu J is the set of common covariates observed in both datasets, X = [Xs} is the set of covariates missing in one the datasets, Z = { { is the shadow variable set of XT", which are correlated with the X but do not directly influence environmental shifts, and X {4) denotes X Z.
Andrews & Oster, 2019; Colnet et al., 2022; Dahabreh et al., 2023; Huang, 2024) and supported by our experiments.
obtained through large-scale surveys or interviews (Collab- oration, 2009; Resche-Rigon et al., 2013; Jolani et al., 2015; Huang et al., 2023; Huang, 2024).
We propose a solution to relax the assumption required by the common approach. Specifically, we only require variables from the separating set to be observed in only one of the two datasets either the RCT data the observational data from the target population, as illustrated in Figure 1.
Given this relaxed assumption, our proposed method, Two- Stage Doubly Robust (2SDR), leverages shadow variables to impute the missing covariates and generalize treatment ef- fects from RCTs to target populations across environments via a two-stage procedure. In the first stage, we introduce a feature selection method to identify shadow variables of the missing covariates, enabling unbiased imputation of these covariates in a doubly robust manner. the second stage, we apply a doubly robust approach to obtain an unbiased es- timate of TATE. We show the identification of causal effects in target environments under this approach and demonstrate its effectiveness through extensive experiments on both syn- thetic and real-world datasets.
In Setting 1, only the covariates from the target population are available or used. The RCT data include all covariates from the separating set, but some of these covariates are missing the target population data. For example, in eval- drug AIDS, RCTs
uating the treatment effect of typically collect various clinical variables rarely measured in routine check-ups, such as the CD4 count, which is an important treatment effect modifier and a variable in the separating set. Since the new drug has not yet been publicly used, the observational data from the target population lack treatment and outcome information and only include basic demographic covariates, such as gender and race, and omit those rarely measured variables available in the RCT data (Hammer et al., 1997; Prejean et al., 2008; Hall et al., 2008).
In summary, the contributions this paper are as follows:
We propose novel identification framework for the generalizability of treatment effects from RCTs to tar- get populations across diverse environments, relaxing the assumptions in prior work.
In Setting 2, the observational data from the target popula- tion contain treatment and outcome information, along with all covariates from the separating set. However, some of these covariates are missing in the RCT data. For example, when RCTs are conducted small sites, limitations such as long tracking periods, budget constraints, and lack of equipment may result in insufficient covariate collection. In contrast, when observational data from the target population are collected at larger sites, treatment and outcome infor- mation, along with more comprehensive covariates, can be
We develop novel method for generalizing treatment effects target populations under environmental shifts, ensuring an unbiased estimate of the TATE. We demonstrate the effectiveness of our method through extensive experiments on synthetic data and applications to two real-wort datasets.
2

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
2. Preliminaries
2.2. TATE Identifiability Conditions
2.1. Problem Formulation
First, to ensure the identifiability of the average treatment effect within Dataset R, make the following assumptions about R (Cole & Stuart, 2010; Stuart et al., 201 I; Imbens & Rubin, 2015; Colnet et al., 2022; Dahabreh et al., 2023).
Throughout the paper, T denotes the binary treatment vari- able, where i; indicates that unit is assigned to the treatment, and 1 0 indicates otherwise. Y is the out- come variable, and (t) denotes the potential outcome under T = t (Rubin, 1974). X (Xe, x") treatment covariates, where X denotes the common covari- ates present in both datasets, and XT denotes the covariates missing in one of the two datasets. S is a binary variable indicating in which dataset Xt is observable, where 8; = indicates that X observed in the RCT data, and s; indicates otherwise. R is binary variable indicating which dataset a unit belongs to, where r; = indicates that unit belongs to the RCT data, and r; 0 indicates otherwise.
Assumption 2.3. (Treatment randomization within the
RCT.) (1), (0) 1L T == 1.
} is the pre-
Assumption 2.4. (Positivity of trial treatment assign-
ment.) P(T | =8 1) 11<+) 1.
X, Assumption 2.5. (Positivity of trial participation.) 0
P(R=1 X) < 1.
Assumption 2.6. (Stable Unit Treatment Value Assump- tion.) The distribution of the potential outcome of one unit is independent of the treatment assignment of another unit.
For simplicity of exposition, we introduce our solution in the context of generalizing an RCT to the target population of one environment. However, our solution can be extended to multiple target populations in various environments. In Setting 1, the RCT data consists of næ independent random tuples (x,, Vs. L,, 8/ l, = 1 from all participants in the RCT, denoted by R. while the observational data from the target population consists of no independent random tuples &x = 0,r, = 0 randomly drawn from the target population, denoted by O. In Setting 2, the RCT data consists of (x,, M/. f,, 8 0,r; = 1) while the the observational data from the target population consists of {xa, Vá, f4, 8; = 1, r, 0 The total number of samples in both datasets is denoted by n nR + no. In this paper, we consider the non-nested trial design scenario, where the two datasets are collected separately from different environ- ments with environmental shifts, which is defined as follows (Kuang et al., 2018; 2020; Dahabreh et al., 2020; 2021).
Under the above assumptions, the average treatment effect of Dataset R is identifiable. Next, we formally introduce the conditions for the generalizability of the RCT estimates, i.e., the identifiability conditions of the TATE (Cole & Stuart, 2010; Tipton, 2013; Kern et al., 2016; Egami & Hartman, 2021; Pearl & Bareinboim, 2022).
Definition 2.7. (Separating Set.) A separating set is a set that contains all variables that simultaneously affect treatment effect heterogeneity and environmental shifts, i.e., Y (0) 1L R W.
Assumption 2.8. (Fully observability of the separating set.) The covariates common both datasets include all the variables from the separating set, i.e., W C X..
Theorem 2.9. (Identification of the TATE.) Under Assump- tions 2.3. 2.4, 2.5, 2.6, and 2.8, the TATE is identified as
T == f (BYY) W = w| E[Y / W =W)
Definition 2.1. (Environmental Shifts.) Environmental shifts refer to the shifts in the distribution and quantity of covariates between Dataset R and Dataset O, i.e., P(X
dF(W w R -8=00 0),
where F(W R= 0) is the cumulative distribution func- tion of W conditional on = 0, and (E W = =w.]]
R = 1) & F(X R - 0), and X= is unobserved in one of the two datasets.
E[Y W w., T = t, R= 1].
Prior work proposes different ways adjust for W based on Theorem 2.9 (Hartman et al., 2015; Dahabreh al., 2019; Lee et al., 2023). However, because of environmental shifts, it is difficult to ensure that all variables from the separating set are observed in both datasets, making Assumption 2.8 violated and existing method ineffective (Nguyen et al., 2017; Colnet al., 2022; Dahabreh et al., 2023).
The specific scenarios of environmental shifts applicable to this paper are discussed in Appendix B.
We are interested estimating the average treatment effect in the target population. This causal estimand is called the Target population Average Treatment Effect (TATE) and is defined as follows (Tipton, 2013; Kern et al., 2016).
Definition 2.2. (Target population Average Treatment Effect.) E| (1) (0) R = 0].
3. Extended TATE Identifiability Conditions
The TATE cannot be directly estimated using Dataset O, as the values of the treatment are totally unobserved. For- tunately, combining Dataset O with Dataset R makes the TATE identifiable under certain assumptions.
In this paper, we propose a novel TATE identifiability frame- work that relaxes the requirement for the separating set to be observable in both datasets to requiring it to be observable in at least one, The key assumption is as follows:
3

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
Assumption 3.1. (Partial observability of the separating set.) All variables in the separating set are observable in at least one of Dataset R or Dataset O, i.e. W C X.
where F(X R = 0) denotes the cumulative distribution
function of X conditional on R
Proof. To prove the identifiability of Equation (l), we must prove both F(X R = 0) and (t) X are identifiable.
To formalize this, we introduce an indicator variable S,
where 5 = 1 if the separating set is observed and S = 0 if it is missing. As illustrated in Figure 1, Assumption 3.1 indicates one of the following two settings:
Step 1. Proof of the identifiability of F(X | R R=0).
Step 1.1. Proof in Setting 2 (separating set in O).
(Setting 1.) Dataset O has no treatment and outcome information. The separating set is observed in Dataset R. i.e., in Dataset R. S = 1, and in Dataset O, S = O. (Setting 2.) Dataset O contains treatment and outcome information. The separating set is observed in Dataset O, i.e., in Dataset R. S = 0, and in Dataset O 8 = 1.
In Setting 2. all covariates X in Dataset O are observable. Therefore, F(X R = 0) is directly identifiable.
Step 1.2. Proof in Setting (separating set in R).
In Setting 1, however, as the values of X are missing in Dataset O which may contain variables from the separating set, P(X, R = 0) P(X X R 0) is not available. Therefore. to prove the identifiability of F(X x R 12=00 0), we must prove F(X X R = 0) is identifiable.
In Setting 1, the relationship between S and R is S = R, whereas in Setting 2, the relationship is S = 1 - R. Both settings are common in real-world scenarios (Hammer et al., 1997; Prejean et al., 2008; Collaboration, 2009; Resche- Rigon al., 2013; Jolani et al., 2015), as illustrated by the two examples provided in Section 1.
Since S = R in Setting 1, we have PPX P(X",XX,R' R= 0) = P(X X = 0). We show that this distribution can be identified using the shadow variables Z and common covariates X The corresponding lemmas are stated below, with detailed proofs provided in Appendices D) and E.
Assumption 3.2. (Shadow variable assumption.) Among the common covariates X shared by datasets R and O, there exists set of covariates Z C X that satisfy the following conditions: (1) Z is conditionally correlated with
Lemma 3.4. Under Assumptions 2.5 and 3.2, P(X"
X S = 0) is identified as
X. i.e., Z KL X X 1. (2) Z is conditionally
independent of S, i.e., Z 11 S X Xe, where X denotes
X = OR(X X*) P(X
X S = 1)
P(X m
the set difference X Z. i.e., X Z).
E[OR(X m Xe)
X = 1]
(2)
Assumption 3.2 requires that among the common covari- ates, there exist shadow variables Z that are correlated with the covariates X missing in one of the datasets but do not directly influence environmental shifts (as indicated by S); see, e.g., Z = { in Figure 1. This assumption is reasonable in many real-world settings, as not all observed variables impact the environmental shifts. For example, in studies of drug effects on AIDS, body weight can serve as shadow variable Z: it has been shown to be associated with CD4 count (a missing covariate), but does not directly influence RCT participant selection (Womack et al., 2007) (measured by S). Moreover, this assumption is testable us- ing observed data (d' Haultfoeuille, 2010; Miao et al., 2024; Li et al., 2024b;c).
where
PPS = X",
P(S = 1 Xm X OR(X"T, Xx)
P(S X m = 0,X9)
(3)
P(S=0 X m = Xc)
Here, OR(X", X*) is the odds ratio function, and 0 a ref- erence value, which can be any other value within the value range of X The identification of Equation (2) requires the identification of TP(X X S = 1) and OR(X". Xº). The former is directly identifiable in Dataset R. The latter can be identified based on Lemma 3.5 below, with detailed proofs provided in Appendix E.
Theorem 3.3. (Extended TATE Identifiability Conditions.) Under Assumptions 2.3, 2.4, 2.5, 2.6, 3.1, and 3.2, if the completeness condition holds', the TATE is identified as
Lemma 3.5. Under Assumptions 2.5 and 3.2, if the com- pleteness condition holds, OR(X... ,x') is identified as
== f (1) X = x E{Y(0)) X = xx)
OR(X X.) = OR( XOR 0, x.), (4)
dF(X = X R= 0).
where
IIf for all square-integrable functions 1(X" X"). (X X9) n x S
OR(X X²)
0 almost surely if
OR(X (1)) EXOR(X X*) X (5)
and only if h(x"
x*) = almost surely.
4

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
is identified by
Goal: Proof of the ieentfiahility of
E Th ) X S 1] P(Z X 0) PPZZ| X - (6)
Step I: Proof of the identifabblity F(XJR 0)
Séep 2: Prouf the identifability of EJP(()
Since P(X x
= 0)
identifiable and P(X S 0)
is available from the observed data, P(X X = 0) is identified as
Step III: Proof Setting
Step 1.2: Proof Setting
Step 2.1: Proof Setting
Step 2.2: Proof Setting 2
Lemma 3.4 and Lemma 3.5
Lemma 3.4 and Leama 3.5
P(X", X S = 0) = P(X X S = P(X S 5=0),
which is equivalent to P(X X R = 0).
Therefore,
P(X & 0) sdemitiable
PX" Y(O, - 1) idenssiable
F(X| R = 0) is also identifiable in Setting 1.
FOX.R - 0) identifiable.
FOLTY identifiblle.
Step 2. Proof of the identifiability of E (t) X].
Step 2.1. Proof in Setting (separating set in R).
Figure 2. Proof process of Theorem 4.1.
In Setting 1, all covariates X in Dataset R are observable and include all the variables from the separating set under Assumption 3.1. Therefore, under Assumptions 2.3, 2.4, 2.5, and 2.6, (() X] is identified by [E( (1) | XX] B||| X, T = t, R = 1] based on Theorem 2.9.
Definition 3.6. (Selection scores.) The selection score of is nT (X) TP(S = 1 X), and the selection score of is n. (X) P(R 1 x)..
Corollary 3.7. (Identification of the selection scores.) Un- der Assumptions 2.3, 2.4, 2.5, 2.6, 3.1, and 3.2, na (X) is identified as
Step 2.2. Proof in Setting 2 (separating set O).
In Setting 2, prove the identifiability of ( X]. we must prove the identifiability of P(X, T, (t) R 1).
-1
P(S = X9)
Under Assumption 2.3, we have T IL X
R = 1. There-
T. (X) = 1 1+ x9)
fore, P(X,T,YU (t) | R = 1) is identified as
P(S = X
and n. (X) is identified as म. (X) T, (X) in Setting I and T, XX = 1 - (X) ) in Setting 2.
P(X, (t)(t) R = 1) = P(X
X (t), R == 1)
x P(X (()(t) R=
The detailed proof of Corollary 3.7 is in Appendix F.
where P(X", Y(t), T| R 1) is available in Dataset R,
and P(X m
X X() R 1) is identified by
4. Two-Stage Doubly Robust TATE Estimation
I(X X(,Y(),, R= P(X,R=1 (t) 1 P(X:, Y(I,R=1)
Based on Theorem 3.3 and Corollary 3.7, propose novel Two-Stage Doubly Robust (2SDR) algorithm to esti- mate the TATE. 2SDR consists of two stages:
x
Here, P(X),((t), R = 1) is identifiable in Dataset R;
(Stage L.) Shadow variable selection and imputation of
P(X, R = 1) = P(X, S =0) 0) (because 5 = 1 R in Set- ting 2), which is identifiable based on Lemmas 3.4 and 3.5; and (t) X, R = 1) is identifiable under Assumption 3.1, where (1) X, R = 1) = P(Y(() X, R 0), which is available Dataset O.
the missing covariates.
(Stage II.) Selection score and TATE estimation.
4.1. Stage of 2SDR
4.1.1. SHADOW VARIABLE SELECTION
Consequently, P.X, T, (t) | R = 1) and, thus, ({) X are identifiable in Dataset R based on Theorem 2.9.
Theorem 3.3 and Corollary 3.7 indicate that identifying the TATE and the selection score necessitates the shadow variables Z. However, under Assumption 3.2, although Z is included in the common covariates X', we do not know which variables satisfy Assumption 3.2. Therefore, we first propose automatically selecting the variables from X that satisfy Assumption 3.2, i.e., selecting Z from X
The overall process of the proof is illustrated in Figure 2.
Theorem 3.3 is sufficient for the estimation of the TATE. However, in order to estimate the TATE in a doubly robust manner, we further introduce the selection scores.
Assumption 3.2 consists of two testable sub-assumptions.
5

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
Therefore, in the process of shadow variable selection, 2SDR conducts hypothesis testing on the variables in X to eliminate those that do not pass the test, leaving the variables that pass the test as the set of shadow variables.
Stage
Stage
Shadow Variable Selection
Imputation of
TATE Estimation
To reduce the number of tine-consuming hypothesis tests, we first apply Adaptive Lasso (Zou, 2006) for quick initial screening the variables in X", selecting those strongly correlated with X as Assumption 3.2(1) requires that Z be conditionally correlated with X The set of filtered variables is denoted as X C X
Regression-based X Estimation
Adaptive Lasso
DRImutatiion
Regression based TATE Estimation
Next, we perform hypothesis testing for Assumption 3.2 on the filtered variables. Assumption 3.2(1) involves only the dataset where the covariates are observable, so standard conditional independence test methods with reject thresh- old ai can be directly applied (Zhang et al., 2011; Sen al., 2017; Strobl al., 2019; Zheng et al., 2024). However, Assumption 3.2(2) involves both datasets, and the values of X are missing in one of them. Therefore, we cannot directly apply a standard conditional independence test to verify this aub-assumption, but instead, we conduct the test based on the following theorem (d Haultfoeuille, 2010).
OR Estimation
DR TATE Estimation
Qu
Solive
Estimation
TATE
Figure 3. Overview of 2SDR.
The time complexity of the shadow variable selection pro- depends the specific method used to test Assumption 3.2(1), which, in our implementation, is the Randomized Conditional Independence Test (RCIT) (Strobl et al., 2019), As a result, the time complexity of this process is (mpp²), where is the number of variables in X. A detailed time complexity analysis can be found in Appendix
Theorem 4.1. (Hypothesis testing for Assumption 3.2(2).) Suppose that P(X > 0) holds almost surely. Then, As- sumption 3.2(2), i.e., Z IL S Xm Xmm,x,, Xs, can be rejected if and only if there exists solution the following equation that belongs to (0, 1].
S E Q(XmI X9)
-1):
Z (
4.1.2. IMPUTATION OF THE MISSING COVARIATES
O.
X
Through the shadow variable selection process, obtain the set of shadow variables Z. Next, based on Theorem 3.3 and Corollary 3.7, we use Z to impute the missing values of X in doubly robust manner, as detailed below.
where Q the unknown unknown,uunctio to be solved
The proof of Theorem 4.1 can be found in Appendix A.3 of d' H'Haultfouille (2010).
First, we estimate the odds ratio function using Equations (3) and (4). Specifically, we begin by employing kernel density estimation (Silverman, 2018) to estimate f(Z
Based on Theorem 4.1, we solve for Q by minimizing the following objective function:
X
(
1) and f(Z X S 0), where f(Z X S) is the probability density function Z conditional on X and S. Then, we estimate OR(X". X*) by minimizing the following objective function:
to - ni
Q(F x
where Q constrained to the range (0, 1] by an activation function, such as the sigmoid function, Z xf denotes the candidate shadow variables, and X denotes the set difference o z If lq converges to a value less than a threshold a22, the candidate shadow variables pass the test.
f(z, x" 0)
OR(x," x²)
OR
f(z,
x x",, 31
Eiey
To ensure that the completeness condition for the shadow variables is satisfied, we aim select d shadow variables from X where d equals the number of variables in X (Miao et al., 2024). Therefore, we iterate through X!, se- lecting a candidate shadow variable in each iteration and performing hypothesis testing. The loop continues until d candidates pass the test, at which point the corresponding shadow variables are selected, and the loop terminates.
where f(z X² denotes the estimated probability den- sity functions. Subsequently, we obtain the estimated odds ratio function (R xx) using Equation (4).
Next, we learn a function (Xx) to estimate E X S = 0] using a doubly robust method (Miao Tch- etgen Tchetgen, 2016; Kennedy, 2023) by minimizing the
6

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
following objective function:
This enables to perform doubly robust estimation of the TATE (Dahabreh & Hernán, 2019; Dahabreh al., 2019).
4x
(x() -x, ),
RR 1 --
1.4 (4.(1 (u (xx))
i
f
((x;) ((())
110
(x:))
where x denotes the doubly robust estimate of E|X"
nR R-R 1 (x,))
((1-1,) (Vs (o((xx))
X x 0], detailed follows.
nR
no
(x,)
1 - -xx)
x 5; rū; (x.T - 8(x{)) + 8(x)),
no
(fn (x:) -o(())))
Here, w, = (x;), where म. is the estimate of a.. and å the regression- based estimate of E X x S 0).
n0
Here, ((X) P(T X) is the propensity score, which is constant under Assumption 2.3; 7, is the estimate of and can be directly obtained from the first stage, i.e., # +, for Setting and 7, -1- n. for Setting 2; and Āo and βi are the estimates of E X, = 0, = 1] and E[Y X, T = S, = 1], respectively, obtained by minimizing the following objective functions:
Based on Corollary 3.7, A. is obtained by
-1
*x 1 OR(X" X²). 1 (x9
(Xx)
where y is the estimate of P(S = 1 X³), obtained by minimizing the following objective function:
1 BR
tuo
Σ
((1 1;) (HHO(x)) 1.)))
nR
e, Σ s1) loog11 - (xx)) ss lxg()
1
(4
v1)))
nR
Based on Equation (2), 8 obtained by minimizing the following objective function:
The proposed doubly robust TATE estimation model is con- sistent, as guaranteed by the following theorem.
2
OR (x.", xg) (x{) OR (xf) x
Theorem 4.3. Under Assumptions 2.3, 2.4, 2.5, 2.6, 3.1, and 3.2, if the imputation model is consistent, the TATE esti- mator & is consistent if the OR model is correctly specified and either the regression model of the outcome Y or the selection score model of S is correctly specified.
88
Σ P:N:
5(x;)
where / the estimate of E[X X = 1). obtained by minimizing the following objective function:
The proof of Theorem 4.3 is provided Appendix H.
La = TI
Σ (0(x))-x;)) 1:s=1
x
5. Experiments
Consequently, we can use the estimate of E X
5.1. Experimental Setup
0), i.e., (xx) to impute the missing values of X
In our experiments, we compared the proposed method with the following baselines: (1) The average treatment effect ob-
The proposed doubly robust imputation model is consistent, as guaranteed by the following theorem.
tained directly from the RCT data (RCT), (2) Inverse Proba- bility of Sampling Weighting (IPSW) (Cole & Stuart, 2010), (3) Calibration Weighting (CW) (Hartman et al., 2015), (4) G-formula (Dahabreh et al., 2019), (5) Augmented IPSW (AIPSW) (Dahabreh al., 2019), and (6) Augmented CW (ACW) (Lee al., 2023), along with their versions incor-
Theorem 4.2. Under Assumptions 2.3, 2.5, 2.6, 3.1, and 3.2, the doubly robust imputation model consistent if the OR(X"",X model is correctly specified and either the regression model of the missing covariates X or the selection score model of correctly specified.
porating data imputation (Hughes et al., 2019; Mayer et al., 2023), denoted by their name with the 'Imp' subscript.
The proof of Theorem 4.2 is provided in Appendix G.
We conducted experiments on both synthetic and real-world datasets, with each experiment repeated 20 times. We re- port the mean and standard deviation (std) of the Mean Absolute Error (MAE) between the TATE estimates and
4.2. Stage II of 2SDR
In the first stage, we impute the missing values of X making all the covariates in both datasets fully observable.

---

To evaluate the effectiveness of the proposed method both cases, we generated datasets based on Setting to simulate the case where the values of X are missing in Dataset O and Setting 2 to simulate the case where the values of Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
Table TATE estimation results (MAE) on the low-dimensional synthetic dataset (mean-tstd), with hold values indicating the hest
performance.
1o. = 2000, To =
10000
Y/. 1000, no = 10000
71 500, n4c2 = 10000
METHODS
SETTING
SETTING
SETTING 1
SETTING
SETTING
SETTING 2
RCT
1.640#1.008
1.751=0.913
1.685#1.000 1.047
1.803
2.037#1.460 2.260=1.313
IPSW
6.460=#0301
6.564=0.375
7.0344:.212
0.84240.129
6.999=0.152
7.066 167
CW
2.062+1.551
2.197=1.075
2.566=11.800
2.789+1.211
4.982-2.765
4.540:12.154
G-FORMULA
0.663+0.260
0.659-0.266
0.720-10.263
0.692-10.77
1.145=0.814
0.83410.710
AIPSW
0.663+0.261
0.660-0.265
0.720:10.264
0.692+003777
1.145=0.815
0.834
ACW
0.66210.260
0.659-0.266
0.720:10.233
0.69310.377
1.145=0.814
0.833
IPSWImp Imp
6.350=10.159
6,458=0.188
0.829:0.126
6.88740.194
6.910=0.101
6.924 103
CW
2.02011.569
2.006=1.020
2.410=1.235
2.453
3.794=3.436
3.108+2.322
G-FORMUL-Ana
0.539+0.305
0.815-0.310
0.614+0.355
O. 0.998+0.450
0.961=0.225
1.787+
AIPSW Imp
0.539+0.303
0.829-0.306
0.614+0.355
1.084+0.453
0.960=0.625
1.792+00949
ACWImp Imp
0.539+00.30
0.816=0.310
0.614#0.356
0.998+004451
0.961=0.625
1.789 9.944
2SDR
0.26810.211
0.284_0.224
0.4234
0.431
0.333_0.387
0.489
the ground truth values. In each experiment, we randomly split the dataset into training, validation, and test sets with 60/20/2 /20 ratio. Implementation details of the proposed method, as well as the software and hardware we used, are provided in Appendix J.
achieves the best performance across all RCT scales, demon- strating its robustness. (2) In Setting 1, the performance of the baselines using imputation is better than that without imputation. However, in Setting 2, the regression- baaeesdoon-bseed baselines, including doubly robust ones, fail to produce better estimates using imputation. This observation can be attributed to the fact that, in Setting 1, the regression process involves the fully observed covariates from the RCT data, and the imputation error only affects the prediction based the imputed covariates from the observational data. In contrast, in Setting 2, the regression process involves im- puted covariate values from the RCT data, accumulating imputation errors, and introducing bias into the regression model. On the other hand, 2SDR performs well in both set- tings because it selects shadow variables and uses them to obtain unbiased imputation. (3) The doubly robust baselines do not perform better, both the selection scores and out- come estimates are biased when variables in the separating set are missing. In contrast, 2SDR ensures that both the selection score and outcome estimate are unbiased, leading to its superior performance.
The source code of 2SDR is available at https:://// /gthhbb. com/ JJU.achcngliii /SDR.
5.2. Experiments on Synthetic Datasets
5.2.1. DATASETS
X are missing in Dataset R. The detailed data generation process is in Appendix K.
In general, due to the high cost of RCTs, nR is typ- ically much smaller than no (Kohavi & Longbotham, 2011; Kallus et al., 2018). Therefore, uate the robustness of the proposed method with small- scale RCT data, we fixed no to 10000 while setting nı {500, 1000, 2000} compare the performance of 2SDR with the baselines across different RCT scales.
order to eval-
Additional experimental results, including results on the figh-ddmensiioal dataset, a comparison of the imputation accuracy between the proposed method and the baselines using imputation, the hyperparameter analysis, and the ro- bustness analysis of the proposed method when Assumption 3.2 is violated, are presented in Appendix L.
To evaluate the performance of the proposed method in the case of a larger number of covariates, we also generated high-dimensional dataset. The detailed data generation process can be found in Appendix K.
5.3. Experiments on Real-Worke Datasets
5.3.1. DATASETS
5.2.2. RESULTS
To validate the effectiveness of the proposed method in real- world applications. we conducted experiments on two real datasets, the AIDS Clinical Trial Group (ACTG) and Jobs Training Partnership Act (JTPA) datasets, with the ACTG
We report the TATE estimation results under different RCT scales in Table 1, with the following observations and con- clusions: (1) As the RCT scale decreases, the TATE estima- tion error increases for all methods. However, 2SDR always
8

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
5.3.2. RESULTS
Table TATE estimation results (MAF) on real-world datasets (meanisstd scaled by 10"), with bold values indicating the best performance.
We report the TATE estimation results on the two real-world datasets in Table 2, with the following observations and conclusions: (1) On the ACTG dataset, the performance of all baselines is even worse than using the ATE obtained di- rectly from the RCT data. This is because the pre-treatment CD4 count, which significantly impacts the outcome, is missing in the observational data from the target population. As result, methods that fail impute the missing pre- treatment CD4 count values accurately (2) Similarly, on the JTPA dataset, due to the absence of the years of education in the RCT data, which significantly impacts the outcome, the majority of the baseline models exhibit substantial bias when estimating the TATE, perform- ing even worse than using the ATE obtained directly from the RCT data, (3) 2SDR achieves the best performance on both real-world datasets, demonstrating its practical value in real-world scenarios.
METHODS
ACTG
JTPA
RCT
0.032+0.016
1.366++07709
IPSW
640.4+98.80
1.06440.519
CW
0.367=0.044
9.749+0.759
G-FORMULA
0.035-10.024
2.247:1.9965
AIPSW
27.91
2.249110.96
highty biased,
ACW
0.035:00024
2.295
IPSW Imp
656.5=159.1
1.230+0.688
CWImp
0,340=0.036
1.905
G- G-FOMMULTI
0.037=0.026
2.229=1.553
AIPS IImp
14.97+6.543
2.230+1.555
ACWIN Imp
0.037+0.026
2.229+1.553
2SDR
0.011=0.010
0.992
dataset corresponding to Setting 1 and the JTPA dataset to Setting 2, detailed as follows.
6. Conclusion
In this paper, we focus on the problem of generalizing treat- ment effects from RCTs to target populations across dif- ferent environments. To tackle the challenges of selection bias and missing covariates posed by environmental shifts, we propose novel TATE identifiability framework that relaxes the assumptions made in prior work. Based on this theoretical foundation, we propose a novel doubly ro- bust TATE estimation method, 2SDR, which effectively addresses these challenges. Extensive experimental results on synthetic and real-world datasets demonstrate the effec- tiveness of the proposed method and its practical value in real-world applications. The main limitation of 2SDR is that when the dimension of the covariates is extremely high, the shadow variable selection process in the first stage be- comes time-consuming. Additionally, if the number of the partially observable covariates exceeds that of the common covariates shared by both data, becomes challenging to ensure the validity of Assumption 3.2. In future work, we will explore using dimensionality reduction methods, such as representation learning, to address the above limitation.
The ACTG dataset. The RCT data in this dataset is from the ACTG 175, which aimed to investigate the treatment effect of specific drug on the CD4 count of AIDS patients. The dataset contains covariates such as age, gender, weight, race, and pre-treatment CD4 count (Hammer al., 1997). Following Cole & Stuart (2010); Dahabreh et al. (2023), we aim to generalize the results from the ACTG 175 trial to the target population of individuals infecting HIV in the United States in 2006 (Prejean et al., 2008; Hall et al., 2008). In contrast to the ACTG 175, the age and pre-treatment CD4 count are missing in the target population, aligning with Setting 1. Since there is ground-truth TATE in the original observational data from the target population, we cannot evaluate the performance of the TATE estimation results. Therefore, we randomly selected 558 individuals from the ACTG 175 who were either under 30 or over 50 years old and omitted their age, pre-treatment CD4 count, and post-treatment CD4 count to serve as the observational data from the target population, with the remaining 1581 individuals forming the RCT data.
The JTPA dataset. The national JTPA study, which was conducted from 1987 to 1989, aimed to assess the effective- ness of job training programs in helping individuals find em- ployment and increase their earnings (Doolittle Traeger, 1990; Bloom et al., 1993). Following Huang (2024), we selected data from the site of Coosa Valley, Georgia, from the original JTPA trial the RCT data, with their years of education and high school or equivalency diploma status omitted. The data from the remaining 15 sites were used as the observational data from the target population. The RCT data contains 788 individuals, and the observational data contains 5314 4 individuals.
A Acknowledgements
This work was supported in part by the National Key Research and Development Program of China (2024YFE0203700), National Natural Science Foundation of China (62376243, 62441605), and "Pioneer" and "Lead- ing Goose" R&D Program of Zhejiang (2025C02037). All opinions in this paper are those of the authors and do not necessarily reflect the views of the funding agencies.
9

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
Impact Statement
Dahabreh, J., Robertson, S. E.,, Steingrimsson, J. A., Stu- art, E. A., and Hernan, M. A. Extending inferences from randomized trial to new target population. Statistics in medicine, 19(14:1199-220
This paper presents work whose goal to advance the field of Machine Learning. this paper, we report experimen- tal results on two real-world datasets to evaluate the per- formance of our method in practical scenarios. However, the results should not be interpreted as providing direct guidance for treatment assignments in the corresponding real-world applications.
2020.
Dahabreh, I. J., Haneuse, S. A., Robins, J. M., Robertson, S. E., Buchanan, A. L., Stuart, E. A., and Hernán, M. A. Study designs for extending causal inferences from a randomized trial to a target population. anmericanjouraa of epidemiology, 190(8):1632-160) 1642, 2021.
References
Dahabreh, I. J., Robins, J. M., Haneuse, S. J.-P., Saeed, I., Robertson, S. E., Stuart, E. A., and Hernán, M. A. Sensi- tivity analysis using bias functions for studies extending inferences from randomized trial to a target population. Statistics in medicine, 42(13):2029-200
Andrews, I. and Oster, E. A simple approximation for evaluating external validity bias. Economics Letters, 178: 58 62, 2019.
2023.
Bloom, H. S. et al. The national jtpa study. title ii-a impacts on earnings and employment at 18 months. 1993.
Doolittle, F. and Traeger, L. Implementing the national jtpa study. 1990.
Bottou, L. et al. Stochastic gradient learning in neural networks. Proceedings of Neuro-Numes, 91 1991.
Duan, J., Pelger, M., and Xiong, R. Causal inference for large dimensional non-statie non-stationary panels with two-way endogenous treatment and latent confounders. Available at SSRN, 2024a.
Buchanan, A. L., Hudgens, M. G., Cole, R., Mollan, K. R., Sax, P. E., Daar, E. S., Adimora, A. A., Eron, J., and Mugavero, M. J. Generalizing evidence from randomized trials using inverse probability of sampling weights. Journal of the Royal Statistical Society Series A: Statistics in Society,
Duan, J., Pelger, M., and Xiong, R. Target pea: Transfer learning large dimensional panel data. Journal of Econo- metrics, 244(2): 105552,,,,, 2024b.
191-1191-1111 1209, 2018.
d' Haultfoeuille, X. new instrumental method for dealing with endogenous selection. Journal of Econometrics, 154 11):1-15, 2010.
Cole, S. R. and Stuart, E. A. Generalizing evidence from randomized clinical trials to target populations: the actg 320 trial. American journal of epidemiology, 172(1): 107-115, 2010.
Egami, N. and Hartman, E. Covariate selection for general- izing experimental results: Application large-scale development program in uganda. Journal of the Royal Statistical Society Series A: Statistics in Society, 184(4): 1524-1548, 2021.
Collaboration, F. S. Systematically missing confounders in individual participant data meta-analysis of observational cohort studies. Statistics in medicine, 28(8):1218-125) 2009.
Fisher, R. A. The design of experiments. Springer, 1971.
Colnet, B., Josse, J., Varoquaux, G., and Scornet, F. Causal effect on target population: a sensitivity analysis to handle missing covariates. Journal of Causal Inference, 10(1):372-4114 414,, 2022.
Hall, H. I., Song, R., Rhodes, P., Prejean, J., An, Q.
Lee, L. M., Karon, J., Brookmeyer, R., Kaplan, E. H., McKenna, M. T., et al. Estimation of hiv incidence in the united states. Jama, 300(5):520-55)) 5290 2008
Colnet, B., Mayer, I., Chen, G.. Dieng, A., Li, R., Varo- quaux, G., Vert, J.PP., Josse, J., and Yang, S. Causal inference methods for combining randomized trials and observational studies: a review. Statistical science, 39(1): 165-191, 2024.
Hammer, S. M., Squires, K. E., Hughes, M. D., Grimes, J. M., Demeter, M., Currier, J. S., Eron Jr. J., Fein- berg, J. E., Balfour Jr, H. H., Deyton, I.. R., et al. controlled trial of two nuclcoside analogues plus indi- navir in persons with human immunodeficiency virus infection and cd4 cell counts of 200 per cubic millime- ter or less. New England Journal of Medicine, 337(11): 725-733, 1997.
Dahabreh, I. J. and Hernán, M. A. Extending inferences from a randomized trial to a target population. European journal of epidemiology. 34:719-722 2019.
Dahabreh, I. J., Robertson, S. E., Tchetgen, E. J., Stuart, E. A., and Hernán, M. A. Generalizing causal inferences from individuals in randomized trials to all trialeeligible individuals. Biometrics, 75(2):685-66) 6944, 2019.
Hariton, E. and Locascio, J. Randomised controlled tri- als-the gold standard for effectiveness research. BJOG: an international journal of obstetrics and gynaecology, 125(13):1716, 2018
10

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
Hartman, E., Grieve, R., Ramsahai, R., and Sekhon, J. S. From sample average treatment effect population erage treatment effect on the treated: combining experi- mental with observational studies to estimate population treatment effects. Journal of the Royal Statistical Society Series A: Statistics in Society, 178(3):55-777, 2015.
on knowledge discovery & data mining, pp. 1617-1626. 2018.
Kuang. K., Xiong, R., Cui, P., Athey, S., and Li, B. Stable prediction with model misspecification and agnostic dis- tribution shift. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 1485-4492 2020.
Hernán, M. A. and Robins, J. M. Causal Inference: What If. Boca Raton: Chapman & Hall/CRC, 2020.
Lee, D., Yang, S., Dong, L., Wang. X., Zeng, D., and Cai, J. Improving trial generalizability using observational studies. Biometrics, 79(2):1213-1225, 2023.
Huang, M., Egami, N., Hartman, E., and Miratrix, L. Lever- aging population outcomes to improve the generalization of experimental results: Application to the jtpa study. The Annals of Applied Statistics, 17(3):2139-21) 2164, 2023.
Lesko, C. R. Buchanan, A. L.., Westreich, D., Edwards, J. K., Hudgens, M. G., and Cole, S. R. Generalizing study results: a potential outcomes perspective. Epidemiology, 28(4):553-561 2017.
Huang, M. Y. Sensitivity analysis for the generalization of experimental results. Journal of the Royal Statistical Society Series Statistics Society, pp. qnae012, 2024.
Li, B., Li, H., Wu, A., Zhu, M., Cao, Q. Kuang, K., et al. A generative approach for treatment effect estimation under collider bias: from an out-of-distribution perspec- tive. In Forty-first International Conference on Machine Learning, 2024a.
Hughes, R. A., Heron, J., Sterne, J. A., and Tilling, K. Ac- counting for missing data in statistical analyses: multiple imputation is not always the answer. International journal journaal of epidemiology, 48(4): 1914::2224113300 1304, 2019.
Li, B., Li, H., Xiong, R., Wu, A., Wu, F., and Kuang, K. Learning shadow variable representation for treatment effect estimation under collider bias. In Forty-first Inter- national Conference on Machine Learning, 2024b.
Imbens, G. W. and Rubin, D. B. Causal inference in statis- tics, social, and biomedical sciences. Cambridge univer- sity press, 2015.
Li, B., Wu, A., Xiong, R., and Kuang, K. Two-stage shadow inclusion estimation: an IV approach for causal inference under latent confounding and collider bias. In Forty-first International Conference on Machine Learning. 2024c.
Jolani, S., Debray, T. P., Koffijberg, H., van Buuren, S., and Moons, K. G. Imputation of systematically missing predictors in an individual participant data meta- mnta-analysis: generalized approach using mice. Statistics in medicine, 34(11):1841-1663 2015.
Li, C. and Fan, X. On nonparametric conditional indepen- dence tests for continuous variables. Wiley Interdisci- plinary Reviews: 2020.
Kallus, N., Puli, A. M., and Shalit, U. Removing hidden confounding by experimental grounding. Advances in neural information processing systems, 31, 2018.
Computational Statistics, 12(3)e14889
Little, R. J. and Rubin, D. B. Statistical analysis with missing data. John Wiley & Sons, 2019.
Kennedy, F. H. Towards optimal doubly robust estimation of heterogeneous causal effects. Electronic Journal of Statistics, 17(2):3008-3000 2023.
Luo, H., Zhuang, F,, Xie, R., Zhu, H., Wang, D., An, Z., and Xu, Y. A survey on causal inference for recommendation. The Innovation, 2024.
Kern, H. L., Stuart, E. A., Hill, J., and Green, D. P. As- sessing methods for generalizing experimental impact estimates to target populations. Journal of research on educational effectiveness, 91):103-127, 2016.
Mayer, I., Josse, J., and Group, T. Generalizing treatment effects with incomplete covariates: Identifying assump- tions and multiple imputation algorithms. Biometrical Journal, 65(5):21000) 100294, 2023.
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In International Conference Leurning Representations, 2015.
Miao, W. and Tchetgen Tchetgen, E. J. On varieties of doubly robust estimators under missingness not at random with a shadow variable. Biometrika, 103(2):475-482. 2016.
Kohavi, R. and Longbotham, R. Unexpected results in online controlled experiments. Acm Sigkdd Explorations Newsletter, 12(2):31-35 2011.
Miao, W., Liu, L., Li, Y., Tchetgen Tchetgen, E. J., and Geng, Z. Identification and semiparametric efficiency the- ory of nonignorable missing data with a shadow variable. ACM/JMS Journal of Data Science, 1(2):1-23, 2024.
Kuang, K., Cui, P., Athey, S., Xiong, R., and Li, B. Stable prediction across unknown environments. In proceed- ings of the 24th ACM SIGKDD international conference

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
Newey, W. K. and McFadden, D. Large sample estimation and hypothesis testing. Handbook of econometrics, 4: 2111-2245, 1994.
Stuart, E. A., Cole, S. R., Bradshaw, C. P., and Leaf, P. J. The use of propensity scores to assess the generalizability of results from randomized trials. Journal of the Royal Statistical Society Series A: Statistics in Society, 174(2): 369-386, 2011.
Nguyen. T. Q.. Ebnesajjad, C.,, Cole, S. R., and Stuart, E. A. Sensitivity analysis for an unobserved moderator in ret- eestarget-populatie generalization of treatment effects. The Annals of Applied Statistics, pp. 225-247, 2017.
Tipton, E. Improving generalizations from experiments using propensity score subclassification: Assumptions, properties, and contexts. Journal of Educational and Behavioral Statistics, 38(3):239-266, 2013.
Nguyen, T. Q., Ackerman, B., Schmid, I., Cole, S. R., and Stuart, E. A. Sensitivity analyses for effect modifiers not observed in the target population when generalizing treatment effects from randomized controlled trial: As- sumptions, models, effect scales, data scenarios, and im- plementation details. PloS one, 13(12)=0008875, 2018.
Womack, J., Tien, C., Feldman, J., Shin, J. H., Fennie, K., Anastos, K., Cohen, M. H., Bacon, M. C.,, and Minkoff, H. Obesity and immune cell counts in women. Metabolism, 56(7):998-1004, 2007.
O' Muircheartaigh, C. and Hedges, L. V. Generalizing from unrepresentative experiments: a stratified propensity score approach. Journal of the Royal Statistical Society Seriex C: Applied Statistics, 63(())195-2110, 2014.
Wu, C. and Hamada, M. S. Experiments: planning, anal- ysis, and optimization. John Wiley & Sons, 2011.
Xiong, R. and Pelger, M. Large dimensional latent factor modeling with missing observations and applications to causal inference. Journal of Econometrics, 233(1): 301, 2023.
Pearl, J. and Bareinboim, E. Transportability of causal and statistical relations: A formal approach. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 25, pp. 247-254, 2011.
Xiong, R., Athey, S., Bayati, M., and Imbens, G. Optimal experimental design for staggered rollouts. Management Science, 70(8):5317-5336, 2024a.
Pearl, J. and Bareinboim, E. External validity: From do-calculus to transportability across populations. In Probabilistic and causal inference: The works of Judea Pearl, pp. 451-482. Association for Computing Machin- ery, 2022.
Xiong, R., Chin, A., and Taylor, J. Data-driven switch- back experiments: Theoretical tradcoff's and empirical bayes designs. arXiv preprint arXiv:2406.0677 2024b.
Prejean, J.. Song, R., An, Q.. and Hall, H. Subpopulation es- timates from the hiv incidence surveillance cestem-united states, 2006. MMWR: Morbidity & Mortality Weekly Re- port, 5736), 2008.
Zhang, K., Peters, J., Janzing, D., and Schölkopf, B. Kernel- based conditional independence test and application in causal discovery. In 27th Conference on Uncertainty in Artificial Intelligence (UAI 2011), pp. 804-813, AUAI Press, 2011.
Resche-Rigon, M., White, I. R., Bartlett, J. W., Peters, S. A., Thompson, S. G., and Group, P.-I. S. Multiple imputa- tion for handling systematically missing confounders in meta-analysis of individual participant data. Statistics in medicine, 32(28):4890-4900 2013.
Zheng, Y., Huang, B., Chen, W., Ramsey, J., Gong, M., Cai, R., Shimizu, S., Spirtes, P., and Zhang, K. Causal-leam: Causal discovery in python. Journal of Machine Learning Research, 25(60):1-8, 2024.
Rubin, D. B. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of edu- cational Psychology, 66(5):688, 1974.
Zou, H. The adaptive lasso and its oracle properties. Journal of the American statistical association, 101(476):1418- 1429, 2006.
Sen, R., Suresh, A. T., Shanmugam, K., Dimakis, A. G., and Shakkottai. S. Model-powered conditional independence test. Advances in neural information processing systems, 30, 2017.
Zou, H. and Hastie, T. Regularization and variable selection via the clastic net. Journal of the Royal Statistical Society Series B: Statistical Methodology, 67(2):001-320, 2005.
Silverman, B. W. Density estimation for statistics and data analysis. Routledge, 2018
Strobl, E. V., Zhang, K., and Visweswaran, S. Approximate kernel-based conditional independence tests for fast non- parametric causal discovery. Journal of Causal Inference, 7(1):2018001) 2019.
12

---

similar to Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
A. Related Work
RCTs have been widely used to study the effects of new interventions in a wide range of disciplines (Fisher, 1971; Wu & Hamada, 2011: Xiong et al., 2024a;b). Recently, many studies have aimed generalize treatment effects from RCTs to target populations by combining RCT data with observational data from the target population to estimate the TATE, Prior work show that the TATE can be identified by separating set (Cole & Stuart, 2010; Tipton, 2013; Kern et al., 2016; Egami & Hartman, 2021; Pearl & Bareinboim, 2022). The separating set is set of variables affecting both treatment effect heterogeneity and environmental shifts. Many approaches are proposed to adjust for the separating set through different technologies for generalizing treatment effects from RCTs to target populations (Stuart et al., 201 I; Pearl & Bareinboim, 2011; O' Muircheartaigh & Hedges, 2014; Hartman et al., 2015; Lesko et al.. 2017; Buchanan et al., 2018; Dahabreh & Hernán, 2019; Dahabreh et al., 2019; Lee et al., 2023; Li et al., 2024a). For example, Cole & Stuart (2010); Stuart et al. (2011) proposed first estimating the selection score, i.e., (P = 1 X). and then using to reweight the RCT samples in order to adjust for X. Hartman et al. (2015) proposed the Calibration Weighting (CW) method, which learns sample weight matrix that constrains the covariate distribution in the RCT data align with that of the observational data from the target population. The sample weight matrix simulates the selection score and avoids issues arising from model misspecification. Dahabreh & Hernán (2019); Dahabreh et al. (2019) proposed a regression- -basd method, G-formula, and a doubly robust method, Augmented IPSW (IPSW), to adjust for the covariates more effectively. Lee et al. (2023) also proposed a doubly robust version of CW, Augmented CW (ACW), to enhance the robustness of the estimation.
All the above methods assume that the covariates shared by both groups contain the separating set. However, in real-world scenarios under environmental shifts, it is challenging to ensure that the covariates collected in both groups are exactly the same, leading to the missing covariates problem-where certain covariates present in one group are entirely absent in the other. If the missing covariates include variables from the separating set, previous methods may fail to perform as expected. Recently, several sensitivity analysis methods were proposed to detect whether the fully observed covariates contain all the variables from the separation set (Nguyen et al., 2017; Andrews & Oster, 2019; Colnet et al., 2022; Dahabreh et al., 2023; Huang, 2024). However, to the best of our knowledge, no existing work can identify and estimate the TATE when the separating set is observable in only one of the two datasets.
This work also complements the recent literature on treatment effect estimation and parameter inference for panel data with nonrandom missingness (Xiong& & Pelger, 2023; Duan et al., 2024a). In our setting, covariates form partially observed matrices in either the RCT or observational data, and the missingness is affected by experimental shifts. This is closely related to Duan et al. (2024b), which transfers information across panels under general our use of 2SDR to transfer information from RCT to observational data.
B. Discussion on the Scenarios of Environmental Shifts Applicable to This Paper
The applicable scenarios of environmental shifts for this paper can be summarized as follows:
Shifts in covariates. As stated in Definition 2.1, the original definition of environmental shifts in this paper is covariate shifts, i.e., PXX| R 1) + P(X R = 0).
Shifts in both the outcome and covariates. Although this is not explicitly stated in Definition 2.1, due to the fact that some covariates may be causes of the outcome, the distribution of the outcome will also shift along with the distribution
of such covariates, i.e., P(Y R = 1) A TP(Y R 0), or alternatively. P(X,Y Y R = 1) + P(X, Y R -). Shifts in the conditional distribution of the outcome. Given that (I) IL R X holds under Assumption 3.1, we have P(Y X, R = 1)= = IP(Y X, R = 0). Therefore, our problem setting essentially assumes the absence of Y X shifts. However, for the common covariates X shared by the two datasets, there still exist X shifts, i.c.
P(Y X R 1) * P(Y X = 0). Assumption 3.2 required by the proposed 2SDR is still satisfied under Y X shifts, and therefore, 2SDR can address Y X shifts. However, existing methods rely on Assumption 2.8, which requires that (t) 1L R| X holds. Under Y X shifts, Assumption 2.8 does not hold, and thus, in contrast to 2SDR, previous methods cannot address Y X shifts.
C. Proof of Theorem 2.9
Proof. Under Assumptions 2.3, 2.4, and 2.6 we have (t) W, R = 11= w,T=t,R=11
From Definition 2.7, we obtain E|y| W, T, R =
1] =
E
W, T] = E
W T, R = 1] under Assumption 2.5.
13

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
Therefore, based Definition 2.2, we have
(L W w, = - W w, R 01) (F w R = 0)
-(EEY(I) | W = w| E(Y W = w)) dF(WF =w w -
D. Proof of Lemma 3.4
Proof. First, substituting Equation (3) into Equation (2) yields
P X
PIS-1X==0X9 P(X
X 5 = 1)
m S. = 0)
P(S=
m
P
E
FS=0 X X9) -(S-IX
X S
P(S=1 X X
P(S=0X==0X
Under Assumptions 2.5 and 3.2, we have
E
P(S = 0
X
X9)
= -
S 1
P(S
X m X
P(S=
X = X.)
P(S = 0
X x x9)
P(S =15=10 X = 0,X;)
Σ
IP x X -
P(S- 1
X = x x0)
P(S X m = 0,X))
xm
Σ
P(S = 0
X = x X9)
P(S =
X = 0,X*)
P(S X x X²)
P(X x x
P(S- 1
m x X
P(S
X m 0,X*)
P(S X
xm
IP(S 1 X = 0,X*)
Σ
P(S = X x x=, P(X = x X9)
P(S = 0 X = 0.X PP(S = 1 X
X"I
PPS = 1 X = 0,
Σ P(S = 0, =x X9)
IPS = 0 X = 0,X IPS = X²)
PPSS 1 T X
PPS 0 X9
PPS = 0 X = 0, X
PPS = 1 X9)
Therefore, we have
P(= -(=-IX==0X F(X
X S = 1)
P P
E
P( X
X S =1
F(S=1X
X
PIS=IX"=0
PX 5 = 1)
P(S
X X
P 0X
-
X9)
F(5=0 x.x
P[
IP(S = 0 X P
PPX X 1)
P(S X X X.)
P(S = 0| Xm x P($ X
P(S 1 X
IP - Xth Xm PPS = 0 PPSS 0,
P(S 0.)
P(X m
X = 0).
Consequently, the right-hand side of Equation (2) equals the left-hand side, which proves the correctness of Equation (2).
14

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
E. Proof of Lemma 3.5
Proof. First, substituting Equation (5) into Equation (4) yields
OR(X ,
OR(X t X9) B TM X9) -
OR(X" 0, 99) (I", X9) -
OR(X",,X. )
OR(X" = 0, X9)
where (R(X 0, x 1, obtained by substituting X into Equation (3).
Therefore, the right-hand side of Equation (4) equals the left-hand side, which proves the correctness of Equation (4).
Next, under Assumptions 2.5 and 3.2, we have
x = P(S x xt). P(S ,
Therefore, according to Equation (3), we have
OR(X (x) = OR(X.. ;x))
Based on the proof in Appendix D, we have
P(S = 1 X 0,
P(S 0 X
E[OR(X TM X") X S = 1|=
P(S = 0 X = 0, X.X
P(S = X
Substituting the above three equations into Equation (5) yields
OR(X X*)
O X²)
-[==1|===0,'' P(S=
P(S=0)X"=0,XX P((=I|X-)
P(S X X P(S 1
P(S X X²) P(S X
P(S X X9) P(S = X
P(S = Xo) P(S = X*)
15

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
Therefore, under Assumptions 2.5 and 3.2, have
B[OR(X X*) X = P(S = X X9) P(S = X
= E
X 5 S
P(S = 1 X P(S = 0 X
P(S X X P(S X
E
X
P(S = X X P(S - 0 X
P(S X x X P(S
X
P(X =x 1x))
P(S =
X 0 xx",,, x:)
P(S 0 X
P(S = X
P(S X x X
P(X x 1x9)
P(S = X
P(S
X m x X
x
IP(S = 1 X
P(S=0 X
IP(S = 0 X²)
P(S = 1 X
IP(S = 1,X9)
P(S 0, X Z)
IP(S = 0, P(S -,,
PPZ X=
( X 1)
Consequently, the right-hand side of Equation (6) equals the left-hand side, which proves the correctness of Equation (6).
Equation (6) is Fredholm integral equation of the first kind, with OR(X X to be solved for. Based on Theorem from Miao et al. (2024) and Lemma 3.4, Equation (6) has a unique solution under Assumptions 2.5 and 3.2. Therefore, OR(X" X is identifiable.
F. Proof of Corollary 3.7
Proof. From Equation (3), it follows that
P(S = x = (1+ OR(XXXXX""XX x.) P(S 0 XTh 0,X*)
-1
F(S = 1 x = 0,
The identifiability of this equation depends on the identifiability of OR(X", x²) and P(S X =,,X", both of which can be identified with Z under Assumptions 2.3, 2.4, 2.5, 2.6, 3.1, and 3.2.
First, under Assumptions 2.5 and 3.2, we have
P(S x
P(S = X 0,
1+ P(S X9). OR(X". OR(X"..'
The correctness of the above equation is guaranteed by substituting Equations (3) and (5) and the corresponding equation of E[OR(X"., X*) X S 1] in Appendix into its right-hand side, which equals the left-hand side.
Next, based on Lemma 3.5, (R(X X and OR(X"" X are identifiable under Assumptions 2.3, 2.4, 2.5, 2.6, 3.1, and 3.2, which consequently ensures the identifiability of P(S = X 0, X³).
16

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
With both P X | 0, ) and OR(X" X identified, P(S X) is identified
P(S=0) X
P(S = -(S=1|X) 1+ OR( (x) D[S=1 X+)
ORIX X +
ORRX X
P(S=1[X*) ORIX", X
P(S=1|X> U X PP=00 X
DR(X" XX)
P(S = X
1+ OR(X". X
P(S = 1 X
OR(X=.X:
P(S = 0 X
1+ OF X X9)
IP(S = 1 X9)
Moreover, P(R 1| X) is identified as
PPS = 1 P(S = X)
for Setting for Setting
P(R = P(R=1IX) 1|)
since S in Setting 1, and R 1 5 in Setting 2.
G. Proof of Theorem 4.2
Proof. We only need to prove the unbiasedness of vi and then the consistency of & in large samples holds under the conditions specified in Newey & McFadden (1994); Miao & Tchetgen Tchetgen (2016).
Step 1. Unbiasedness of the odds ratio model.
Given the conditions specified in Silverman (2018), f(z X),5) is a consistent estimate of P(Z S)
Therefore, based Equation (6), if the OR model is correctly specified, the following equation holds.
f(z X 5 =) f(z X S = =) = 0.
O x²)
Therefore, the unbiasedness of the OR model holds, and thus the unbiasedness of the odds ratio model also holds based on Equation (4).
Step 2. Unbiasedness of the doubly robust imputation model.
To prove the unbiasedness of & ())),, we need prove that (xx) X
0 holds.
Under Assumption 3.2, it reduces to proving that the following equation holds.
X m 8(Xx)
E (X*) X
S
+ +8(x9)
-X
ira (X)
S
-1)
(xm - -8(x9)
A.. (X)
E
S
-1 11
- 8(xx) (xmm
X x
A. (X)
(S X) -1) (X (x--
i (X)
We provide proofs for the following two cases.
17

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
(I) The regression model of the missing covariates is correctly specified while the selection score model not. Based on Equation (2), we have
E
X
X
S 0]
X
PXX
x
X 5= = 0]
X
OR(X Th
x X 1)
EOR
X S=
X"
OR xm, X P( = x x9) P(S= Th = X*)
E|OR(X X x = P(S X9)
E (S OR (X X9) X.)
X
S O
m X0) X
Therefore, we have
E X9) x x
X -) OR(X") x²) X9
and
[ OR (X"h, -x" = X
E [S OR (X" X ) X -
OR -X
0)) X
E [(S OR (x x-) ) X E x 0] EiS OR(X", X
=0
Then, have
S -1) 1) (X (X" E|X" S 01)
E
(X)
S S. (x). P(S - X = 0.X*)
(X -[X" X S = =01) 0
=E
PPS = X 0,X²)
(S. [Xm x²)
P(S = 0 _ = =0 X.)
(X" - E x = 01) x
E
E
P(S = X 0, X³)
Given the above equations, the regression model of m is correctly specified and the odds ratio model is unbiasedly estimated, 8 (Xx) is unbiased estimate of E[X X S 0]. Consequently, we have
[ (x(()-1)
S
(xm -1):
8(X9) 0.
E
Therefore, the unbiasedness of (X*) holds.
(2) The selection score model is correctly specified while the regression model of the missing covariates is not. In this case, as the selection score model of S is correctly specified, 7(X) is an unbiased estimate of P(S = 1 X²). Therefore, given the unbiasedness the odds ratio model, based on Corollary 3.7, the unbiasedness of न. (X) holds under Assumptions 2.3, 2.4, 2.5, 2.6, 3.1, and 3.2. Consequently, we have
P (SS=1
X (X)
Therefore, the unbiasedness of & (X)) holds.
H. Proof of Theorem 4.3
Proof. When either t, (X) or (i,()) consistent, the consistency theory of the doubly robust TATE estimator in large samples has already been established in Little & Rubin (2019). Therefore, we focus on proving the consistency of R, (X) and (X) here.
18

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
(I) Under Assumptions 2.3, 2.4, 2.5, 2.6, 3.1, and 3.2, if the selection score model of S and the OR model are correctly specified, at (X) is consistent, proved in Appendix G. Since #, (X) equals f (X) or 7 (X) based Corollary 3.7,
(X) is also consistent.
(2) Given that the imputation model is consistent, if the outcome regression model is correctly specified, (x(X) estimated using the imputed data still maintains consistency under the conditions specified in Newey & McFadden (1994); Little & Rubin (2019).
Therefore, at least one of (X) and fi (X) consistent, and thus is consistent.
Time Complexity Analysis of the Shadow Variable Selection Process in Stage I of 2SDR
In Setting 1, where the values of X are missing in Dataset O, the time complexity of the initial screening procedure is Onn 1 (Zou, 2006), where = p - d the number of variables in X and p is the number of variables in X; the time complexity of the test for Assumption3.2(1 is O(no p² nR p²) (Strobl et al., 2019); and the time complexity of the test for Assumption3.2c4
which uses the mini-batch stochastic gradient descent method for optimization, is O(nr k²) (Bottou et al., 1991). Similarly, in Setting 2, where the values of X are missing in Dataset R. the time complexity of the initial screening procedure is O 1-2); the time complexity of the test for Assumption3 ((1 is (mmp² + np²) ); and the time complexity of the test for Assumption3.2(1) is O p² + RR p²). As a result, the time complexity of the entire shadow variable selection process is O(np²) ) = O p² ng p²))
The time complexity may vary depending on the chosen method for testing conditional independence. Our approach is flexible and can incorporate any Conditional Independence Testing (CIT) method, with the specific choice of method depending on its characteristies and the practical context (Li & Fan, 2020; Zheng et al., 2024).
J. Experimental Details
J.1. Software and Hardware Used
Software used: Python 3.9 with PyTorch 1.13.0.
Hardware used: Windows 11 operating system with a 13th Gen Intel(R) Core(TM) i7-13700K CPU and an NVIDIA GeForce RTX 3080 GPU (with CUDA version 12.1).
J.2. Implementation Details
In the experiments conducted on the diw-iimesioonl synthetic datasets and the real-world datasets, we used Elastic Nets (Zou & Hastie, 2005) for continuous variable estimation and logistic regression for binary variable estimation. In the experiments conducted on the high-dimensional synthetic dataset, neural networks were employed for these tasks, using the Adam optimizer (Kingma & Ba, 2015) with the initial learning rate being 0.003. We used cross-validation to determine the hyperparameters. Based on Theorem 3.3, for the methods involving imputation, during the coss-validation process on the datasets under Setting 2, evaluated the performance of the models trained on the RCT training set not only on the RCT validation set but also on the validation set from the observational data from the target population to ensure the accuracy of the imputation.
K. Data Generation Process for the Synthetic Datasets
K.1. Data Generation Process for the Low-Dimensional Datasets
We first used simulation framework similar that in Colnet et al. (2022) to generate the covariates. Specifically, we generated X2 ~ (O, 1) and generated X1, X3, X1, and X; as
X1, X4 ~ N (0)
(15
0.) )
0.8
and
(())
(1
0.8)
1 ).)).
X3. Xs ~ N
0.8
19

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
Table TATE estimation results (MAF) the high-dimensional synthetic dataset (mean:sstd), with hold values indicating the hest
performance.
METHODS
SETTING 1
SETTING
RCT
5.049 166
2.62342.284
IPSW
8.530=3.741
9.412=t4.631
CW
74.47144.95
06.51131.88
G-FORMULA
0.670:10.365
0.686:10.462
AIPSW
0.664110.367
0.686
ACW
0.668=0.366
0.685=00.400
IPS Imp
8.173=3.752
9.533
CWImp Imp
79.23=64.57
9.859=6.671
G-FORMUL
0.612+0.366
1.518+0.750
AIPSW WeTWIn Imm
0.396
4.542+3.860
ACW Atmm
0.612+0.365
1.518+007751
2SDR
0.43310.291
0.46110.277
Next, to simulate environmental shifts between the RCT data and the observational data from the target population, we generated R using the following logit model:
lo X Xa)) -0.5x + 0.5X; - 0.3X4
where 6, (0,
In Setting 1, the set of observable covariates in the RCT data is {X1,. X4. XsJ. while the set of observable covariates in the observational data from the target population is {Xi, X4, Xs). with X2 being unobservable in both datasets. In Setting 2, the set of observable covariates in the RCT datas is {X1, X1, Xs). while the set observable covariates in the observational data is (X1, X3, X4, Xs).
For the RCT data, randomly generated T ((0.5), where B(-) denotes the Bernoulli distribution. In contrast, for the observational data from the target population, all values of I were set to 0.
Finally, we generated Y as
Y = 5X, + T (5X, + 5X, - -3X,) + Cy.
where cy ~ N0.
K.2. Data Generation Process for the High-dimensional Dataset
The generation of R and Y in the high- digh-siimeniinall dataset followed the same process as in the low-dimensional datasets, with slight modification in the generation of the covariates. Specifically, we first generated X₃ ~ ~N, I2o), X₂ ~ N/ Is), and X, ~ (O Io), where I, denotes the identity matrix with e rows and columns. Subsequently, we generated X; and X, as
X; = X₁A,
x, = X,B,
where A is a 10 5 matrix with each element sampled from A/ 1), and B is 20 10 matrix with each element sampled from 1).
In the nigh-dimensional dataset, we set nı = 2000 and no = 10000. The final dataset contains 50 covariates, including 5 partially observable, completely unobservable, and 40 fully observable variables.
20

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
Table TATE estimation results (MAR) of 2SDR under different degrees of violation of Assumption 3.2 (mean+sstd).
DEGREES
CASE I
CASE
SEVERE
0.355:10.319
1.383310.280
SLIGHT
0.31110.293
0.34111.281
NONE
0.268
0.268:10.212
Table Imputation error (MSE) on different datasets (mean:tstd), with bold values indicating better performance.
DATASETS
BASELINE
2SDR
LOW-DIM2060 SETTING
0.684+0.027
0.509=0.008
LOW-DIM 1000 SETTING
0.694110004
0.552-0.038
LOW-DIMs00 SETTING
0.786110.060
0.555
LOW-DIM2000 SETTING
0.386:10.013
0.230-0.009
LOW-DIM1000 SETTING
0.397=0.019
0.245-0.011
LOW-DIMsoc SETTING 2
0.447=0.023
0.310=0.057
HIGH- DIM SETTING
0.900=0.020
0.418-0.186
HIGH-DIM SETTING
0.986+00034
0.651
ACTG
1.897+0.180
1454-0.033
JTPA
2.130+0.111
0.992-0.033
L. Supplementary Experimental Results
L.1. Experimental Results on the High-Dimensional Synthetic Dataset
We report the TATE estimation results on the high-dimensional synthetic dataset in Table 1. It can observed that, although, like all the baselines, the performance of 2SDR gets worse compared to the results on the corresponding low-dimensional dataset, it still achieves the best performance, demonstrating its effectiveness in handling high-dimensional data.
L.2. Robustness Analysis when Assumption 3.2 Is Violated
To evaluate the robustness 2SDR when the core assumption required by it At-Asuuptiion 3.2-is violated, we modified the synthetic datasets to no longer satisfy Assumption 3.2 and conducted experiments on them to observe the performance of 2SDR. Specifically, we modified the synthetic datasets simulate the following cases:
Case Assumption 3.2(2) violated. We changed the coefficient of X; on R from 0 (indicating no violation of the assumption, represented by 'None') {0.1 (indicating slight violation of the assumption, represented by 'Slight'), 0.3 (indicating severe violation of the assumption, represented by 'Severe')) to introduce weak dependence between the shadow variable and environmental shifts.
Case 2: Assumption 3.2(2) is violated. We reduced the correlation coefficient between Xs and X3 from 0.8 (None) to {0.3 (Slight), 0.1 (Severe)), so that the shadow variable only has a weak predictive ability for the missing covariate.
As shown in Table 4, while the performance of 2SDR deteriorates when Assumption 3.2 is violated, it does progressively as the degree of violation increases. The results demonstrate that 2SDR maintains a certain level of robustness even when Assumption 3.2 is not satisfied.
L.3. Imputation Accuracy Analysis
To evaluate the imputation accuracy of 2SDR, we compare the imputation error (Mean-Square Error, MSE) of 2SDR with that of the baselines performing imputation based on T X S = 1], as reported in Table 5. The results show that an increase in the dimensionality of the data and a decrease in the RCT scale lead to an increase in the imputation error. However, 2SDR consistently outperforms the baseline on all datasets, demonstrating its superior imputation accuracy.
21

---

Generalizing Causal Effects from RCTs to Target Populations across Diverse Environments
High-Dimensional Data with ny
2000
Low-Dimensional Data with nı
2000
Good
Good
0.1
20
0.7
0.05
0.05
d1
20
11
0
20
11
0
0.01
0.01
2
20
14
0
Bad
Bad
0.1
0.05
0.01
0.05
0.01
Low-Dimensional Data with ną
1000
Low-Dimensional Data with nı
500
Good
Good
0.1
19
11
0
0,1
18
9
0
0.05
0.05
01
18
d1
0.01
0.01
20
10
0
17
10
0
Bad
Bad
0.1
0.05
0.01
0.1
0.05
0.01
a2
a2
Figure 4. The number of times the shadow variables selected by 2SDR correct within the 20 repetitions of each experiment different synthetic datasets under Setting varied with different thresholds.
L.4. Hyperparameter Analysis
In the shadow variable selection process of the first stage of 2SDR, introduce two hyperparameters, 0ı and 02 the reject thresholds in hypothesis testing. In the experiments on the synthetic datasets under Setting 1, we varied their values to investigate the impact of different thresholds on the accuracy of the selected shadow variables, considering different sample sizes and dimensionalities. We report the number of times the selected shadow variables were correct within the 20 repetitions of each experiment under different thresholds, as shown in Figure 4.
From the results, we have the following observations and conclusions: (1) the experiments on the low-dimensional data, changes in an have little impact. A decrease in 02 leads to the incorrect exclusion of variables that satisfy Assumption 3.2, resulting in the failure to select correct shadow variables. (2) In the experiments on the high- dighndimessionl data, changes in also have significant impact on the accuracy of shadow variable selection, similar to a2. It demonstrates that the conditional independence test becomes more challenging as the dimensionality of the covariates increases. (3) As the sample size decreases, the accuracy of shadow variable selection also declines, indicating that both hypothesis tests require sufficient sample size.
impact the accuracy of shadow variable selection, while changes in 02 have significant
22


Reviews:
[{"review_id": "r7SlcHNVud", "reviewer": "Reviewer_fZcZ", "summary": "This paper studies the problem of generalizing RCTs under environment shifts, particularly shifts in the distribution and quantity of covariates. It relaxes the assumption in the prior literature where the separating set (variables that simultaneously affect treatment effect heterogeneity and environmental shifts) is present in both the RCT dataset and target observational dataset. Instead it assumes that the separating set belongs to at least one of the two datasets. Leveraging a shadow variable, the authors introduce a novel Two-Stage Doubly Robust (2SDR) estimator for the Target Average Treatment Effect (TATE). They support their solution through identification theory and demonstrate its effectiveness on both synthetic and real datasets.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer’s great efforts and insightful comments to improve our manuscript. In below, we address these concerns point by point.\n\n> **[Other Comments Or Suggestions]**\n\nThank you for your valuable suggestions. We will carefully revise our manuscript based on your recommendations.\n1. We will provide more detailed descriptions of the practical examples for Settings 1 and 2.\n2. We will improve the related definitions in the figure and its caption to make them clearer.\n3. We will add a brief summary at the beginning of each proof step to explain what is happening in that step, and we will also provide a diagram to further illustrate it.\n\n> **[Q1] How realistic is it to assume the existence of a shadow variable Z that satisfies Assumption 3.2?**\n\nAssumption 3.2 requires that among the common covariates shared by the two datasets, there exist variables that are correlated with the covariates missing in one of the datasets but do not directly influence environmental shifts (not a direct cause of R). **Assumption 3.2 is reasonable in many real-world scenarios, as, typically, most covariates only indirectly affect R, with relatively few variables being direct causes of R.** Moreover, **Assumption 3.2 is testable**:\n1. Assumption 3.2(1), i.e., $Z\\notindep X^m |\\widetilde{X^c}, S=1$, is testable because all three variables are observable in the dataset with $S=1$, where $S=1$ indicates that $X^m$ is observable in this dataset.\n2. Assumption 3.2(2), i.e., $Z\\indep S |X^m, \\widetilde{X^c}$, is testable for the following reason. Although $X^m$ is unobservable in the dataset with $S=0$, we can still conduct the test based on Theorem 4.1, which does not require the values of $X^m$ in the dataset with $S=0$ since the only expression involving $X^m$ in the equation to be solved, i.e., $\\frac{S}{Q(X^m, \\widetilde{X^c})}$, is always zero when $S=0$.\n\nTherefore, **we can assess whether Assumption 3.2 holds in real-world applications by conducting hypothesis tests on the covariates**.\n\n> **[Q2] Will results deteriorate smoothly w.r.t. the extent to which this Assumption 3.2 is satisfied?**\n\nThank you for your insightful suggestion. **We have conducted experiments in cases where Assumption 3.2 is violated.** Specifically, we conducted experiments on the synthetic dataset under the following cases: \n1. **Case 1: Assumption 3.2(2) is violated.** We changed the coefficient of Z on S from 0 to \\{0.1(Low), 0.3(High)\\} to introduce weak dependence between Z and S;\n2. **Case 2: Assumption 3.2(1) is violated.** We reduced the correlation coefficient between Z and $X^m$ from 0.8 to \\{0.3(Low), 0.1(High)\\}, so that Z only has a weak predictive ability for $X^m$.\n\n|Extent|Case 1|Case 2| \n|:-:|:-:|:-:|\n|High|0.355$\\pm$0.319|0.383$\\pm$0.280|\n|Low|0.311$\\pm$0.293|0.341$\\pm$0.281| \n|None|0.268$\\pm$0.212|0.268$\\pm$0.212| \n\nThe results demonstrates that **while the performance of 2SDR does deteriorate when Assumption 3.2 is violated, it deteriorates smoothly as the extent to which Assumption 3.2 is violated increases**. We will include the results and analysis of this experiment in the revised manuscript.\n\n> **[Q3] Typically, under environmental shifts, there may also be shifts in Y|X shifts. It would be beneficial if the authors discussed situations where we expect only X-shifts and where both X- and Y-shifts might occur, thereby clarifying where 2SDR can be effectively applied?**\n\nThank you for your valuable suggestion. 2SDR relies on Assumption 3.1, which requires that $Y(t)\\indep R|X^m,X^c$ holds. Therefore, the applicable situations for 2SDR can be summarized as follows:\n1. **X-shifts.** As stated in Definition 2.1, the original definition of environmental shift in our work is X-shifts, i.e., $P(X|R=1)\\neq P(X|R=0)$.\n2. **Both X- and Y-shifts caused by X.** Although we did not explicitly state this in Definition 2.1, due to the fact that variables in X may be causes of Y, the distribution of Y will also shift along with the distribution of X, i.e., $P(Y|R=1)\\neq P(Y|R=0)$, or alternatively, $P(X,Y|R=1)\\neq P(X,Y|R=0)$.\n3. **$Y|X^c$ shifts.** Given that $Y(t)\\indep R|X$ holds, we have $P(Y|X,R=1)=P(Y|X,R=0)$. Therefore, our problem setting essentially assumes the absence of Y|X shifts. However, for the common covariates $X^c$ shared by the two datasets, there still exist $Y|X^c$ shifts, i.e., $P(Y|X^c,R=1)\\neq P(Y|X^c,R=0)$. Assumption 3.2 required by 2SDR is still satisfied under $Y|X^c$ shifts, and therefore, 2SDR can address $Y|X^c$ shifts. However, existing methods rely on Assumption 2.8, which requires that $Y(t)\\indep R|X^c$ holds. Under $Y|X^c$ shifts, Assumption 2.8 does not hold, and thus, **in contrast to 2SDR, previous methods cannot address $Y|X^c$ shifts**.\n\nWe will include the above discussion in the revised manuscript.\n\n***\n**We hope the above discussion will fully address your concerns about our work, and we would really appreciate it if you could be generous in raising your score. Thank you!**", "claims_and_evidence": "(1) The paper extends the TATE identifiability conditions, which traditionally require the separating set to be fully observable in both the RCT and the target observational datasets. Instead, it allows for partial observability, requiring the separating set to be present in at least one of the datasets. This generalization is theoretically supported by a novel identifiability framework.\n\n(2) The authors propose a Two-Stage Doubly Robust (2SDR) TATE estimator that leverages a shadow variable Z. Empirically, the estimator outperforms existing baselines.", "methods_and_evaluation_criteria": "Yes, the proposed method appears to be novel, and the evaluation seems to be well-suited and thorough.", "theoretical_claims": "I have reviewed the proofs of Lemma 3.4 and Lemma 3.5, and they seem to be correct to me.", "experimental_designs_or_analyses": "I haven’t reviewed the dataset generation procedures in detail, but the overall experimental design and analysis seem appropriate and suggest that 2SDR outperforms the alternatives.", "relation_to_broader_scientific_literature": "The paper generalizes the TATE identifiability conditions, which traditionally require the separating set to be fully observable in both the RCT and target observational datasets, to allow for partial observability. In this case, the separating set needs to be a subset of one of the datasets.\n\nIn general, generalizing RCT results beyond the sites where they are conducted is an important problem. This paper addresses environmental shifts, particularly changes in the distribution and quantity of covariates, to enhance the generalizability of RCT results.", "essential_references_not_discussed": "The relevant literature appears to be sufficiently discussed.", "other_strengths_and_weaknesses": "Strengths\n\n(1) The extended TATE framework appears to be novel and well-motivated. The relaxation of the separating set to belong to just one of the datasets offers practical value, as it accounts for potential covariate shifts or missing data when generalizing to other sites.\n(2) Experimental results suggest that 2SDR outperforms other baselines on real datasets, particularly in the JTPA dataset experiment, where it improves estimates for sites different from the RCT data.\n\n\nWeaknesses\n\n(1) Assumption 3.2 seems to be a strong condition that may be difficult to satisfy in practice (see questions below).\n(2) A more detailed discussion is needed to convey the intuition behind why the 2SDR estimator performs better.", "other_comments_or_suggestions": "(1) A more detailed practical example for Settings 1 and 2 would be helpful.\n\n(2) Figure 1 is not very clear or readable unless the entire Section 2 is read. There is room for improving the figure by making some of the definitions more explicit.\n\n(3) Readability of Section 4.1.2 needs some improvement (maybe by explaining what is happening in each step or having a diagram).", "questions_for_authors": "(1) How realistic is it to assume the existence of a shadow variable Z that satisfies Assumption 3.2? Under what conditions can we expect this to hold?\n\n(2) Will results deteriorate smoothly w.r.t. the extent to which this Assumption 3.2 is satisfied?\n\n(3) Typically, under environmental shifts, there may also be shifts in Y|X shifts. It would be beneficial if the authors discussed situations where we expect only X-shifts and where both X- and Y-shifts might occur,  thereby clarifying where 2SDR can be effectively applied?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jwG3Z6hEBw", "reviewer": "Reviewer_Geaz", "summary": "This paper deals with generalizing treatment effects estimated from RCTs to different environments where there exists environmental shifts. Existing methods assume that covariates common to both source and target datasets contain the separating set, which is often violated in real-world.\n\nThe authors propose a Two-Stage Doubly Robust (2SDR) method to address this. The key idea is to relax the standard assumption. Instead of requiring the separating set to be present in the common covariates, they only require it to be present in at least one of the datasets. They then use shadow variables (covariates correlated with the missing ones but not directly influencing the environmental shift) to impute the missing covariates. \n\nThe authors also provide theoretical justification and evaluate the method on synthetic and real-world datasets.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer’s great efforts and insightful comments to improve our manuscript. In below, we address these concerns point by point.\n\n> **[Claims And Evidence 1]**\n\nFirst, we would like to clarify that **the method we propose for automatically selecting shadow variables is theoretically guaranteed, based on the testability of Assumption 3.2**:\n1. Assumption 3.2(1), i.e., $Z\\notindep X^m |\\widetilde{X^c}, S=1$, is testable because **all three variables are observable in the dataset with $S=1$**, where $S=1$ indicates that $X^m$ is observable in this dataset.\n2. Assumption 3.2(2), i.e., $Z\\indep S |X^m, \\widetilde{X^c}$, is testable for the following reason. Although $X^m$ is unobservable in the dataset with $S=0$, we can still conduct the test based on Theorem 4.1, which **does not require the values of $X^m$ in the dataset with $S=0$ since the only expression involving $X^m$ in the equation to be solved, i.e., $\\frac{S}{Q(X^m, \\widetilde{X^c})}$, is always zero when $S=0$**.\n\nThe shadow variables selected by the proposed method should pass the above hypothesis tests, ensuring that they satisfy Assumption 3.2.\n\nSecond, **the existence of shadow variables can also be assessed through hypothesis testing**. Moreover, as discussed in the manuscript, Assumption 3.2 is reasonable because not all variables in $X^c$ with predictive ability for $X^m$ are direct causes of $S$ in many real-world scenarios, such as the example of the AIDS study mentioned in the manuscript. Of course, in certain cases, shadow variables may indeed be weak or even absent. **We have followed the reviewer's suggestion and conducted experiments in such cases**.\n\n> **[Claims And Evidence 2]**\n\nOur method, as most DR methods do, requires that at least one of the two models in both stages, either the regression model or the selection score model, be correctly specified to ensure the consistency of the imputation and ATE estimation. **However, such requirement for correct model specification is very common in related work [1, 2], and we can leverage techniques such as neural networks to learn the correct model specification as accurately as possible.** Of course, we value the reviewer's suggestion and **have also conducted experiments to assess the impact of incorrect model specification due to violations of Assumption 3.2**.\n\n> **[Claims And Evidence 3]**\n\nThank you for your insightful suggestion. Our method does not rely on specific CIT methods; any CIT method can be used for our hypothesis testing of Assumption 3.2(1). The choice of the method should be determined based on the specific application context. Due to the character limit in the rebuttal phase, we regret that we cannot provide a detailed discussion of the various CIT methods here. **We promise to include the relevant discussion in the appendix of the revised manuscript**. \n\n> **[Methods And Evaluation Criteria]**\n\nThank you for the suggestion. **We have conducted experiments in cases where Assumption 3.2 is violated.** Specifically, we conducted experiments on the synthetic dataset by \n1. changing the coefficient of Z on S from 0 to \\{0.1, 0.3\\} to introduce weak dependence;\n2. reducing the correlation coefficient between Z and $X^m$ from 0.8 to \\{0.3, 0.1\\}, so that Z only has a weak predictive ability for $X^m$.\n\n|Extent|Case 1|Case 2| \n|:-:|:-:|:-:|\n|High|0.355$\\pm$0.319|0.383$\\pm$0.280|\n|Low|0.311$\\pm$0.293|0.341$\\pm$0.281| \n|None|0.268$\\pm$0.212|0.268$\\pm$0.212| \n\nThe results demonstrates that **while the performance of 2SDR does decline when Assumption 3.2 is violated, the decline is steady as the extent to which Assumption 3.2 is violated increases**.\n\n> **[Other Comments Or Suggestions]**\n\nThank you for your valuable suggestions. \n1. **\"Covariates affecting treatment effect heterogeneity\" are not equivalent to confounders**: Suppose there are three covariates, A, B, and C, and the generation process of Y is $Y=A\\cdot T+B+C$. In this case, A is the only variable affecting treatment effect heterogeneity, as the value of $Y(1)-Y(0)$ changes with variations in A. We will improve the related definitions in the figure and its caption.\n2. **\"Variables affect environmental shifts\" refers to the variables in the causal graph that point to R (the causes of R)**. We will revise this statement to make it clearer.\n\n***\n**We hope the above discussion will fully address your concerns about our work, and we would really appreciate it if you could be generous in raising your score. Thank you!**\n> **References**\n\n[1] Miao, W., Liu, L., Li, Y., Tchetgen Tchetgen, E. J., & Geng, Z. (2024). Identification and semiparametric efficiency theory of nonignorable missing data with a shadow variable. ACM/JMS Journal of Data Science, 1(2), 1-23.\n\n[2] Colnet, B., Mayer, I., Chen, G., Dieng, A., Li, R., Varoquaux, G., ... & Yang, S. (2024). Causal inference methods for combining randomized trials and observational studies: a review. Statistical science, 39(1), 165-191.", "claims_and_evidence": "1. The theory assumes that the chosen shadow variables are sufficient to fully determine the distribution of the missing covariates. There's no guarantee that such a set of shadow variables exists or that the proposed selection procedure will find them. If the shadow variables are only weakly predictive of the missing covariates, the imputation will be poor, leading to bias.\n\n2. The claim of unbiasedness also depends on the correct specification of the imputation models or the odds ratio function. The paper acknowledges the doubly robust property, but this only protects against misspecification of one model in each pair. It does not guarantee unbiasedness if both models in a pair are misspecified, or if the shadow variable assumption is violated. The empirical evaluation doesn't systematically investigate the impact of model misspecification.\n\n3. The selection procedure depends on the reliability of the conditional independence tests (RCIT). I would recommend authors discuss these tests' own assumption and limitations because these tests are not perfect, especially in high-dimensional settings with limited data.", "methods_and_evaluation_criteria": "While I know it may be beyond the scope of this rebuttal, simulating scenarios where the shadow variable assumption is partially violated (e.g., by introducing a weak dependence between Z and S) and quantify the resulting bias in the TATE estimates would significantly strengthen the work.", "theoretical_claims": "I quickly went over the proof of Theorem 3.3, I did not notice significant flaw in the proof.", "experimental_designs_or_analyses": "The experiments in general demonstrate the effectiveness of the proposed method.", "relation_to_broader_scientific_literature": "Estimating causal effects from multiple environments is quite important in a lot of the applications, in particular, experiments with multiple sites.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "Figure 1(a) and the related definitions in its caption need improvement. I am confused what type of variables are \"covariates affecting treatment effect heterogeneity\". Are these variables confounders? Why author state $X_2$ is a covariate affecting neither treatment effect heterogeneity nor environmental shifts in line 68-69?\n\nFurthermore, the wording \"variables affect environmental shifts\" was misleading. Seems like the covariates exhibit distributional differences, but they don't cause the underlying environmental shift.", "questions_for_authors": "Refer to all the weaknesses in the \"Claims and Evidence\" and \"Methods and Evaluation Criteria\" sections above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QTKUpoQ59v", "reviewer": "Reviewer_ogMS", "summary": "This paper studies the identification and estimation problem in generalizing treatment effects from an RCT to an observational dataset. Instead of assuming all relevant separate set is observed, it is assumed that part of them are only observed in one dataset. The identification is made possible by introducing the notion of shadow variables, which is a subset of commonly observed covariates and is conditionally independent of observability given other covariates. The authors then develop the identification theory for the target treatment effect. In addition, they propose a two-stage doubly robust estimator that involves both the selection of shadow variables and doubly robust estimation of the target treatment effect. The efficacy of the proposed methods is demonstrated by extensive numerical experiments.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer’s great efforts and insightful comments to improve our manuscript. In below, we address these concerns point by point.\n\n> **[Q1]**\n\nRegarding the testability of Assumption 3.2, we would like to clarify the following: \n1. Assumption 3.2(1), i.e., $Z\\notindep X^m |\\widetilde{X^c}, S=1$, is testable because **all three variables are observable in the dataset with $S=1$**, where $S=1$ indicates that $X^m$ is observable in this dataset.\n2. Assumption 3.2(2), i.e., $Z\\indep S |X^m, \\widetilde{X^c}$, is testable for the following reason. Although $X^m$ is unobservable in the dataset with $S=0$, we can still conduct the test based on Theorem 4.1, which **does not require the values of $X^m$ in the dataset with $S=0$ since the only expression involving $X^m$ in the equation to be solved, i.e., $\\frac{S}{Q(X^m, \\widetilde{X^c})}$, is always zero when $S=0$**.\n\n> **[Q2]**\n\nThank you for pointing out the issue caused by our accidental omission of the factor $\\frac{P(S=1|X^m,X^c)}{P(S=1|X^c)}$ in the corresponding lines. However, we would like to clarify that this does not affect our final conclusion (the result on lines 674-675). **The corrected proof is as follows:**\n\n$\nE\\left[\\frac{P(S=0|X^m,X^c)}{P(S=1 |X^m,X^c)}\\cdot\\frac{P(S=1|X^m=0,X^c)}{P(S=0|X^m=0,X^c)}\\middle|X^c,S=1\\right]$\n$=\\sum_{x^m}\\frac{P(S=0|X^m=x^m,X^c)}{P(S=1|X^m=x^m,X^c)} \\cdot\\frac{P(S=1|X^m=0,X^c)}{P(S=0|X^m=0,X^c)}\\cdot P(X^m=x^m|X^c,S=1)$\n$=\\sum_{x^m}\\frac{P(S=1|X^m=0,X^c)}{P(S=0|X^m=0,X^c)}\\cdot\\frac{P(S=0|X^m=x^m,X^c)}{P(S=1|X^m=x^m,X^c)}\\cdot\\frac{P(S=1|X^m=x^m,X^c)}{P(S=1|X^c)}\\cdot P(X^m=x^m|X^c)$\n$=\\frac{P(S=1|X^m=0,X^c)}{P(S=0|X^m=0,X^c)\\cdot P(S=1|X^c)}\\cdot\\sum_{x^m}P(S=0|X^m=x^m,X^c)\\cdot P(X^m=x^m|X^c)\\\\\n=\\frac{P(S=1|X^m=0,X^c)}{P(S=0|X^m=0,X^c)\\cdot P(S=1|X^c)}\\cdot\\sum_{x^m}P(S=0,X^m=x^m|X^c)$\n$=\\frac{P(S=1|X^m=0,X^c)\\cdot P(S=0|X^c)}{P(S=0|X^m=0,X^c)\\cdot P(S=1|X^c)}\n$\n\nThank you again for your insightful feedback. We will address this issue in the revised version of the manuscript.\n\n> **[Q3]**\n\n**The identification of $OR(X^m,X^c)$ is guaranteed by Theorem 1 in [1], as referenced in Appendix D.** The core idea relies on the completeness condition being satisfied. The detailed proof can be found in the Appendix (Proof of Theorem 1 on Pages 14-15) of [1]. Due to the character limit in the rebuttal stage, we regret that we are unable to provide the full proof here. We kindly ask that the reviewer refer to the original paper for further details. We will include this proof in Appendix D of the revised manuscript. Thank you for the suggestion. \n\n> **[Q4]**\n\nThank you for pointing out a typo in our manuscript. Indeed, **it should be $T\\indep X^m|R=1$**, where $R=1$ is equivalent to $S=0$ in Setting 2. We will correct this typo in the revised version of the manuscript.\n\n> **[Q5]**\n\nAs cited in the manuscript, Theorem 4.1 is derived from Theorem 2.3 in [2], and **the detailed proof can be found in Appendix A.3 (Page 12) of [2]**. Due to the character limit in the rebuttal stage, we regret that we are unable to provide the full proof here. We kindly ask that the reviewer refer to the original paper for further details. We will include this proof in the appendix of the revised manuscript. Thank you for your valuable suggestion.\n\n> **[Q6]**\n\n**We employed mean value imputation to reduce the complexity of imputation and improve its practicality and efficiency.** Performing distributional-estimation-based imputation would be perfect, but it requires additional estimation of quantities such as variance, higher-order moments, etc., or the use of generative methods, all of which would increase complexity. Mean value imputation is a commonly used technique to reduce complexity [3], and our experimental results demonstrate that it performs well. Of course, Theorem 3.3 supports distributional-estimation-based imputation, and we will explore it in future work. Thank you for your insightful suggestion.\n\n> **[Q7]**\n\nDue to the character limit in the rebuttal stage, we are unable to include the full proof here. **We promise to provide the consistency proofs for both doubly robust imputation and doubly robust ATE estimation in the appendix of the revised version.** Thank you for your insightful suggestion.\n\n***\n\n**We hope the above discussion will fully address your concerns about our work, and we would really appreciate it if you could be generous in raising your score. Thank you!**\n\n> **References**\n\n[1] Miao, W., Liu, L., Li, Y., Tchetgen Tchetgen, E. J., & Geng, Z. (2024). Identification and semiparametric efficiency theory of nonignorable missing data with a shadow variable. ACM/JMS Journal of Data Science, 1(2), 1-23.\n\n[2] d’Haultfoeuille, X. (2010). A new instrumental method for dealing with endogenous selection. Journal of Econometrics, 154(1), 1-15.\n\n[3] Lin, W. C., & Tsai, C. F. (2020). Missing value imputation: a review and analysis of the literature (2006–2017). Artificial Intelligence Review, 53, 1487-1509.", "claims_and_evidence": "The experiments are solid, but I have several confusion regarding identification theory and estimation procedures. It is not fully clear to me why the theory goes through under the given assumptions -- please see my questions below.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "I checked the correctness but cannot arrive at a clear conclusion, and clarification is needed.", "experimental_designs_or_analyses": "Yes, the experiments seem solid.", "relation_to_broader_scientific_literature": "Generalizing treatment effects to target populations is an important topic, and this paper proposes methods that relax existing conditions, which contributes to this literature and may inspire future developments.", "essential_references_not_discussed": "No to my knowledge.", "other_strengths_and_weaknesses": "If all results are correct, this paper would be interesting. Strengths include: rich results, new identification conditions for generalization, new procedures. But given the confusions I list in the \"questions\" section, I would urge the authors to clarify them for me to give a fair judgement.", "other_comments_or_suggestions": "Please see the Questions section below. Given the confusions my current rating is towards reject, but would love to adjust if the authors can address the confusions in the rebuttal.", "questions_for_authors": "1. Why is Assumption 3.2 testable using observational data, given that $X^m$ is only observed in one dataset? \n2. On page 13 (the proof of Lemma 3.4), why Assumptions 2.5 and 3.2 can lead to the first equation (line 664-665)? It seems to use the equivalence between the conditional distributions of $X^m \\mid X^c, S=1$ and $X^m \\mid X^c$, but I'm not sure this can be implied by the two assumptions, and some clarifications are needed.\n3. In Lemma 3.5, what do you mean by \"identified\"? Do you mean these quantities allow you to compute OR($X^m=x^m, X^c=x^c)$ for every value of $x^m, x^c$? If so, only knowing the conditional expectation $\\mathbb{E}[\\tilde{OR}(X^m,X^c) \\mid X^c,S=1]$ wouldn't allow you to get back to a function of $x^m$ and $x^c$. Please clarify so that I can make a more clear judgement. \n4. In step 2 in the identification, why \"Under Assumption 2.3, we have $T\\indep X^m\\mid S=1$\"? If I understand correctly, $S=1$ represents the observational dataset in setting 2, so this cannot be true? I think this should be conditional on $S=0$. \n5. Can you provide a proof for Theorem 4?\n6. Why do you use mean value imputation for $X^m$? Why is this enough for estimating $\\tau$ given that Theorem 3.3 uses the density of $X$ given $R=0$? \n7. Could you develop consistency and inference guarantees for the resulting estimator?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces a two-stage method for generalizing treatment effects from RCTs to target populations facing environmental shifts. Unlike common solutions, the proposed procedure relaxes the assumption that the separating set of covariates is fully observable in both the RCT and target data. Instead, it only requires the separating set to be present in at least one of the two data groups and utilizes shadow variables to impute missing covariates in a two-stage process. 

Reviewers generally acknowledged the novelty and importance of the problem and found the experimental results promising, particularly on real-world datasets like JTPA. A key concern was the realism regarding the existence and properties of shadow variables, which was partially addressed by the author response. More detailed and practical examples will significantly strengthen the paper further.
Decision: Accept (poster)