Title: A Unified Framework for the Transportability of Population-Level Causal Measures
Venue: NeurIPS 2025
Accepted: yes
Keywords: Causal inference

Generalisation

Causal measure
Abstract: Generalization methods offer a powerful solution to one of the key drawbacks of randomized controlled trials (RCTs): their limited representativeness. By enabling the transport of treatment effect estimates to target populations subject to distributional shifts, these methods are increasingly recognized as the future of meta-analysis, the current gold standard in evidence-based medicine.
Yet most existing approaches focus on the risk difference, overlooking the diverse range of causal measures routinely reported in clinical research. Reporting multiple effect measures—both absolute (e.g., risk difference, number needed to treat) and relative (e.g., risk ratio, odds ratio)—is essential to ensure clinical relevance, policy utility, and interpretability across contexts.
To address this gap, we propose a unified framework for transporting a broad class of first-moment population causal effect measures under covariate shift. We provide identification results under two conditional exchangeability assumptions, derive both classical and semiparametric estimators, and evaluate their performance through theoretical analysis, simulations, and real-world applications. Our analysis shows the specificity of different causal measures and thus the interest of studying them all: for instance, two common approaches (one-step, estimating equation) lead to similar estimators for the risk difference but to two distinct estimators for the odds ratio.


Peer reviews:
comments: SUMMARY:
This paper introduces a unified framework for transporting a wide range of population causal effect measures from randomized controlled trials (RCTs) to target populations under covariate shift. It provides identification results under two key assumptions (exchangeability in mean and in effect measure), and derives classical, semiparametric, and doubly-robust estimators, revealing that estimators often differ across causal measures due to nonlinearity.

STRENGTHS AND WEAKNESSES:
**Strengths**:
- The paper generalizes causal transportability to a wide class of first-moment estimands, which is a notable expansion beyond existing methods
- The theoretical claims are convincingly supported 
- The paper is well written and easy to follow



**Weaknesses**:

- All the simulations use large sample sizes ($ n=50,000$), masking potential pathologies under finite or moderate samples, such as weight instability, overfitting of nuisance models, or violation of positivity. Arguably, most trials have significantly smaller sample sizes and therefore it would be valuable to evaluate the proposed estimators in a small sample setting

QUESTIONS:
Do you expect significant performance differences between your estimators in small or moderate sample sizes, especially when nuisance functions are estimated with limited data?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-22

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to generalize transportability theory from average treatment effects to a broader class of first-moment population-level causal measures. The authors derive identification results under two regimes—exchangeability in mean and in effect measure—and propose several estimators (classical, regression-based, semiparametric). They show empirically that estimator performance diverges sharply across nonlinear estimands and assumptions.\n\nSTRENGTHS AND WEAKNESSES:\n*Strengths*\n- Motivation: The paper tackles an under-addressed problem (generalising nonlinear causal estimands across populations) motivated by several real-world application, far beyond medicine. \n- Method: The paper is methodologically rich, providing numerous instantiations of classical and doubly robust estimators. \n\n*Weaknesses*:\n- The introduction is difficult to follow. Several concepts are not properly introduced, e.g.:\n   - l27 the risk difference is introduced with no definition or assumptions on the setting (binary outcome?).\n   - l54-55 'collapsible and not collapsible measures' are not defined.\n\n- Imprecisions/Error (?):\n  - Risk ratio assumes continuous outcome for interpretation, but I cannot find where/if assumed in Section 2. Similarly Odds ratio etc.\n  - l110 m and n are DEPENDENT binomials (use multimodal distribution instead).\n  - Finishing the intro was still unclear if some transportability assumptions are required. I find them crucial and I would suggest to better stress them in the introduction too.\n- Experiments\n  - Synthetic experiments are under-defined and no code is provided. How many covariates are considered (d)? What are the source values in Figure 1 caption?\n  - I find difficult to interpret the objective of the experiments. Additionally I find unclear how to evaluate the real world experiment results in Figure 3.\n\nOverall I appreciate the attempt of this work in formalising transportability of population-level causal measures but I find the writing still immature, containing imprecisions with unclear story conclusions.\n\nQUESTIONS:\n- What if the observed covariates are not fully overlapping among training and target experiment?  What if some covariates are changing distribution on target but they are not directly observed? (see PPCI formulation [1] for reference).\n- Are the assumptions and estimators scaling to an higher number of observations? The real world experiments only consider 6 covariates.\n\n[1] Causal Lifting of Neural Representations: Zero-Shot Generalization for Causal Inferences, arxiv, Cadei et al., 2025\n\nLIMITATIONS:\nSee questions.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 1, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the generalizability of several nonlinear causal estimands under distribution shift, proposing a family of estimators under mean exchangeability and effect measure exchangeability conditions. When mean exchangeability holds, the authors propose weighted, regression, and doubly robust strategies for transporting an estimand that is a known mapping of the mean of two potential outcomes. In addition, the authors discuss generalization strategies for such estimands based on the inverse mapping. Theoretical analysis shows the inferential guarantees of the proposed estimators, and the performance of them is demonstrated via simulations and real data analysis.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n1. This paper studies an important problem of generalizing causal estimands.\n2. The results are solid and rich, covering identification, estimation, and inference. \n3. Numerical results show promising performance.\n\nWeakness:\n\n1. The contributions need to be more clearly stated by comparing to relevant literature (see Questions).\n2. The real data analysis has vague message.\n\nQUESTIONS:\n1. How would you interpret the discrepancy in the messages in the real experiments? Since the truth is not available, it would be helpful to investigate why there is a discrepancy to decide whether it makes sense.\n\n2. I think connections to existing results should be more explicitly discussed to more clearly position the contributions. In particular, [1] discusses quite a few identification strategies for nonlinear causal estimands. How does this work compare to them? If all identification strategies have been studied and this work just focuses on estimators, then the estimators would look not very novel/surprising given the existing results, and this work might be better suited to a (bio)stats journal with more in-depth simulations and real data analysis to show the performance of various estimators in diverse conditions. \n\n\n[1] Colnet, B., J. Josse, G. Varoquaux, and E. Scornet (2023). Risk ratio, odds ratio, risk difference... which causal measure is easier to generalize? arXiv preprint arXiv:2303.16008.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of causal meta analysis, and proposes a unified framework for transporting a board class of first moment causal effect measures. It aims to generalize treatment effect estimates from RCTs to target populations under covariate shifts. The authors extend existing methods for generalizing causal effects by deriving identification results under conditional exchangeability assumptions. The framework incorporates both classical estimators and semiparametric efficiency theory to provide doubly-robust estimators for various causal measures. The method is evaluated through synthetic simulations and real-world applications, showcasing its utility in generalizing both absolute and relative causal effects to broader populations.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n- In general I think formulating meta analysis under a causal framework is an important direction. One way of approaching this is to reframe the problem as generalizing treatment effects from controlled settings to some target populations that may differ in their distribution of covariates. Then, the paper's focus on a diverse set of causal measures beyond the risk difference, which is a major strength. \n- The work is quite thorough. The authors systematically derived identification formulas and a complete suite of estimators (weighting, G-formula, one-step, EE) under two clearly articulated transportability assumptions. This is very organized and I like it a lot. \n\nWeakness:\nThere are a few minor weaknesses especially on discussions, evaluation and potential mistakes in proofs.\n\n-  The paper presents two parallel frameworks based on exchangeability in mean vs effect measure. The latter is weaker but imposes the strong practical requirement of having access to control outcomes for the target. The paper would be strengthened by a more detailed discussion of applied scenarios where this condition is met and more explicit guidance on how a practitioner should choose between these two sets of assumptions and corresponding estimators\n\n-  I am not sure if lemma 2 proof is correct. Its correctness depends on the component matrix\n$$\nE_{T}[VV^{\\top}] - E_{T}[VV^{\\top}] E_{T}[\\sigma(X,\\beta)VV^{\\top}])^{-1}E_{T}[VV^{\\top}]\n$$\nbeing PSD. As the authors suggests, this matrix has the form $A-AB^{-1}A$. But I think the authors might have missed an additional condition for $A-AB^{-1}A$ to be PSD which is $B \\succeq A$ should hold.\n\nHowever, given the definitions $A=E_{T}[VV^{\\top}]$ and $B=E_{T}[\\sigma(X,\\beta)VV^{\\top}]$, and the fact that $\\sigma(X, \\beta) \\in (0,1)$ (if I understand correctly, the authors can elaborate), it would seem the reverse inequality, $A \\succeq B$, holds true instead.\n\n- on eval:  In the real-world application (sec 5.2), the authors conclude that the treatment appears to have \"little to no impact on the target population\". However, the confidence intervals presented in Figure 3 are quite wide. The claim of \"little to no impact\" seems stronger than what can be concluded from the point estimates and their associated uncertainty.\n\n\n\n\n\n**Please note: I do find I enjoy this paper, I am giving borderline reject simply due to those concerns. If the authors can address my concerns, especially the one on lemma 2, I will consider increasing the score.**\n\nQUESTIONS:\nCould the authors please clarify the justification for my questions on lemma2 proof? As proposition 2 seems crucially assumes correctness of lemma 2.\n\nLIMITATIONS:\nSee above.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-06"}