Title: Data Fusion for Partial Identification of Causal Effects
Venue: NeurIPS 2025
Accepted: yes
Keywords: causal inference

partial identification

data fusion

double machine learning

double robustness

causality
Abstract: Data fusion techniques integrate information from heterogeneous data sources to improve learning, generalization, and decision-making across data sciences. In causal inference, these methods leverage rich observational data to improve causal effect estimation, while maintaining the trustworthiness of randomized controlled trials. Existing approaches often relax the strong "no unobserved confounding" assumption by instead assuming exchangeability of counterfactual outcomes across data sources. However, when both assumptions simultaneously fail—a common scenario in practice—current methods cannot identify or estimate causal effects. We address this limitation by proposing a novel partial identification framework that enables researchers to answer key questions such as: *Is the causal effect positive/negative?* and *How severe must assumption violations be to overturn this conclusion?* Our approach introduces interpretable sensitivity parameters that quantify assumption violations and derives corresponding causal effect bounds. We develop doubly robust estimators for these bounds and operationalize breakdown frontier analysis to understand how causal conclusions change as assumption violations increase. We apply our framework to the Project STAR study, which investigates the effect of classroom size on students’ third-grade standardized test performance. Our analysis reveals that the Project STAR results are robust to simultaneous violations of key assumptions, both on average and across various subgroups of interest. This strengthens confidence in the study's conclusions despite potential unmeasured biases in the data.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors consider a setting where we have collected both randomized and observational data related to some target population for which we would like to estimate a treatment effect. Here, the observational data may have unmeasured confounding and there is also a question whether transportability holds between the populations underlying both the two datasets and the target population. The authors propose in this case an approach to estimate bounds for the ATE and CATE when two sensitivity parameters controls the violations of the no unmeasured confounding assumption and the transportability assumption. They further propose an efficient estimator for the bounds of the ATE. They evaluate their estimator on a set of simulations and demonstrate its use on a real-world dataset.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- The paper is overall well-written. I found most things easy to understand, especially Section 1 to 4.\n- I find the approach well-motivated, especially this setting (partial identification in data fusion problems) is definitely overlooked in my opinion and it is nice to see a method like this being proposed. The underlying idea to come up with the bound is relatively simple which I also appreciate.\n- I can not judge on the correctness of the proof for deriving the efficient estimator, but assuming that they are correct, I think these are all important properties to have for the estimator and it is nice to see them being derived theoretically.\n\n### Weaknesses\n- The simulations are relatively simple, focussing only on how the bounds change across various DGPs. It would have also been interesting better understand some of the statistical properties of the proposed estimator such as sample size efficiency, the robustness of the asymptotic properties in Theorem 2, or its sensitivity to choosing the scale in the Boltzmann estimator.  \n- There is one thing in the initial setup of the problem which I don't understand, which I think makes it hard for me to fully appreciate in what kind of study designs this method is applicable. In particular, if I interpret things correctly, we have three populations: the population underlying the RCT ($S=1$), the one underlying the observational study $S=0$, and then the target population.  On one hand, the authors say on lines 44-46 that we can get the ATE if transportability holds **or** if there is no unmeasured confounding in the observational study. This makes me think that the population underlying the observational cohort and the target population are the same, because then we would not need transportability if the observational study is unconfounded. But on the other hand, when reading section 2, I interpret it as being that the assumption A5 (which to me is the transportability condition) is needed together jointly with assumption 4 (NUC) if we would identify and estimate the ATE in the target population using the observational data. There seems to be some sort of contradiction here, I would like to hear the authors' thought on this, and in particular if they could explain, for instance using the STAR dataset as an example, how the experimental/observational populations are related to the target population.\n\nOverall, I like this paper. The reason for giving a borderline accept are some of my confusion on the second bullet point under Weaknesses. I'd be happy to re-evaluate my score if the authors are willing to address this point.\n\nQUESTIONS:\n- I would like to ask if they can reply to my comments on the study design from my second bullet point under \"Weaknesses\"?\n- Given the claims on line 242-243, can you outline the similarities/differences in applying the same steps as in section 5 to deriving efficient estimators for the CATE? Somehow I would expect that estimating the CATE compared to the ATE should be significantly more difficult, especially if we want to obtain e.g. confidence intervals as you get in Theorem 2.\n- I might have missed this point in the paper, but how should we go about selecting the parameter $\\alpha$ in practice for the Boltzmann operator and have you looked into how sensitive this choice is for your estimator's performance?\n\nLIMITATIONS:\nThe authors seem to address all important limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new type of sensitivity analysis for causal data fusion to infer bounds on the CATE / ATE when both an unconfounded trial (potentially misrepresenting the population of interest) and a confounded observational dataset are available. Sensitivity models are imposed, bounds are derived, and estimators based on semiparametric efficiency theory are proposed. The authors evaluate their method using both synthetic and real-world data.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- Rigorous work with theoretically valid bounds that incorporates semiparametric estimation for increased statistical efficiency\n- Data fusion in causal inference is an interesting problem with various potential applications\n\nWeaknesses\n- Consistency assumptions are missing in A1-5\n- The target quantity of interest seems to be the CATE or ATE averaged over both datasets. That does not seem to be the quantity one would usually be interested in, as it may depend heavily on the selection probability $P(S = 1 | X=x)$. Indeed, the CATE in the RCT and the observational data are not the same (if they were, we would achieve point identification). I suspect that in practice, one would be more interested in the quantity $E[Y(1) - Y(0) | X=x, S=0] (i.e., the CATE on the observational dataset) as the observational dataset generalizes better to the population of interest.\n- The related work could be expanded to include recent works on data fusion and partial identification, e.g., https://arxiv.org/abs/2406.02873, https://arxiv.org/abs/2406.02464, https://arxiv.org/abs/2412.11511, https://arxiv.org/abs/2405.07186\n- The novelty as compared to related work is not entirely clear. For example, are the proposed sensitivity models novel or adapted from other streams of sensitivity analysis (e.g., on unobserved confounding)? Are the bounds novel or adapted from existing literature? There is a large literature on both sensitivity models and bounds for unobserved confounding. Can the authors justify their choice of sensitivity model as compared to using an established model such as the marginal sensitivity model (MSM)? \n- The authors claim that their sensitivity parameters are interpretable, but it is not entirely clear to me how one would interpret a specific choice of sensitivity parameter in practice. Can the authors provide an example where sensitivity parameters can be chosen by domain knowledge for their sensitivity model?\n\nQUESTIONS:\nCan the authors provide an example where sensitivity parameters can be chosen by domain knowledge for their sensitivity model?\nCan the authors justify their choice of sensitivity model as compared to using an established model such as the marginal sensitivity model (MSM)? \nCan the authors compare their modeling choices and results to related works and elaborate on what is novel or similar/ adapted from existing literature?\nCan the authors extend their results to CATE/ ATE on the observational dataset?\n\nLIMITATIONS:\nLimitations are discussed in Sec. 7. I would suggest adding limitations regarding the interpretability of the sensitivity parameters, which is a common problem in sensitivity analysis.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper introduces an approach to sensitivity analysis in the context of estimating effects by combining RCT and observational studies. In essence, there are two main sources of non-identifiability here: unobserved confounding in the observational data, and non-transferability between the experimental and observational settings. The paper introduces a model where each of these can be characterized by a single value, and then applies this to visualize how sensitive study conclusions are to these parameters.

STRENGTHS AND WEAKNESSES:
The problem seems well-motivated, the paper is clear, and the visualization apparently easy to interpret. I thought the experimental application was also reasonably well executed.

The main weaknesses in my view are both the relevance of the setting, which seems rather specialized, and the robustness of the sensitivity model.

QUESTIONS:
See strengths and weaknesses. My main hesitation are about the significance of the work, and the degree to which conclusions from the sensitivity analysis rely on an unassessable sensitivity model.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper contributes to the growing literature on data fusion /\ntransportability / generalizability, in which data from a randomized\ncontrolled trial (RCT; \"internally valid\") and observational study\n(OS; \"externally valid\") are to be combined to generalize to a target\npopulation (typically the OS population). The proposed approach does\nnot require that all confounders are observed in the OS (with\nviolation of the usual assumption quantified through a sensitivity\nparameter $\\rho$), and importantly relaxes the very common assumption\nthat the outcome has the same distribution in the RCT and OS,\nconditional on measured covariates and treatment (with violation\nquantified through the sensitivity parameter $\\gamma$). The\nmethodology is based on nonparametric partial identification /\nsensitivity analysis, where under relaxed assumptions, bounds on\ntarget causal quantities are characterized and identified in terms of\nthe observed data distribution, and subsequently estimated. The\nresulting bound functionals are non-smooth, and the proposed\nestimators are *doubly robust* estimators of **smooth approximations**\nto these challenging estimands. Finally, the authors demonstrate their\napproach in simulations and in a real study of classroom size on\nlearning outcomes.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The paper is well written and clear.\n- This work tackles a very important problem, and the goal to relax\n  both the unconfoundedness assumption and the study exchangeability\n  assumption is commendable.\n- The general use of nonparametric partial identification and\n  influence function-based estimation is compelling and convincing.\n- The breakdown frontier approach is interesting, and well\n  illustrated.\n- Empirical illustrations (simulations and the Project STAR\n  application) demonstrate the approach nicely.\n\n**Weaknesses** (see **Questions** below for full detail on each point)\n- The incompatibility issue (Remark 1) resulting from the pair of\n  sensitivity models is quite confusing as currently presented, and\n  possibly improved with a different sensitivity model\n  parametrization.\n- Recent relevant literature on non-smooth functional estimation in\n  related causal problems has not be acknowledged or cited. These\n  works have many insights that could improve this paper.\n- The statement of the main estimation result (Theorem 2) is\n  misleading and, relatedly, its proof is seemingly incomplete.\n- Certain important conceptual and theoretical issues (target\n  estimand, sharpness/tightness) are inadequately or imprecisely\n  discussed.\n\nQUESTIONS:\n- **Sensitivity Model & Incompatibility**. Can the authors expand on\n  why they used a ratio-based sensitivity model, as opposed to simply\n  bounding the differences in means? For instance, one might have\n  chosen $\\rho = \\sup_{\\mathbf{x},\n  t}|\\mathbb{E}[Y(t) \\mid \\mathbf{X} = \\mathbf{x},\n  S = 0, T = 1 - t] - \\mu(\\mathbf{x}, 0, t)|$. This would result\n  in simpler bounds, but perhaps the authors might argue that this\n  model is less interpretable. More importantly, it is unclear why the\n  chosen parametrization of sensitivity to study exchangeability\n  ($\\gamma$) is the appropriate choice. Note that the numerator for\n  this quantity, $\\mathbb{E}[Y(t) \\mid \\mathbf{X} =\n  \\mathbf{x}, S = 0]$, is partially identified. It seems to me rather\n  more natural to only parametrize deviations from the unidentified\n  component, e.g., $\\gamma = \\sup_{\\mathbf{x}, t}|1 -\n  \\frac{\\mathbb{E}[Y(t) \\mid \\mathbf{X} = \\mathbf{x}, S = 0, T = 1 -\n  t]}{\\mathbb{E}[Y(t) \\mid \\mathbf{X} = \\mathbf{x}, S = 1, T = 1 -\n  t]}|$, where the denominator quantity is identified via\n  $\\mu(\\mathbf{x}, 1, t)$ under the Assumptions A1-A3. This would\n  result in a modified bounding scheme, i.e., $v(\\mathbf{x}, t,\n  \\gamma) = e_t(\\mathbf{x}, 0)\\mu(\\mathbf{x}, 0, t) + e_{1 -\n  t}(\\mathbf{x}, 0)(1 + \\gamma)\\mu(\\mathbf{x}, 1, t)$. Moreover, while\n  I am not 100% certain, this parametrization appears more\n  compatible with $\\rho$ and $w(\\mathbf{x}, t, \\rho)$, whereby the\n  incompatibility issues discussed in Remark 1 and Appendix D could \n  be simplified. Any clarity on these issues that the authors can provide I'm\n  confident would be appreciated by readers.\n- **Sharpness & Bounded Outcomes**. Are the bounds sharp / tight?\n  Before the statement of Lemma 1, it is claimed that the bounds are\n  \"tight\". However, it seems that only validity of the bounds is\n  proven. In fact, it seems to me that the bounds can be sharpened\n  under the assumptions of the paper. Concretely, suppose $b, B > 0$\n  is such that $Y \\in [b, B]$---the authors assume that \"the outcome\n  space is _bounded_ and _positive_\". Shouldn't we have $v(\\mathbf{x},\n  t, \\gamma) = \\min[(1 + \\gamma)\\mu(\\mathbf{x}, 1, t), B]$, and\n  $w(\\mathbf{x}, t, \\rho) = e_t(\\mathbf{x}, 0) \\mu(\\mathbf{x}, 0, t) +\n  e_{1 - t}\\min{[(1 + \\rho)\\mu(\\mathbf{x}, 0, t), B]}$? Analogous\n  lower bound adjustments could be made on the basis of $b > 0$ (there\n  are potential issues with the sensitivity model if the outcome could\n  take values equal to or very near zero). It would greatly stregthen\n  the result if sharpness could be proven. Regardless, a more precise,\n  careful discussion of sharpness / tightness will help clarify the\n  contribution.\n- **Estimation Considerations**. The functionals that identify the\n  bounds are challenging, non-smooth (i.e., not pathwise\n  differentiable) parameters. The authors choose an interesting smooth\n  approximation-based approach, coupled with influence function-based\n  estimators. While appropriate, I have some concerns and questions.\n  1) **Existing literature**. Perhaps most importantly, examples\n     belonging to the class of statistical problems studied here have\n     been investigated in recent years, and important works have\n     seemingly not been acknowledged---see *References* below. For\n     example, Luedtke & van der Laan (2016) studied the functional\n     $\\mathbb{E}(\\max{[\\tau(\\mathbf{X}), 0]})$, for\n     which similar issues arise. They discuss necessary and sufficient\n     conditions for pathwise differentiability, and propose procedures\n     for efficient confidence interval construction. Bonvini & Kennedy\n     (2022) also encounter similar functionals, and impose *margin\n     conditions* sufficient to yield pathwise differentiability\n     (obviating the need for smooth approximations). Levis et\n     al. (2025) and Ben-Michael (2025) also encounter such non-smooth\n     functionals, and carefully study different smooth\n     approximation-based estimators as well as direct estimators under\n     margin conditions. These or related works should likely be cited.\n  2) **Approximation error**. For any choice $\\alpha > 0$, the\n     Boltzmann operator-based smooth approximation bounds exhibit\n     error compared to the originally derived non-smooth bounds. While\n     Lemma 2 establishes that the approximate bounds converge to the\n     original bounds as $\\alpha \\to \\infty$, the result would be\n     greatly strengthened by *quantifying* the approximation error for\n     any finite $\\alpha > 0$---see also Levis et al. (2025) and\n     Ben-Michael (2025) for examples of such analyses. There is likely\n     a formal trade-off / tension between approximation error and the\n     impact of the tuning parameter $\\alpha$ on the second order\n     remainder for the smooth bounds.\n  3) **Nuisance error rates**. The statement of Theorem 2 is, in my\n     view, highly misleading, as asymptotic normality is seemingly\n     achieved without any conditions on the rates of convergence of\n     nuisance estimators $\\widehat{g}$, $\\widehat{e}$,\n     $\\widehat{\\mu}$. While mentioned in passing in the proof in the\n     appendix, these necessary rate conditions need to be explicitly\n     stated in the Theorem itself. Moreover, in their proof, the\n     authors mention an \"empirical process term\" and a \"second order\n     remainder term\", but do not actually state what these are. This\n     needs to be laid out in greater detail. Importantly, I do not see\n     a derivation of the second order bias anywhere in the\n     appendices. It is crucial that the form of this bias is\n     determined in order to conclude that the given nuisance error\n     rates are sufficient for this term to be asymptotically\n     negligible.\n- **Target Estimand**. Can the authors provide some context for why they\n  prefer the CATE, $\\mathbb{E}(Y(1) - Y(0) \\mid \\mathbf{X} =\n  \\mathbf{x})$, which mixes the RCT and OS populations, rather than\n  the effect in the OS population ($S = 0$, often deemed \"externally\n  valid\") on its own? This is not a huge issue, since the proposed\n  methodology implicitly bounds this effect, but a more in-depth\n  discussion of the chosen estimand would help.\n\n\n*References*\n1. Luedtke AR, van der Laan MJ. Statistical inference for the mean\n   outcome under a possibly non-unique optimal treatment\n   strategy. Annals of statistics. 2016 Mar 17;44(2):713.\n2. Bonvini M, Kennedy EH. Sensitivity analysis via the proportion of\n   unmeasured confounding. Journal of the American Statistical\n   Association. 2022 Sep 14;117(539):1540-50.\n3. Levis AW, Bonvini M, Zeng Z, Keele L, Kennedy\n   EH. Covariate-assisted bounds on causal effects with instrumental\n   variables. Journal of the Royal Statistical Society Series B:\n   Statistical Methodology. 2025 May 27:qkaf028.\n4. Ben-Michael E. Partial identification via conditional linear\n   programs: estimation and policy learning. arXiv preprint\n   arXiv:2506.12215. 2025 Jun 13.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}