Prediction-Powered Causal Inferences
Authors: Riccardo Cadei
Ilker Demirel
Piersilvio De Bartolomeis
Lukas Lindorfer
Sylvia Cremer
Cordelia Schmid
Francesco Locatello
Venue: NeurIPS 2025 poster ()
Keywords: Causal Representation Learning
Causal Inference
Domain Generalization
Primary area: probabilistic_methods

Abstract: In many scientific experiments, the data annotating cost constraints the pace for testing novel hypotheses. Yet, modern machine learning pipelines offer a promising solution—provided their predictions yield correct conclusions. We focus on Prediction-Powered Causal Inferences (PPCI), i.e., estimating the treatment effect in an unlabeled target experiment, relying on training data with the same outcome annotated but potentially different treatment or effect modifiers. We first show that conditional calibration guarantees valid PPCI at population level. Then, we introduce a sufficient representation constraint transferring validity across experiments,  which we propose to enforce in practice in Deconfounded Empirical Risk Minimization, our new model-agnostic training objective. We validate our method on synthetic and real-world scientific data, solving impossible problem instances for Empirical Risk Minimization even with standard invariance constraints. In particular, for the first time, we achieve valid causal inference on a scientific experiment with complex recording and no human annotations, fine-tuning a foundational model on our similar annotated experiment.

Prediction-Povered Causal Inferences
Riccardo Cadei', Ilker Demirel Piersilvio De Bartolomeis Lukas Lindorfer',
Sylvia Cremer', Cordelia Schmid", Francesco Locatello
Institute of Science and Technology Austria (ISTA) "Massachusetts Institute of Technology (MIT)
*Department of Computer Science, ETH Zurich
IIRIA, Ecole Normale Supéricure. CNRS. PSL Research University Equual contribution.
Abstract
In many scientific experiments, the data annotating cost constraints the pace for testing novel hypotheses. Yet, modern machine learning pipelines offer promising solution-providedd their predictions yield correct conclusions. We focus Prediction-Powwreed Causal Inferences (PPCT), i.e., estimating the treatment effect in an unlabeled target experiment, relying on training data with the same outcome annotated but potentially different treatment effect modifiers. We first show that conditional calibration guarantees valid PPCI at population level. Then, we introduce sufficient representation constraint transferring validity across experiments, which we propose to enforce in practice in Deconfounded Empirical Risk Minimization, our new -agess-ggossicc training objective. We validate our method on synthetic and real-world scientific data, solving impossible problem instances for Empirical Risk Minimization even with standard invariance constraints. In particular, for the first time, we achieve valid causal inference on a scientific experiment with complex recording and no human annotations, fine-tuning foundational model on our similar annotated experiment.
Introduction
Artificial Intelligence systems hold great promise for accelerating scientific discovery by providing flexible models capable of automating complex tasks. We already depend deep learning predictions
across various scientific domains, including biology [Jumper et al.) 2021 Tunyasuuuuakool et al.
2021). medicine [Elmarakeby al. 2021 Mullowney et al. [023]] sustainability [Zhong al.
2020 Rolnick et al.) 2022]. and social sciences Perzak et al.) 2022 Daoud et al., 2023]. While these models offer transformative potential for scientific research, their black-box nature poses challenges, especially when used to analyze new experimental data. They can perpetuate hidden biases, which difficult to detect and quantify, potentially invalidating any analysis resting their predictions [Cadei et al. 2024).
Recent works proposed leveraging predictive models for efficient and still valid statistical inferences on partially labeled data (Angelopoulos et al. 2023a]] b), training the predictor on the annotated sample and then estimating the statistical quantities of interest, including the model's 's predictions the unlabeled sample. In this paper, we focus on causal inferences from (outcome) unlabeled experiments. We similarly aim to retrieve the missing labels from their complex measurements, i.c., entangled and nigh-dimensionall such as images or vidcos, through a predictive model. However, we consider train- ing such model from similar yet different annotated experiments, e.g., different treatments effect modifiers. We refer to this problem as Prediction- Poweerdd Causal Inference (PPCI), differing from classic Peediction-Powered Inference for its motivation, i.e., causal, and setting, i.e., generalization.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

To tackle this problem. we first characterize when predictor is valid for a given PPCI problem. We show that (statistical) conditional calibration with respect to the observed outcome parents preserves identifiability and still guarantees consistent estimation, e.g., via Augmented Inverse Propensity Weighted (AIPW) estimator [Robins et al. 1994]. for the treatment effect the unlaheled experiment. However, the validity of a predictor may not transfer even between similar experiments and assuming an infinite training sample by Empirical Risk Minimization, even with standard invariance constraints [Arjovsky et al. 2019 Krueger et al. 2021J. We then propose a sufficient condition to enforce in the representation backbone of the predictor, causal lifting the model to transfer causal validity on the target experiment (together with other standard, yet idealized, assumptions).
We further propose a novel, simple yet effective implementation of such a constraint via loss reweight- ing, which we name Deconfounded Empirical Risk Minimization (DERM). To test it, we considered ISTAnt experiment [Cadei et al. 2024] (unique real-world benchmark for treatment effect estimation with complex measurements), ignoring the outcome annotations, and trained the predictive model over a new annotated experiment of ours with the same annotation mechanism, but different recording plat- form (lower quality) and treatments. We further validate and confirm the results on synthetic manip- ulation of MNIST dataset [eCCnn 1998) by controlling the data-generating process and causal effect.
More broadly, this paper emphasizes the "representation learning" aspect of "causal representation learning" [Schölkopf et al. 202T], which has traditionally focused on the theoretical identification of a representation without concerning any specific task. In Bengio et al. [2013], good representations are defined as ones "that make easier to extract useful information when building classifiers or other predictors." In similar spirit, we focus representations that make extracting causal information possible with some downstream estimator. We hope that our viewpoint can also offer new benchmarking opportunities that are currently missing in the causal representation learning literature and have great potential, especially in the context of scientific discoveries.
Overall, our main contributions are:
formulation and foundational theory for Prediction-Poweree Causal Inferences, i.e., prediction-powert causal estimands identification and estimators properties,
ii a new constraint to transfer causal validity across similar experiments, causal lifting neural representations with -ut-of-distribtiio fine-tuning, alongside a practical implementation (DERM).
ii.. first valid causal inference scientific experiment (ISTAnt) with complex measurements and no human annotations, by fine-tuning model on our similar annotated experiment.
2 Prediction-Powered Causal Inference
In this Section, we formulate the PPCI problem, introducing the terminology and general identification and estimation results characterizing a causally valid outcome model.
Problem: Prediction-Powerdd Causal Inference (PPCI)
Let (T, X,Y P be random variables, where T T denotes treatment assignment, W E W the observed covariates, Y E y the outcome of interest, and X E X' a complex measurement capturing Y information. Let g X' y/ be fixed predictor of Y from X.
Given an i.i.d. sample [(T;, W,, X,) 1.1.d. P, where the outcomes are not observed directly, identify and estimate the average potential outcome under a treatment intervention"] i.e., (() = E| do(T 1)), or related estimands, from the cettiictio-pooeere sample ((T., W. (X,))
"We focus on population- level causal estimands, but the problem generalizes group-level quantities too by additionally conditioning on a covariates subset.
Several scientific problems fit under this framework, where a human domain expert, trained on similar data, acts as a predictor y, elaborating and annotating the high-dimensional measurements, aiming to collect evidence to answer the overarching scientific question. Replacing this step with deep learning model requires guarantees, which generally differ from standalone prediction accuracy metrics. Systematic errors in specific subgroup can invalidate any causal reasoning (Cadei et al.)

---

2024 Cadei and Internó 2025]. However, the PPCI potential comes from the fact that good outcome models can be learned also from similar experiments and transferred zero-shot, e.g.. changing setup and treatment of interest, as long as the (true) annotation mechanism stays invariant, Such generalization goal the main motivation behind the investigation of this problem, going beyond (statistical) Peediction-Powerts Inference (Angelopoulos et al., 2023a b) that is designed instead for efficiency in-distribution. As a practical, real-world motivating problem, we aim to achieve valid Average Treatment Effect estimation ISTAnt without expensive human annotations, but by annotating instead with foundational models fine-tuned over our annotated similar experiment, with different treatments and recording platform. See the problem illustration in Figure 1
Ours
ISTAnn
agg cetlage
receedlar
gpontting"
CSV
CSV
T
W
Y
X'
((X) C Estimate AT EJY [do'T" IH zly doll
T
W
X
Train Eine-tiee
( an outosme model ÿ
- *A"]]
Figure I: Pipeline for Average Treatment Effect inference on ISTAnt experiment without human annotations. We designed, recorded, and annotated our similar experiment with different treatments and recording platform and leveraged fine-tune pre-trained model for imputation.
Before discussing the generalization challenges and our proposed mitigation, we formalize the properties predictor requires to guarantee valid PPCI. Note that, in the following discussions, we always start by assuming identifiability of the average potential outcome(s), e.g., Randomized Controlled Trial. Indeed, if the problem has no solution even with access the true outcomes, the corresponding PPCI problem loses meaning |
Definition 2.1 (Valid Predictor). An outcome predictor g is valid for a PPCI problem and only if the average potential outcome(s) of interest Ty (t) identify in a statistical estimand, i.e., treatment function removing the do-operator, and Ty (t)=
For example, given PPCI problem S with binary treatment, let the average potential outcome upon intervening on the treatment value be identified via the adjustment formula by standard causal inference assumptions i.c.. Ty (1) Eaw [Ey (V[7 = 1, W]]
where W is a valid adjustment set. A predictor is valid for S if Ty (t)=Tg(x)(t).i) i...
Ew [E [T W]] Eww Ex (X W]].
(2)
Then, we want characterize the testable statistical property model has satisfy to be valid.
Definition 2.2 (Conditional Calibration). Let X, Y P random variables with X E X and E V. A predictor X + + is conditionally calibrated over if and only if:
E|Y 9(X) a5
(3)
Lemma 2.1 Iiddiction-Poweeee Identification). Given PPCI problem T with identifiable
ATE (given the ground-truth annotations). If an outcome model X + y) is conditionally calibrated with respect to the treatment and the valid adjustment set considered for identification. then it is (causally) valid for 30
The lemma follows directly from backdoor adjustment, see proof in Appendix A and offers a sufficient condition for validity. worth observing that being calibrated with respect to all the
Unless an unobserved and valid adjustment set can identified from complex measurements X, such a discussion is outside the scope of this paper. Consistenccy exchangeability, and overlap.
3

---

observed covariates and not only the valid adjustment set is just a stronger sufficient condition, still holding the thesis. For example, in experiments where there are few discrete experimental conditions. it is safe to enforce/test it for all the covariates, without requiring a prior knowledge of which is the valid adjustment set.
Lemma 2.2 (Prediction- Prediction-Poweree Estimation). Given PPCI problem P, with identifiable
ATE (given the ground -ruth annotations). If an outcome model g : X ++ + is conditionally calibrated with respect to the treatment and a valid adjustment set, then the corresponding AIPW estimator over the prediction-powered sample preserves doubly-robusunee and valid asymptotic confidence intervals, i.e.,
NNO,
(4)
where V the asymptotic variance.
As before, the result is intuitive and now extends such a sufficient condition to preserve the property of an estimator. The proof, despite being technical, follows directly from classical AIPW results, and we present it in Appendix A practice, it important to note that, while several estimators may exist for the same causal estimand, we need to rely on a valid predictor for the statistical estimand we identify the causal one with. For example, in a randomized controlled trial, we may identify the ATE with the associational difference, and therefore require calibration only with respect to the treatment; if, in practice, we want to use an estimator relying on covariates conditioning to improve efficiency, e.g., AIPW, we must require the outcome model to condition on all such covariates too. We still have not discussed how to enforce or even test such a property, without access to outcome annotations on the target experiment, and start illustrating the fundamental generalization challenges in learning a valid outcome model.
2.1 Out-of-distribution generalization challenges
The main challenge in PPCI is training a valid outcome model to replace expensive and slow annotation procedures. It has to be expressive enough to process the complex input measurements, while being calibrated with respect to the experimental settings. Although modern pre-trained models are getting more and more capable and generalist [Bubeck et al., 2024], they still cannot guarantee calibration a priori [Chen al. 2022). As human annotator learns annotation procedure from similar experiments with the same measurement type (c.g., videos) and outcome of interest (c.g.. ants' behaviours), we may aim to fine-tune pre-trained models similar annotated experiments, leading towards scientifically valid expert models. For this purpose, let's first provide a notion of similarity among experiments.
Definition 2.3 (Similarity). Two PPCI problems P and 9 are similar if their outcome- mechanism is invariant, i.e.,
P(Y X 2) =) PPY'''' = x) Vo E supp(X) supp(X')
This definition naturally generalizes to whatever measuremenntouutcome couple (X,Y) P such that the annotation mechanism PY X x) is invariant. Note that our setting is different from the invariant causal predictions [Peters et al., 2016), as in our case P(Y |X x) is not causal mechanism that we want to identify using an invariance assumption over multiple environments, but rather X is measurement of Y SSila et al. 2006 Yao et al., 2025 (and thus child of Y), potentially observed over a single environment P. We can rely on this anticausal invariance whenever the same annotation protocol could be used for the two experiments (for example, humans trained to annotate S are capable of annotating G' without new training or instructions). Still, vanilla fine-tuning by standard Empirical Risk Minimization over similar experiments doesn't guarantee learning valid outcome model due to peculiar generalization challenges raised by the spurious shortcuts appearing in anticausal prediction problems [Schölkopf et al.) 2012 Peters et al.) (0166), for example, from the visual appearance of certain effect modifiers. We distinguish here between statistical challenges with a finite training sample and potential issues even in the infinite sample regime.
Finite training sample If the training sample is not representative of the full target population, the model may over-rely on its spurious conditional mcasurement-outtomm correlation, c.g., some

---

irrelevant background information. Concretely, the measurements in such a subgroup are over- representing a certain treatment value or effect modifiers, e.g., up violating positivity, which may bias any training and invalidate any causal downstream prowertion-powere estimation.
Example 1: Background confounding
Consider PPCT for ISTAnI, annotating only the first batch of nine videos for training. If certain behaviour is underrepresented in a specific plate, e.g., because the plate is in the control group, the model can leverage such spurious shortcut in subsequent batches. relying on background information like the pen markings on the floor to identify the plate. This invalidates any downstream prrdiction-powertt analysis, as future plates may be treated, but the model may assume they are not, matching the target statistics with the control group.
Infinite training sample Additionally, treatment information captured in the measurement that is not mediated by the outcome may be misleading for generalization, even in the infinite training sample setting.
other words, a double blind experiment needs to be blind also for the model, which should not be able to guess the treatment and other effect modifiers if not through the effect. Otherwise, the causal relationships in the training data can be wrongly transferred to the target experiment, potentially never happens in the training treatment, i.c., supp(Y" = t') C supp(Y'), ), the model will wrongly transfer this information to target population with different treatment but with the same appearance, invalidating any causal downstream iiliiction-poweere estimation there.
different variables and distributions. For example, if an outcome value
Example 2: Different treatments with similar appearance
Consider two experiments where the treatments are visually identical, discernible from the control, and differing in the effect, e.g., different pathogen suspension exposure with liquid residual in the measurements. While learning, outcome model may over-rely on the training treatment effect, e.g., excluding the possibility of a specific behaviour, and wrongly transfer during PPCI on the target experiment. Note that this is not the case for our experiment and ISTAnt where the treatment is not visible, as for well well-designed double blind experiments.
3 Causal Lifting of Neural Representations
In the previous Section, we introduced and discussed PPCI and its peculiar challenges to learn valid outcome model from training data out-of-distrriutiin. this Section, we introduce a new representation learning constraint to prevent such generalization challenges, and a corresponding implementation, tailored for our application of interest.
Definition 3.1 (Causal Lifting constraint). Given PPCI problem P, with identifiable ATE (given
the ground-truth annotations), and predictor g hoo: X + y. The encoder & Rd satisfies the Causal Lifting constraint if:
(o(X), ZIV)
(6)
where Z = [T, W). and W C W is valid adjustment set for the ATE identification.
The Causal Lifting constraint is a condition on the model encoder to prevent the possibility of leveraging unnecessary spurious outcome information from the visible experimental settings.
Theorem 3.1 (Causal Lifting transfers causal validity). Given two similar PPCI problems
9 and G with identifiable ATE. Let g h o o X + be an outcome model for ep' with h Bayes-optimal, and assume that the representation transfers, i.e., P = y)
P()(X'))Y' =y) Then the Causal Lifting constraint on P implies g is valid on P.
5

---

Theorem 3.1 offers a sufficient condition to transfer model validity from a training experiment to target experiment by assuming the Causal Lifting constraint holds on target; see proof in Appendix [A] It combines the strengths of modern deep learning models, able to extract sufficient representations on training tasks, i.e., (()) = X) together with efficient fitting on limited data, i.e., Bayes Optimal classifier assumption. We remark that the assumption that the representation transfers is reasonable, since invariant annotation protocol commonly exists. In practice, the Causal Lifting constraint is required to hold on the target experiment, where we cannot get any guarantees without labels. However, referring to the generalization challenges examples Section
in Example 1, the causal lifting constraint prevents the outcome model to leverage the spuri- ous shortcuts from the background information by enforcing it to be position independent, i.e., background-agnosticc,
in Example 2, it prevents the outcome model to leverage the spurious training treatment effect information, by enforcing it to be treatment agnostic.
Despite the unverifiability of this assumption, we practically rely on the bet that transferring such property is an easier generalization task than transferring conditional calibration directly. We then propose to enforce the causal lifting constraint during training/fine-tuiing on the freatment and the valid adjustment sel for the target experiment, when also observed. If not observed but constant in the training experiment, the condition is satisfied by default (no signal for the model learn such experimental settings). As pre-trained models over huge generic corpora get capable of detecting all visible information that can correlate with a prediction target, the causal-lifting constraint has to be enforced explicitly during transfer learning, forcing the artificial annotation process to be "blind" with respect to certain information. Similar to the Conditional Calibration assumption, the outcome is well predictable from the measurement, enforcing the causal lifting constraint on more experimental settings than necessary does not hurt.
3.1 Deconfounded Empirical Risk Minimization
The Causal Lifting constraint is conditional independence condition, and as such, enforcing it during training is well-known problem in the Representation Learning literature. Different direct and indirect approaches have already been proposed, and in Section 4 we overview the major paradigms. Here, we propose a simple implementation that is tailored to common assumptions in scientific experiments (and that are true in ISTAnt), i.e., that experimental settings are low-dimensional and discrete (Pearl et al. 2000 Rosenbaum et al. 2010]. Other more general implementations are certainly possible if such assumptions do not hold, but this is beyond the scope of this paper. We leverage resampling approach [Kirichenko et al. 2022 Li and Vasconcelos) 2019]. reweighting the training distribution to simulate a distribution that has correlation between outcome and experimental settings, while keeping the annotation mechanism invariant. We define the manipulated distribution of Z as function of Z, Y and their joint distribution:
if [supp(Y|22 = 2) = 0
P(Y 12"=)
P(Z =z)
P(Y y. 7* =) 2):
Var 2)
otherwise
(supp(Y Y = 2)
Var(Y = x')
2'EZ
for all y E v 2 E 2, where joint distribution depends only on the experimental settings values, i.e., the conditional outcome distribution F( 2) is uniform, while the experimental settings marginal P(Z 2) weights more the observations with the least outcome-ing infomm--ffrmaaiiee experimental settings (high conditional variance) over the training population, and ignores the most informative ones (low or null variance).
= 0}. By design such
If the conditional outcome support is full for each not fully informative covariates value, i.c.
supp(Y|22 2) sY) Vz E Z Var(Y) = =)>>
(8)
then, the conditional outcome distribution is constant and Y 11 Zt. It follows that an outcome model trained on such distribution has no signal from the outcome to learn the uncorrelated experimental settings, naturally enforcing the Causal Lifting constraint.

---

We then propose to train/fine-tune the factual outcome model minimizing the empirical risk, e.g., via Stochastic Gradient Descent, on such "disentangled' distribution reweighting each observation by factor:
P(Y y, P(Y = y,
computed una-tantum before starting the training, estimating the joint training distribution by fre- quency (denominator), and the conditional variances by sample variance (nominator). We refer to this procedure as Deconfounded Empirical Risk Minimization (DERM).
Finally, we remark that when the condition in Equation @does''t ' hold, it is not possible to specify a 'isennanngled' joint distribution without ignoring the experimental settings, i.e., P = = 0, where the conditional outcome's 's support is strictly contained in the marginal outcome's s support on the training distribution. Our manipulated distribution will still consider these samples, but reducing their weight with respect to the predictivity of the observed experimental setting (approximately x Var(Y = =)). Therefore, some spurious correlations may still be learned, but this is now in a trade-off with ignoring potentially substantial part of the reference sample. While this does not apply to our application of interest, tailored modifications our joint distribution can be proposed case-by-case.
Related Works
Prediction-Powereee Inference The factual outcome estimation problem for causal inferences from complex observations was first introduced by Cadei et al. (2024). We provide here its first formulation and corresponding foundational discussions and valid approach to the problem, i.e., causal lifting pretrained neural representation. Particularly, we formalize what they describe as "encoder bias", and our DERM is the first proposal their call for "new methodologies to mitigate this bias during adaptation" under similar yet different fine-tuning distribution. Demirel et al. [2024), Wang et al. [2024] already attempted to discuss similar generalization challenges with artificial predictions for causal inferences. However, they both ignored the main problem motivation to retrieve the latent outcomes from complex measurements, and not just weak signal from limited experiment settings, relying instead unrealistic distributional assumptions. They ignored any entangled and high- high-dimensional measurement the outcome of interest and assumed that a few experimental settings alone are sufficient for factual outcome estimation, together with support overlapping. i.e., without generalization, making any model application-specinn ceneral-purposse foundational models. Let's further observe that their framework is special case of ours when X is interpretable and low-dimensional, and the target experiment in-distribution. In contrast to the classic Ieddiction-Powerts Inference (PPI) [Angelopoulos et al., 2023a b). which improves estimation efficiency by imputing unlabeled in-distribution data via a predictive model, and recent causal inference extensions (Wang et al. 2024 De Bartolomeis et al. [025 Poulet et al.,
and ignoring any connection with
2025) relying on counterfactual predictions, PPCI focuses on imputing missing, factual outcomes to generalize across similar experiments, yet out-of out-of-distribution
Learning representations invariant certain attributes is widely studied problem in machine learning [Moyer et al. 2018 Arjovsky et al., 2019 Gulrajani and Lopez Paz 2021 Krueger et al., 2022 Yao et al.) 2024a]. Practically, we aim learn representations of X that can predict the outcome Y, but are invariant to the experimental variables Z. In agreement with Yao et al. 2024aa. we combine a sufficiency objective with our causal lifting constraint in the representation space, enforcing the suitable representation invariance. Several alternative approaches can be considered enforce such conditional independence constraints: (i) Conditional Mutual Information Minimization (Song et al.
Invariant representations
et al.) (2020) Gupta et al. 2021]. (ii) Adversarial
Independence Regularization such as Louizos et al. [013] which modifies variational autoencoder
(VAE) architecture in Kingma [0144] to learnfai representations that are invariant sensitive attributes,
by training against an adversary that tries to predict those variables, (iii) Conditional Contrastive Learning such as Ma et al. [2021] whereby one learns representations invariant to certain attributes by optimizing a conditional contrastive loss, (iv) Variational Information Bottleneck methods where one learns useful and sufficient representations invariant to specific domain [Alemi et al. 2016 Li
jtt al. 2022). We chose our DERM procedure as matches well our application of interest, but other applications may prefer different conditional independence implementations.
7

---

Causal Representation Learning In the broader context of causal representation learning methods (Schölkopf al. 202T], our proposal largely focuses on representation learning applications causal
inference: learning representations of data that make it possible to estimate causal estimands. We find in contrast with most recent works in causal representation learning, which uniquely focused on complete identifiability of all the variables or blocks, see Varici al. [2024), von Kügelgen [2024] for recent overviews targeting general settings. The main exceptions are Yao et al. [2024a bj. The former leverages domain generalization regularizers to debias treatment effect estimation in ISTAnt from selection bias. However, their proposal is not sufficient to prevent confounding when no data from the target experiment is given. The latter uses multi-view causal representation learning models to model confounding for adjustment in an observational climate application. In our paper, we also discuss conditions for identification, but we focus specific causal estimand, as opposed to block- identifiability of causal variables. As opposed to virtually all previous work in causal representation learning, perspective offers clear evaluation and benchmarking potential real-world scientific experiments even theoretically under-specified settings via the accuracy of the causal estimate, also recently proposed by Yao al. 20255
5 Experiments
In pursuit of scientific validity, we evaluate our method (DERM) on the ISTAnt experiment (Cadei al. 2024], and additional synthetic experiments controlling for the true causal effect.
ISTAnt We performed similar experiment to the ISTAnt experiment on our recording platform, considering three different treatments and collecting 44 annotated 30-minuth videos, and we leveraged such annotated videos to fine-tune tune a pretrained model generalizing the predictions to ISTAnt and then estimating the ATE by AIPW on such mrediction-poweee
sample. We considered the AIPW estimation on the (true) ISTAnt human annotated outcomes as the ground truth ATE (approximately +-40 short clips with 2 clips per second, i.e., 2 20s, in grooming time per video). Further details the modeling choices, hyper-parameter, and fine-tuning are discussed in Appendix Figure 3 shows the differences in filming setup among the two experiments.
Ground Futh
vREx
VITS VIT
CUP-VITS
CUPVTIL
DNCx2
(a) Our experiment
(b) ISTAnt
Figure 2: РР-ATE inference on ISTAnt, fine-tuning different pre-trained encoders on our similar experiment, varying the training objective. Reported the 95% confidence intervals (via AIPW asymptotic normality) and compared against the same inference using the annotated outcomes. In green the correct confidence intervals (overlapping 33+%) in red the incorrect.
Figure 3: Filming box and ex- ample frame from our experiment and ISTAnt. The two experiments mainly differ in lighting conditions, treatments, experimental nests (wall height) and color marking.
Figure reports the ATE estimation 95% confidence intervals per wethod-encoded (five different pre- trained backbones) by AIPW asymptotic normality, selecting per group the model across minimizing the treatment effect bias reference. We trained multiple nonlinear heads, i.e., multi-lay Malti--lyerrane perceptron, on top via (i) vanilla ERM, (ii) Variance Risk Extrapolation (vREx) Krueger et al.) [202]], (iii) Invariance Risk Minimization Arjovsky et al. (2019) (IRM) and (iv) DERM (ours). As expected, fine-tuning vanilla ERM leads to consistently underestimatingiiinorin the effect (no validity guarantees). IRM and vREx, proposed by Yao et al. (2024a], performs better, despite still being

---

occasionally invalid. DERM is the only method consistently providing valid ATE inference for all the pre-trained models, without ever estimating the effect with the wrong sign, unlike ERM, vREx and IRM.
CausalMNIST
We replicate the analysis on colored manipulation of the MNIST dataset, e-g.. estimating the effect of the background color or pen color on the digit value, allowing complete control of the causal effects and uncertainty quantification by Monte Carlo simulations. We train a convolutional neural network on an experiment and we test PPCI on four similar observational studies, varying the treatment effect mechanism and experimental settings distribution, challenging the full support assumption. i.e., Equation 8 A full description of the data-generating processes and analysis is reported in Appendix [ The results are reported in Table clearly showing the DERM potential to solve unsolvable instances for both ERM and invariant training (see soft shifts columns), while suffering when challenging its assumptions (see hard shifts columns).
Table I: PPCI results on CausalMNIST varying the target experiment population (in-, out-of- distribution), the treatment effect mechanism (linear, non-linear) and challenging the full-support guarantees (soft, hard shifts). We report the ATE estimation bias and standard deviation over 50 repetitions of the data generating process.
OoD (linear effect)
OoD (non-linear effect)
Method
In In-Distribution
Soft shift
Hard shift
Soft shift
Hard shift
Ground Truth
0.00 = 0.02
0.00 = 0.02
0.00 = 0.02
0.04 = 0.05
0.03 = 0.05
ERM
0.00 = 0.02
0,86 = 0.14
1.05 = 0.15
0.64 = 0.18
0.85 = 0.17
vREx
0.01 = 0.03
0.83 0.15
1.05 = 0.14
0.55 +
0.82 = 0.14
IRM
0.01 + 0.03
0.76 + 0.18
1.02 = 0.12
0.45 + 0.18
0.77 = 0.17
DERM (ours)
0.10 = 0.07
0.14 = 0.14
0.75 = 0.05
0.08 + 0.27
0.45 + 0.12
6
Conclusion
In this paper, we formalize the problem of obtaining valid causal inference when the outcome variable latent, and captured by complex measurements. Motivated by scientific applications, we discuss how to train an outcome predictor on similar annotated data, e.g., collected in prior experiments, and achieve valid causal conclusions on the prediction predietion-powered target experiment. We developed the first identification and estimation results for the problem, alongside a practical constraint on the model representation to transfer causal validity under standard yet idcalized assumptions. We additionally offered novel and effective implementation of the such constraint (Deconfounded Empirical Risk Minimization), and showed that it achieves zero-shot identification of the ATE on ISTAnt by leveraging (our) similar annotated ecological experiment. Overall, work offers paradigm shift from the traditional Causal Representation Learning literature (Schölkopf et al. [2021] paving the way toward learning representations that power downstream causal estimates on real-world data. We think that this has the potential to play a critical role for deep learning models to accelerate the analysis of scientific data and ultimately new discoveries.
The main limitation of this work is that, in practice, it is not possible to verify whether either the conditional calibration property the representation constraint actually holds on the target population without annotations. Beyond our key assumption that the representation transfers, we also did not discuss the model convergence on finite training datasets, so even if enforcing the representation constraint on training would transfer, we may still suffer from finite sample effects. Another limitation is neglecting several settings that are, potentially, practically relevant, e.g., not overlapping effect modifiers among similar experiments, or the violation of Equation [ As our target experiments did not have these problems, left them for future work. Finally, we rest on the assumption that the underlying causal inference problem is statistically identifiable, but this is necessary and can be ensured with a well-designed experimental protocol. Despite these limitations, we hope our work can inspire a new generation of practical Causal Representation Learning methodologies and create opportunities for more systematic and practically relevant benchmarking.
By AIPW asymptotic normality on the true factual outcomes.

---

Acknowledgments
We thank the Causal Learning and Artificial Intelligence group at ISTA for the continuous feedback on the project and valuable discussions. We thank the Social Immunity group at ISTA, particularly Jinook Oh, for the annotation program and Michaela Hoenigsberger for supporting our ecological experiment. Riccardo Cadei is supported by Google Research Scholar Award and Google Initiated Gift to Francesco Locatello. This research was funded in part by the Austrian Science Fund (FWF) 10.55776/COE12). It was further partially supported by the ISTA Interdisciplinary Project Committee for the collaborative project "ALED" between Francesco Locatello and Sylvia Cremer. For open access purposes, the author has applied CC BY public copyright license to any author accepted manuscript version arising from this submission.
10

---

References
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool. Russ Bates, Augustin Zidek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. nature, 596(7873):583-588) 2021.
Kathryn Tunyasuvunakool, Jonas Adler, Zachary Wu, Tim Green, Michal Zielinski, Augustin Židek, Alex Bridgland, Andrew Cowie, Clemens Meyer, Agata Laydon, et al. Highly accurate protein structure prediction for the human proteome. Nature, 596(7873):390-596, 2021.
Haitham A Elmarakeby, Justin Hwang, Rand Arafeh, Jett Crowdis, Sydney Gang, David Liu, Saud H AlDubayan, Keyan Salari, Steven Kregel, Camden Richter, et al. Biologically informed deep neural network for prostate cancer discovery. Nature, 598(7880)3348 352, 2021.
Michael W Mullowney, Katherine R Duncan, Somayah S Elsayed, Neha Garg, Justin JJ van der Hooft, Nathaniel Martin, David Meijer, Barbara Terlouw, Friederike Biermann, Kai Blin, et al. Artificial intelligence for natural product drug discovery. Nature Reviews Drug Discovery, 22(11): 895 916, 2023.
Miao Zhong, Kevin Tran, Yimeng Min, Chuanhao Wang, Ziyun Wang, Cao-Thang Dinh, Phil De Luna, Zongqian Yu, Armin Sedighian Rasouli, Peter Brodersen, et al. Accelerated discovery of co2 electrocatalysts using active machine learning. Nature, 581 (7807))) 2020.
David Rolnick, Priya Donti, Lynn H Kaack, Kelly Kochanski, Alexandre Lacoste, Kris Sankaran, Andrew Slavin Ross, Nikola Mllojevic-uuponi Natasha Jaques, Anna Waldman-Brown, et al. Tackling climate change with machine learning. ACM Computing Surveys (CSUR), 1-92): 2022.
Connor T Jerzak, Fredrik Johansson, and Adel Daoud. Image-based treatment effect heterogeneity. arXiv preprint arXivv2206.066 064117 2022.
Adel Daoud, Felipe Jordán, Makkunda Sharma. Fredrik Johansson, Devdatt Dubhashi. Sourabh Paul, and Subhashis Banerjee. Using satellite images and deep learning measure health and living standards in india. Social Indicators Research, (67(1):475-50)) 2023.
Riccardo Cadei, Lukas Lindorfer, Sylvia Cremer, Cordelia Schmid, and Francesco Locatello. Smoke and mirrors causal downstream tasks. arXiv preprint 1711v:2405..71551 2024.
Anastasios N Angelopoulos, Stephen Bates, Clara Fannjiang, Michael Jordan, and Tijana Zrnic inddiction-owwered inference. Science, 382 674, 2023a.
Anastasios N Angelopoulos, John C Duchi, and Tijana Zmic. Ppi++: Efficient prediction-powered inference. arXiv preprint arXiv:2311.01453. 2023b.
James Robins, Andrea Rotnitzky, and Lue Ping Zhao Estimation of regression coefficients when some regressors are not always observed. Journal of the American statistical Association, 89(427): 846-866, 1994.
Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint urXiv:19077 002999, 2019.
David Krueger, Ethan Caballero, Been-HHerrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. tion (rex). In International conference on machine learning, pages 5816- PMLR, 2021.
generalization via risk extrapola-
Yann LeCun. The mnist database of handwritten digits. http://yann. lecun, com/exdb/mnist// 1998.
Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. Proceedings of the IEEE, 109(5):612-66)) 634, 2021.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE tronsactions on pattern analysis and machine intelligence, 3588) 2013.
11

---

Riccardo Cadei and Christian Internò. The narcissus hypothesis: Descending to the rung of illusion. arXiv preprint arXiv:2509.1799, 2025.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee. Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arxiv 2023. arXiv preprint arXiv:2303.12712: 10, 2024.
Yangyi Chen, Lifan Yuan, Ganqu Cui, Zhiyuan Liu, and Heng Ji. A close look into the calibration of pre-trained language models. arXiv preprint arXiv:2211.0015/ 2022.
Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen. Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society Series B: Statistical Methodology, 78(5):947-0112, 2016.
Ricardo Silva, Richard Scheines, Clark Glymour, Peter Spirtes, and David Maxwell Chickering. Learning the structure of linear latent variable models. Journal of Machine Learning Research, 7 (2), 2006.
Dingling Yao, Shimeng Huang, Riccardo Cadei, Kun Zhang, and Francesco Locatello. The third pillar of causal analysis? measurement perspective causal representations, arXiv preprint arXiv:2505. 17708, 2025.
Bernhard Schölkopf, Dominik Janzing, Jonas Peters, Eleni Sgouritsa, Kun Zhang, and Joris Mooij. On causal and anticausal learning. In Proceedings of the 29th International Coference on International Conference on Machine Learming, pages 459-466, 2012.
Judea Pearl et al. Models, reasoning and inference. Cambridge, UK: CambrrageeUnvversiisPeeess,, 19 (2):3, 2000.
Paul Rosenbaum, P Rosenbaum, and Briskman. Design of observational studies, volume 10. Springer, 2010.
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness to spurious correlations. arXiv preprint S0tV:22204.02333, 2022.
Yi Li and Nuno Vasconcelos. Repair: Removing representation bias by dataset resampling. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 1572-9588) 2019.
l1ker Demirel, Ahmed Alaa, Anthony Philippakis, and David Sontag. Prediction-powered generaliza- tion of causal inferences. arXiv preprint (rxiv:24006028733, 2024.
Yuxin Wang. Maresa Schröder, Dennis Frauen, Jonas Schweisthal, Konstantin Hess, and Stefan Feuerriegel. Constructing confidence intervals for average treatment effects from multiple datasets. arXiv preprint 1rXiv:24/2./15111 2024.
Piersilvio De Bartolomeis, Javier Abad, Guanbo Wang, Konstantin Donhauser, Raymond M. Duch, Fanny Yang, and Issa Dahabreh. Efficient randomized experiments using foundation models. arXiv preprint arxiv:2502. 0...222..22,,... 2025.
Poulet, Maylis Tran, Sophie Tezenas du Montcel, Bruno Dubois, Stanley Dur- rleman, and Bruno Jedynak. Prediction-powered inference for clinical trials. medRxiv, pages 2025-01, 2025.
Daniel Moyer, Shuyang Gao, Rob Brekelmans, Aram Galstyan, and Greg Ver Steeg. Invariant representations without adversarial training. Advances in neural information processing systems, 31, 2018.
Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. In International Conference on Learning Representations, 2021.
Dingling Yao, Dario Rancati, Riccardo Cadei, Marco Fumero, and Francesco Locatello. Unifying causal representation learning with the invariance principle. arXiv preprint arXiv:2499.0277. 2024a.
12

---

Jiaming Song, Pratyusha Kalluri, Aditya Grover, Shengjia Zhao, and Stefano Ermon. Learning controllable fair representations. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 6173.. PMLR, 2019.
Pengyu Cheng, Weituo Hao, Shuyang Dai, Jiachang Liu, Zhe Gan, and Lawrence Carin. Club: A contrastive log-ratio upper bound of mutual information. In International conference on machine learning, pages 1779-1788. PMLR, 2020.
Umang Gupta, Aaron M Ferber, Bistra Dilkina, and Greg Ver Steeg. Controllable guarantees for fair outcomes via contrastive information estimation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35-9, pages 7610-7619, 2021.
Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, and Richard Zemel. The variational fair autoencoder. arXiv preprint arXiv:1511.008.0 2015.
Diederik Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.
Martin Q Ma, Yao-Hung Hubert Tsai, Paul Pu Liang, Han Zhao, Kun Zhang, Ruslan Salakhutdinov, and Louis-Philippe Morency. Conditional contrastive learning for improving fairness in self- supervised learning. arXiv preprint ar arXiv:2106.02866 2021.
Alexander A Alemi, Ian Fischer, Joshua V Dillon, and Kevin Murphy. Deep variational information bottleneck. arXiv preprint arXiv:1612.0044: 2016.
Bo Li, Yifei Shen, Yezhen Wang. Wenzhen Zhu, Dongsheng Li, Kurt Keutzer, and Han Zhao. Invariant information bottleneck for domain generalization. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36-7, pages 7399-7407, 2022.
Burak Varici, Emre Acartürk, Karthikeyan Shanmugam, and Ali Tajer. General identifiability and achievability for causal representation learning. In International Conference on Artificial Intelligence and Statistics, pages 2314-2322. PMLR, 2024.
Julius von Kügelgen. Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Enviromment. PhD thesis, Cambridge University, 2024.
Dingling Yao, Caroline Muller, and Francesco Locatello. Marrying causal representation learning with dynamical systems for science. Advances in Neural Information Processing Systems (NeurIPS), 2024b.
Donald B Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of educational Psychology, 66(5):688, 1974.
Edward H Kennedy, Sivaraman Balakrishnan, and Max G'sell. Sharp instruments for classifying compliers and generalizing causal effects. 2020.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x 16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 2020.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings the IEEE/CVF International Conference on Computer Vision, pages 11975-11986. 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8763. PMLR, 2021.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023.
13

---

Sören R Künzel, Jasjeet S Sekhon, Peter J Bickel, and Bin Yu. Metaleamnrrs for estimating heteroge- neous treatment effects using machine learning. Proceedings of the national academy of sciences, 16(10))4156-4165, 2019.
Hugh A Chipman, Edward George, and Robert E McCulloch. Bart: Bayesian additive regression trees. 2010.
Stefan Wager and Susan Athey. Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113(523):1228-122 1242, 2018.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: See the problem formulation and analysis in Section 2 the method in Section 3 and the experiments in Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
Limitations are discussed in Section together with the conclusion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

Detailed statements and proofs of the theoretical results (2 Lemmas and 1 Theorem) are reported in Appendix A In the main paper slightly more concise formulations are reported in favor of the reading smoothness.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Detailed description of the experiments is reported in Section and Appendices B Our new experimental ecology dataset preview is anonymously shared on Figshare Our method is exhaustively described in Section 3.1 We additionally share the Python implementation of all the experiments the Supplementary Materials.
at httpss//tigshare.com wecchhrrr.com//// htt.s//ooo/////////// 5944400661eccbb/73335555
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the partieular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
16

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We share all the code implementation in the supplementary material. Our new experimental ecology dataset preview is anonymously shared Figshare We further rely on ISTAnt dataset publicly 26484934.v2 The synthetic experiments
available at https://doi.org wwwpe/orgg//O./000.0.0 (0..0044m9..iggeeee highaare.26484994... on CausalMNIST relies on MNIST dataset LeCun 1998), publicly available. Additional experimental details are reported in Section and Appendices C Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cc)
Seallic/giiees/ooee SodeeSSumiisiiioool11.....
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
ptaiss.cc/publicguuudees wtw.:/ddeeeeeoeSSeeeeeeeeeeeeeee
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
See experimental details in Section 5 and Appendices B
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: For the real-world experiment, confidence intervals by AIPW asymptotic normality are reported in Figure [ For the synthetic experiment, both the data generating processes and trainings are repeated 50 times, and we report mean and standard deviation in
17

---

Table 1 fully describing valid confidence intervals by sample mean asymptotic normality (Monte Carlo simulation).
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi-
dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Computer resource requirements are discussed Appendices
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics Ethics/httss:/
Answer: [Yes]
Justification: Data collection all experimental work were performed in compliance with international, national and institutional regulations and ethical guidelines. Beyond the new dataset, the authors carefully reviewed the NeurlPS Code of Ethics and believe none of the concerns from NeurIPS Code of Ethics applies to this work.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
18

---

10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: We believe this work has no specific positive or negative societal impacts except to potentially accelerate science an important topic for biodiversity (which is discussed in the paper).
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Both data and analysis don't poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
19

---

We use our proprietary assets and publicly available public assets (pre-trained Hugging Face models and MNIST and ISTAnt dataset), which we correctly acknowledge and cite.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include a URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thwode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Our main asset is the world dataset. A preview is anonymously shared on Figshare at
we plan to publicly release (under license) upon acceptance. A full description of the dataset is reported in Section 5 and Appendix B
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mooel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
We do not work with human subjects or crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
20

---

15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
We do not work with human subjects or crowdsoureing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
21

---

Appendix
Table of Contents
A Proofs
22
A Proof of Lemma 2.1
22
Proof Lemma
23
Proof Theorem
25
B TSTAnt
26
B.1
Training annotated experiment data recording (ours)
26
B.2 Training and analyses details
26
CNuSTIMNIST
28
Data Generating Process
28
Additional Ablations: Causal Estimators
................................
29
Training and analyses details
30
A
Proofs
Note: In the rest of the manuscript we use the expression "under standard causal identification assumptions" refer the canonical conditions for identifiability observational study (and randomized controlled trial too) [Rubin 1974), i.e..
Stable Unit Treatment Value Assumption (SUTVA), i.e., not interference and no hidden version of the treatment,
Overlap Assumption, i.e., 0 < B[ = w < for all w W,
Conditional Exchangeability (or Unconfounduess) Assumption, i.e.,
P(Y]do((T 1), W = w) P w) VW E W,LET.
A.1 Proof of Lemma 2.1
Lemma (Prediction-Powered identification assumptions. If an outcome model g X + y is conditionally calibrated with respect to the treatment and a valid adjustment set, then it is (causally) valid for P.
Powered Identification). Given a PPCI problem P with standard causal
Proof. Given the identification of the causal estimand. the thesis follows directly by the tower rule over valid adjustment set W and leveraging the conditional calibration:
Ty (t) (ElY|do(? t)] = Eir Ey Y -1), W]]
(10)
(11)
Ew Ey
(12)
Ew Ex W]] (t) We T.
(13)
22

---

A.2 Proof of Lemma 2.2
Lemma (Prediction-Powereed Estimation). Given a PPCI problem P, with identifiable ATE
(given the ground-truth annotations). If an outcome model X + y is conditionally calibrated with respect to the treatment and valid adjustment set, then the corresponding AIPW estimator over the prediction prediction-pwweed powered sample with nuisance function estimators satisfying and valid asymptotic confidence
|| - μlll l|ē el| = op(n (n-n))), intervals, i..,
1/2), preserves
Vn(fo(x) - Tr) +V(O
(14)
where V the asymptotic variance.
Proof. Doubly-Robustness follows directly from the conditional calibration assumption and standard doubly-robust argument for AIPW [Robins al. 1994]. We then focus here on proving the asymptotically normality for Ty of the estimator:
Ti (9(x,) - (Wsx)(W...1) E(W,)
xx = Σ (o,x) - Po(x) +
(15)
1 1 T; ē(W,)
(g(x,) (x,
where (/(x((w: t) is an estimator the true predicted -ottoome model Mg(X) (w. f) = [[g(X) w, T = t). and ((w) is an estimator of the true propensity score e(w) = P(T = 1| = w).
Given generic outcome model and propensity score let us define the influence function of the estimator:
T;
((O.p,eeg) = p(W,, (WW, 0)
e( e(W,) (g(X,) p(W,, 1))
(16)
- T:
1 e(W,) (o(X:) m(W.,
where O, W, X,). Then
fxix) Σg
(17)
1=1
We can rewrite our estimator as:
1
1 -r] (0(0GH,e,))
Fo(x) - TY T1
-.)
(18)
n
where p(w, t) E W w. T the true outcome model and by conditional calibration:
n(w, t) E\y(W w. T= = = Elg(X) W w, =] Vw E W.tET.
(19)
Assuming that the second moment of the random variable & is bounded, by standard central limit theorem argument, the first term satisfies
Vn
Σ ((ORpe)))) - TT 4 N E
(20)
remains to show that the second term multiplied by / goes to zero in probability, i.e. it is asymptotically negligible. To do so, observe that we can rewrite the second term as
Σ
- -P.-PAA)) + P(A.),
(21)
1=1
where I and P,, are the true and empirical target measures; P...) E[.) as it is standard in empirical process theory. Our goal is therefore to show that
(P - (A)) ) + P(A)) = oz
(22)
Ti
T2
23

---

Controlling the term Ti The first term T₁ is easy to control, as it follows directly from the following lemma.
Lemma A.1. Kennedy et al. 2020| Let f(z) be a function estimated from sample NN (Zu+1:... I,), and let P,, denote the empirical measure over (Z1...., Za), which is independent of zN. Then
(11 [18-51)
Op (P, -P) =Op
(23)
vni
Since we have from assumptions that P((); 2,g) - (,;;,,, = =pp(1, it holds that
Ti on (oppnn11//)) (n-1)/)
Controlling the term T2 The second term requires some care. We will focus on the term involving
T. 1; the case for T, follows by symmetry. For 1, after some simple calculations,
have:
T;
A = (W,, -W,,, 1)) +
(o(X,) A(W,, 1))
e(W,)
(24)
e(W.) (g(x;) u(W,, 1)).
Note that we can drop the last term since, by assumption, and μ equal on average. Therefore, we can write:
E B Fl 1) -(W,,, e(W,) (o(X.) A(W., 1))]
(25)
By conditional calibration, we can substitute g(X,) with (WW,, 1) and group:
E A = (Ao,)) 1) u(W, 1)) + e(W,)
(n(W,,1) - (acx)
(26)
(Wo,)) 1) 1)),1)) (1 (1- ē(W)
]
E
(27)
(Rocx) u(W, 8(W) - Ti
(28)
E(W)
Conditioning on W; we obtain:
-)) (u(W,1) (x(x)
E E
((WW)
(29)
To bound this term, we use the positivity assumption that ē(w) 2 2 0 for some constant
e(W) - e(W)
E[J < E
||(W,,) 1) -V,
(30)
é(W)
E [le(W) - e(W)) In(W, Po(x) (W,
(31)
Applying Cauchy-Schwarz inequality:
i [le(W) - e(W) (n(W, 1) - Pocx) (W, 1) < lle- @l2llless 1) (. 1)122.
(32)
the estimators and á achieve suitable convergence rates such that their product of Lg norms is or(n- -1/2), then:
lle ellallu(;,)) Pocx) = or-nn112)
(33)
This completes the proof.
24

---

A.3 Proof of Theorem 3.1
Theorem (Causal Lifting transfers causal validity). Given two similar PPCI problems P and 9' with identifiable ATES given the ground trunt annotations. Let = h a X y be an outcome model for D with h Bayes-optimal. and assume that the representation transfers, i.e. P()(X)(Y) (X)(( for all {TP(Y = 0)) 0}. Then the Causal Lifting constraint on P implies is valid on P.
=
Intuition: The theorem shows that Causal Lifting, supported by other transferability assumptions, guarantees to transfer validity among PPCI problems, regardless of potential shifts in the joint distribution of (Z, Y). e.g., different treatment effect. The core of the proof relies establishing y. Z z), given
hard representation transferabilliyy, i.e., P()(X) = y. Z z) =P (Ø( by the assumed representation transferability (on standalone Y) and the causal lifting constraint. This done within the expectation of the conditional calibration, which we have already shown implies validity. Note that, indeed, the (marginal) representation transferabilityy conditioning over Y alone, does not guarantee transferability conditioning over Z too, potentially due by systematic biases within the encoder (e.g. fully relrieving the outcome signal for certain subgroup and partially missing it for others).
Proof. First, let remark that if 3 P', then Bayes optimality is sufficient for validity. Therefore, we now only focus on the true generalization setting with @ + 3. By Lemma 2.1 is sufficient to show conditional calibration to show the outcome model causal validity. Then, by linearity of the expected value we aim to show:
Ey Y
Z a.s. Ex [b(o(X)) Z].
(34)
where Z = [T' W]. and W is valid adjustment set. By the tower rule, we can expand the RHS:
Ex(h(o(X)) |] Ey
Ex(m(o(x))
(35)
((Y.))
By assumptions by E EV, 2 Z: ((y,2)= Ex(h(d(X)) Y y, Ex(h(o(X)) Y = = E'x(((((X)) Y = =y] = -E'XEE' YY (x()) Y ==
(36)
(Casual Lifting constraint)
(37)
(representation transfers)
(38)
-Bayes-optima predictor)
(39)
= Ex EE'- (XX)
Y =
(representation transfers)
(40)
function of (X)
Ex Y X]
Y = yl
(suf ficiency)
(41)
Ex Ey
Y
X Y = y)
(similarity)
(42)
(law of iterated expectations)
(43)
Where with the expected value with superscript prime we refer to the expected value over the data distribution of the problem @", Substituting back <(Y, Z) we have the thesis.
25

---

B ISTAnt
In this Section, we describe in detail our procedure to test and achieve valid causal inferences on ISTAnt without human annotations but only complex measurements (video), relying on fine-tuning foundational model on our similar annotated experiment.
B.1 Training annotated experiment and data recording (ours)
We experiment very much like the ISTAnt experiment with triplets of worker ants (one treated focal ant, and two nestmate ants), following the step-by-step design described in their Appendix C [Cadei et 2024]. We recorded 5 batches of 9 simultaneously pen marking for the dish palettes positioning, producing 45 original videos, of which one had to be excluded for experimental problems, leaving 44 analyzable videos. Then, for each video, grooming events from the nestmate ants to the focal ant were annotated by single domain expert, and we focus on the 'or' events, i.e., "Is one of the nestmate ants grooming the focal one?", considering up two possible behavior change per second (similarly for ISTAnt). We used comparable experimental setup (i.e., camera set-up, random treatment assignment, etc.) except for the following, guarantceing invariant annotation mechanism, i.c., similar experiment.
replicates with similar background
Treatments: Whereas ISTAnt used two micro- micro-particle applications, our experimental treatments also constitute micro-particle application in two different treatments (n 15 each), but also one treatment completely free of micro-particles (control, = 14), all applied to the focal ant. The three treatments of the ants are visually indistinguishable, independent of micro-particle application.
Light conditions: We created lower-quality illumination of the nests by implementing ring of light around the experiment container, resulting in more inhomogeneous lighting and high-lux ("cold") light effect, compared to the light diffusion by milky plexiglass sheet proposed in the original experiment. Also, our ant nests had a higher rim from the focal plane where the ants were placed, causing some obscuring of ant observation along the walls. See a comparison of the filming set-up and an example the resulting recording in Figure We also considered slightly lower resolution, i.e., 700x700 pixels.
Longer Videos: Whereas ISTAnt annotated 10-minute-long videos, we here annotated 30-minute-long videos, even if the ant grooming activity generally decreases with time from the first exposure to new environment. Our videos were recorded at 30fps and analysed at 2fps, totaling 158 400 annotated frames in the 44 videos.
Other potential distribution shifts: Other sources of variations from the original experi- ment are:
Whereas ISTAnt used orange and blue color dots, we used yellow and blue.
Whereas in ISTAnt, grooming presence or absence was annotated for each frame, we here annotated single grooming event even if the ant stopped grooming for up to one second but then kept grooming after that, with no other behaviors being performed in between. This means that intermediate frames between grooming frames were also annotated as grooming despite the ant pausing its behavior. Such less exact grooming annotations are faster to perform for the human annotator.
The person performing annotation in this experiment was different from the annotators in the ISTAnt dataset, leading to some possible observer effects.
Let's observe that our work models the general pipeline in ecological experiments, where multiple experiment variants are recorded over time, e.g., upgrading the data acquisition technique, and we aim to generalize from a lower to higher quality similar experiment.
B.2 Training and analyses details
We considered the full dataset so recorded for finetuning a pre-trained Vision Transformer, i.e., [Rafford et al., 2021] DINOv2 [Oquab al. [023], as proposed Cadei et al. (2024) and we left ISTtnt for testing causal estimation performances relying artificial predictions. For each pre-trained encoder, we fine-tuned
ViT-B [Dosovitskiy et al. 2020]. VIT-L. (Zhai et al. 2023].
26

---

multi-laye perception head (2 hidden layers with 256 nodes each and ReLU activation) on top of its class token via Adam optimizer (3, 0.9, 82 = 0.9, 10 5) for ERM, vREx (finetuning the invariance constraint in (0.01, 0.1, 1, 10)) and DERM (ours) for 15 epochs and batch size 256. So, we fine-tuned the learning rates in [0.0005, 0.5), selecting the hest-performing hyper-parameters for each model-method, minimizing the Treatment Effect Bias the training sample, while guaranteeing good predictive performances, i.c., accuracy greater than 0.8, on a small validation set (1 000 random frames). We computed the ATE at the video level (aggregating the predictions per frame) via the AIPW estimator. We used XGBoost for the model outcome and estimated the propensity score via sample mean (constant) since the treatment assignments are randomized, i.e., RCT. For the outcome model, we consider the following experiment settings for controlling: experiment day, time of the day. batch, position in the batch, and annotator. We run all the analyses using 48GB of RAM, 20 CPU cores, and a single node GPU (NVIDIA GeForce RTX2080T1). The main bottleneck in the analysis is the feature extraction from the pre-trained Vision Transformers. We estimate 72 GPU hours to run the full analysis, despite given candidate outcome model already finetuned the standalone reffiction-poweeee causal inference component, excluding feature extraction the target experiment takes less than a GPU minute.
27

---

C
CausaIMNIST
To exhaustively validate our method, we replicated the comparison between DERM (our) enforcing the Causal Lifting constraint and ERM (baseline), vREx and IRM (invariant trainings) in a controlled setting, manipulating the MNIST dataset with coloring, allowing us to (i) cheaply replicate fictitious experiments several times, bootstrapping confidence intervals, and (ii) control the underlying causal effects (only empirically estimated in real-world experiments).
C.1 Data Generating Process
We considered the following training data distribution PA;
W Be(0.5)
(44)
= Be(0.02)
(45)
T Be(0.5)
(46)
= W (0,,((, + (n, + U. Uniff( (0, 1,2,3))
(47)
X (T, W, U,
(48)
representing a Randomized Controlled Trial where fx is deterministic manipulation of a random digit image nx from MNIST dataset enforcing the background color W (red or green) and pen color T (black or white) and padding size U (0 or 8). By exchangeability assumption:
ATE
E|y do 1)] E|Y ddo(T 1)]
E|y T 1] E 0]
(49)
1.5
Six examples of colored handwritten digits from MNUST are reported in Figure
2
3
2
Figure 4: Random samples from a CausalMNIST sample.
Then, we considered the generic similar target Observational Study, with distribution:
W = Be(pw)
(50)
U Be(pu)
(51)
T Be(0.1). (1- + Be(0.9) W
(52)
X = fx (T, W,Y,U,nx)
(53)
with linear outcome/effect (null):
== W. Unif( (0, 1,2 + Unif( (0, + U Unif( (0, 1,2,3)
(54)
where simply:
ATE E|Y do - = 1)] =
(55)
and non-linear outcome/effect:
Y (T Unif({0, 1,2,3)) + Unif (0, 1,2,3,4,5, 4,5, 6))
(56)
with respect to the experimental settings, where, by adjustment formula:
ATE = E| do = 1)] El do = 1)] = P(U 0) (E 1, = 0] E 0,U 0)
(57)
= (1 - pv) 1.5
28

---

and we considered 4 different instances IP Pe, PP and If varying the outcome model and experimental settings parameters. P and PC have null effect, while p and F have heterogeneous effect. pD and Ip are closer in distribution to the training since pv << 1, while pC and Te are more out-of distribution since the padding variable U balanced, i.e., py 0.5, but during training it is rarely observed activated. For simplicity we refer to p& as the distribution of the the target experiment population with linear effect and "soft" (experimental settings) shift, IPC with linear effect and "hard" shift, ip with non-linear effect and "soft" shift, pD with non-linear effect and "hard" shift. In Table 2 we summarize the different distribution parameters, also reporting the corresponding (exact) ATE value.
Table 2: Summary of the training and target experiment distributions.
OoD (linear effect)
OoD (non-linear effect)
In- In-Distribution
Soft shift
Hard shift
Soft shift
Hard shift
Distribution
p4
p
pC
pD
p
Pw
0.5
0.05
0.5
0.2
0.5
Pu
0.02
0.05
0.5
0.2
0.5
Randomized
True
False
False
False
False
Effect
Linear
Null
Null
Non Linear
Non Linear
ATE
1.5
0
1.2
0.75
These data generating processes represent fictitious experiments where we aim to quantify the effect of the pen color on the number draw (if asked to pick one), relying on machine learning model for handwritten-digits classification. Particularly, the shift in the treatment effect between training (ATE= 1.5) and target population (ATEE {0, 0.75, 1.2]), may be interpreted as testing a different, still homogeneous group of individuals, or (ii) varying the effect by changing the brand of the pens,
unobserved variable (perfectly retrievable by the pen color on the training distribution), while keeping
the same colors, reflecting the crucial challenges described in Section 2.
C.2 Additional Ablations: Causal Estimators
In addition to the main experiments on CausalMNIST we repeated all the experiments fine-tuning the predictive model via ERM and DERM, and varying the causal estimators. We particularly considered AIPW, X-Learner Künnzll et al. 2019). BART [Chipman et al. 2010). and Causal Forest [Wager and Athey 2018). Results are reported in Table As expected, the same biases observed with AIPW consistently confirmed by all the other estimators, confirming how biases in the prediction-powered
powered
samples invalidate any downstream analysis, regardless of the (causal) estimator (Cadei et 2024
Cadei and Internò 2025
Table ATE bias with standard deviation over 50 repetitions on prediction-poweere samples varying the causal estimators. Consistent biases are confirmed by different treatment effect estimators over the same prediction-powered sample.
In-Distrihution
OnD (linear effect)
OnD (non-linear effect) Soft shift
Method
Estimator
Soft shift
Hard shift
Hard shift
AIPW
0.00 0.02
0.86 + 0.14
1.05 + 0.15
0.64 + 0.18
().85 + 0.17
X-Learer
0.01 + 0.02
0,82 + 0.16
0.92 = 0.15
0.68 + 0.19
0.79 + 0.18
ERM
BART
0.01 + 0.02
0.81 + 0.19
0.89 = 0.13
0.59 + 0.21
0.75 + 0.16
Causal Forest
0.01 0.02
0.82 + 0.13
0.90 + 0.11
0.64 0.16
0.76 + 0.12
AIPW
0.10 0.07
0.14
0.75 - 0.05
0.08 + 0.27
0.45 0.12
X-Learner
0.32 0.27
0.00 + 0.25
0.69 = 0.12
0.05 + 0.27
0.53 0.14
DERM
BART
0.19 0.14
0.07 + 0.14
0.68 0.15
0.06 + 0.18
0.52 0.18
Causal Forest
0.22 0.19
0.11 + 0.13
0.67 + 0.09
0.03 + 0.24
0.50 0.15
29

---

C.3 Training and analyses details
We sampled 10 000 observations from PA to train digits classifier (a Convolutional Neural Network) and tested it in PPCI in-distribution (10 000 more sample from PA) and out -i--o---itributtion (zero- shot) on 10 000 obervations for each p pe, pP,, p We replicated the modeling choices for CausalMNIST proposed in Cadei et al. 2024 and described in their Appendix E.2 (without relying on pre-trained models). Particularly, the proposed network consists of two convolutional layers followed by two fully connected layers. The first convolutional layer applies 20 filters of size 5x5 with ReLU activation, followed by a 2x2 max-pooling layer. The second convolutional layer applies 50 filters of size 5x5 with ReLU activation, followed by another 2x2 max-pooling layer. The output feature maps are flattened and passed to a fully connected layer with 500 neurons and ReLU activation. The final fully connected layer reduces the output to ten logits (one per digit) which we apply softmax activation to model the probabilities directly. Table reports a full description of the training details for such network. Particularly we tuned the number of epochs in {10, 11, ..... 50} and learning rate in {0.01, 0.001, 0.0001, 0.000.00000 0.00001} by minimizing the Mean Squared Error on a small validation set (1 000 images) and finally retraining the model on the full training sample with the optimal parameters.
Table Training details for the Convolutaional Neural Network training on CausalMNIST.
Hyper-parameters
Value
Loss Learning Rate
Cross Entropy
0.0001
Optimizer Batch Size Epochs
Adam (31 = 0.9, [3 = 0.9, e = 80-8)
32
40
For each learning objective, i.e., DERM (ours), ERM, vREx, and IRM, we trained a model on the training sample, imputed the outcome in each target sample, and estimated the ATE via causal estimator (AIPW for main experiments, AIPW, X-Learner Causal Forest, and BART for the additional experiments). Within the AIPW and X-Learner estimator, we used the XGBoost regressor for the outcome regression and the XGBoost classifier to estimate the propensity score on observational studies (or vanilla sample mean randomized controlled trials since constant). We rely on default parameters for both Causal Forest and BART (with 10 trees) as suggested in the original papers. For both IRM and vREx, replicated the same prree-parameter tuning of the invariant coefficient from our experiments on ISTAnt generalization, selecting in both cases the invariant coefficient == 0.1. We repeated each experiment 50 times, including resampling the data, and bootstrapped the confidence interval of the ATE estimates. We run all the analysis using 10GB of RAM, 8 CPU cores, and single node GPU (NVIDIA GeForce RTX2080T1). The main bottleneck of each experiment is ve-generating a new version of CausalMNIST from MNIST dataset, and then training the model. The Prediction-Powerts ATE estimation significantly faster. We estimate total of 12 GPU hours to reproduce all the experiments described in this section.
30


Reviews:
[{"review_id": "opb6gn13s7", "reviewer": "Reviewer_xo43", "summary": "This paper proposes a new framework for Prediction-Powered Causal Inference (PPCI), which enables zero-shot estimation of treatment effects from high-dimensional, unlabeled data using pre-trained neural predictors. \n\nThe core idea is to reuse models trained on similar experiments to infer causal quantities in new experiments without additional labeling. \n\nThe authors identify conditional calibration as a key statistical property that such predictors must satisfy. However, standard empirical risk minimization (ERM) often fails to produce conditionally calibrated models due to spurious correlations in the data. To address this, the paper introduces the concept of Causal Lifting, a constraint that prevents outcome predictors from leveraging spurious cues related to treatment or experimental settings. They implement this constraint via a practical training method called DERM, which reweights training samples to eliminate correlations between experimental settings and outcomes.", "questions": "The paper does not compare with well-established causal effect estimation methods such as TARNet, CEVAE, BART, or X-learner, which may perform competitively under different assumptions. Could the authors justify this omission and clarify whether such methods are applicable or complementary in the context of zero-shot PPCI?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- The paper establishes rigorous conditions for when predictors yield valid causal estimates, notably through the conditional calibration property and the causal lifting constraint. These theoretical insights connect predictive modeling to causal identifiability.\n\n- The authors propose DERM as a simple and effective method to enforce the causal lifting constraint. It is well-motivated and practically implementable through reweighting based on outcome variance.\n\n- The paper reports the first valid zero-shot PPCI result on ISTAnt, showcasing a significant advancement over prior methods like ERM, IRM, and v-REx.\n\n- In Table 1, although DERM exhibits slightly higher error in the in-distribution setting compared to ERM and other baselines, it significantly outperforms them in OoD environments. This trade-off is intentional and aligned with the paper’s goal of enabling zero-shot causal inference across unseen experimental settings. The results validate that DERM effectively mitigates reliance on spurious correlations present in the source domain, which are often exploited by baselines to achieve lower in-distribution error but lead to invalid causal conclusions under distribution shifts. \n\nWeaknesses:\n\n1/ The framework relies on key assumptions: conditional calibration and the causal lifting constraint. These assumptions cannot be verified on the target dataset. This limits practical confidence in the causal validity of the estimates in real-world deployments.\n\n2/ From my understanding, the effectiveness of DERM depends on Eq. (9) (full support of outcomes across covariates). If this condition fails (as explored in the synthetic \"hard shift\" setting), DERM may still pick up spurious correlations, potentially harming causal estimation.\n\n3/ While comparisons are made with ERM, IRM, and v-REx, the paper does not evaluate against classical or modern causal effect estimation methods like TARNet, CEVAE, BART, or X-learner, which could offer strong baselines under different assumptions.\n\n4/ Typos (does not effect my evaluation):\n\nLine 309: convolutiona\n\nLine 306: experrimental\n\nTable 1 caption: (soft, hard shifts). we report", "was_revised": true, "final_justification": "Authors added more experiments to address my concerns.", "rebuttal": "We thank the reviewer @xo43 for the detailed feedback. Overall, the general message to “_connect predictive modeling to causal identifiability_”, detecting “_rigorous conditions_” and proposing a “_simple and effective method_”  seems delivered. The concerns relate to (i) assumptions test and justification, and (ii) causal model baselines. We comment on the former, while we disagree on the latter for factual points, and plan to adapt the final version of the manuscript with corresponding clarifications.\n\n(i) **Assumptions clarifications** (Weaknesses 1 and 2)\n\n- _Conditional Calibration_\n  - Conditional Calibration on the target distribution is not an assumption, but a sufficient condition for valid causal inference. It can be enforced and tested on the training distribution, and we show in Thm 3.1 under which assumptions it generalizes to a similar experiment even if out-of-distribution. \n\n- _Causal Lifting Constraint_\n  - The key assumption enabling Thm. 3.1 is that the Causal Lifting constraint holds on the target distribution. We can train and test whether this constraint holds on the training, but the reviewer is correct that, in general, it may not transfer. \n  - Between the two evils, we propose to enforce the Causal Lifting constraint and not directly Conditional Calibration. Indeed, a model optimized for standalone Conditional Calibration may still fail to transfer when dealing with new effect modifiers with similar appearance but different effects from the ones in the training experiment (see Example 2). This approach is similar in spirit to the considered Domain Generalization baselines, i.e., vREx and IRM, enforcing invariant prediction performances (ideally unbiased) varying the experiment settings. Instead, the Causal Lifting constraint robustly prevents such shortcuts directly in the representation space by ignoring any source of possible spurious correlations. \n  - Ultimately, it is true that the Causal Lifting constraint is not a testable assumption on the target distribution (at least zero-shot). However, we believe that the approach has merit as we demonstrated empirically just enforcing the Causal Lifting constraint on the training experiment and validly estimating the treatment effect on a similar target experiment with different treatments and recording platforms.\n\n- _Full Support Assumption_\n  - DERM is our practical implementation of the Causal Lifting constraint, by sample reweighting, relying on the full support of outcomes across covariates, i.e., Eq. 9. This is easy to test, e.g., check that the behaviour of interest is happening at least once for each experimental setting; still, we agree that there could be scenarios where such a support assumption is challenged/violated, so losing its guarantees (we acknowledge it in the Conclusion). In practice, DERM still outperforms ERM, IRM,vREx, even in such a regime (see Hard Shifts experiments on CausalMNIST), and the assumption is not violated in our real-world application.\n\n\n\n(ii)  **Comparison with other Causal Estimators** (Weakness 3)\n\n- The reviewer wonders if additionally benchmarking our pipeline for PPCI, i.e., causal lifting of foundational model and AIPW estimator, against standalone established causal effect estimators. However, it is not possible by design, since all the proposed causal estimators rely on the factual outcomes of the target experiment, which are not observed directly in PPCI, and only observed as raw measurements, e.g., pixels. More precisely:\n  - In XLearner, E[Y|T=t, W=w] cannot be estimated on the target experiment because Y is not observed. On the training population, the corresponding conditional expectation can be completely different, as the effect can vary.\n  - TARNet requires unavailable (target) factual outcome supervision for the network training. Possible variants, additionally relying on a pre-trained encoder, may be considered for fine-tuning on the training distribution (e.g., concatenating X and W) but without generalization guarantees (similar in spirit to our ERM baseline). \n  - Similarly, BART requires unavailable (target) factual outcome supervision for each regression. Even if trained on the training distribution, it has no guarantee to generalize to a target experiment with different causal effects, and anyway, it wouldn’t scale on complex data structures, e.g., images.\n  - CEVAE still requires unavailable (target) factual outcome supervision for training. Even if trained on the training distribution, it has no guarantee to generalize, and anyway, it is an overkill to assume unobserved confounders when they are actually observed (W) and potentially controlled too, e.g., RCT."}, {"review_id": "eXJc2yh4sB", "reviewer": "Reviewer_7mz6", "summary": "The paper studies zero-shot Prediction-Powered Causal Inference (PPCI), where a pretrained model predicts outcomes on a new experiment without any labeled data. It formalizes validity and introduces a novel “Causal Lifting” constraint, $I(\\phi(X), Z \\mid Y) = 0$ to ensure that representations carry no spurious signals. A practical instantiation - Deconfounded Empirical Risk Minimization reweights training samples to enforce lifting. Experiments on a replicated ISTAnt ecological dataset demonstrate that DERM is the only method to yield valid zero-shot ATE estimates across diverse pretrained backbones and shifts.", "questions": "Can the authors comment on my concerns?", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper tackles a timely, scientifically motivated problem via its formalization of PPCI and causal lifting constraints, while DERM offers a straightforward, model-agnostic recipe that achieves zero-shot ATE validity where baselines including ERM, IRM, etc fail. \n\nWeakness:\n\n- It is very difficult to assess whether the implementation/tuning of different methods are correctly aligned. In Fig 2, all failure cases for ERM, IRM and vRex are all from ViT-S and -L models. Is it a coincidence? I think it might be possible that there are confounding among the performance of different methods. For example, I noticed that the Adam optimizer uses beta1=0.9, beta2=0.9, which arguably is different from common optimal settings and can bias the performance of certain model-method pairs. Moreover, there are certain parameters that are not properly ablated (eg, training epochs, batchsize etc) . In my practical experience those parameters can result in big swings in final results (ie relative rankings) for causal inference algorithms and might induce significant confounding effect for evaluation. \n\n- The other minor thing is that the work relies heavily on the assumption that the annotation mechanism truly transfers between experiments - which is an unverifiable condition. Furthermore, not everyone agree that the prediction/annotation problem is an anti-causal problem. In fact some often think of X (recordings) and Y (human labels) as two different representation spaces projected from nature variables via causal mechanism and cognitive/measurement mechanism). In this causal model, it is a strong assumption that the full mechanism between X and Y can be transferred across environments.  \n\nBut in general I like this paper.", "was_revised": true, "final_justification": "After the rebuttal as well as looking at other reviewers comments, I decided to recommend borderline accept.", "rebuttal": "We thank the reviewer @7mz6 for the careful and hands-on feedback. Overall, the general message to “_tackle a timely, scientifically motivated problem via its formalization_” and “_offering a straightforward, model-agnostic_” solution seems delivered. Two concerns are raised, which we respond to here and plan to adapt the final version of the manuscript accordingly.\n\n**Model selection** (Weakness 1)\n\n- We considered the best-performing model hyperparameters (which encoder tokens to use, head architecture, batch-size, etc.) on ISTAnt according to the extensive ablation study in Cadei et al. (2024). These were the only arbitrary choices and were performed using ERM, so they may **at most favour the ERM** baseline, which still drastically fails. For each method (ERM, vREX, IRM, ours), we tuned the learning rate and invariance regularizer of IRM and vREX for enough epochs (15) to allow convergence. We performed a grid search, discarding hyperparameters that led to factual outcome prediction accuracy smaller than 0.8 on a validation set from the training experiment (1000 samples). Next, we selected for each method the hyperparameters leading to the smallest Treatment Effect Bias [Cadei et al., 2024] estimated on the full prediction-powered training experiment (i.e., no leaking from the target distribution). We plan to clarify these steps in the final version of the paper, in Appendix B.2.  \n- Adam's second moment coefficient is a **typo**, and we confirm we used instead default values ($\\beta_1=0.9$, $\\beta_2=0.999$), as shared in the code. \n- It is true that some encoders are more performant/confounded than others, also in agreement with the original paper from Cadei et al. (2024), yet our approach always succeeded in enforcing valid estimates. New benchmarks are required for a more exhaustive comparison among pre-trained models and inner causal entanglement evaluation, i.e., $I(\\phi(X), Z|Y)$, but this is out of the scope of this paper.  \n\n**Annotation Mechanism Assumption** (Weakness 2)\n- Annotation Mechanism invariance is **assumed anyway by domain (human) experts**, even without relying on artificial predictions. Taking our application as an example, experimental ecologists practically learn by experience how to recognize behaviours, and then assume it is transferable when they run new experiments and collect the same labels. Let’s further observe that in practice, we are never relying on their causal relationship, i.e., between the high-dimensional recordings (X) and the outcome behaviours (Y).\n\n\n[1] Cadei, Riccardo et al. “Smoke and Mirrors in Causal Downstream Tasks” NeurIPS 2024"}, {"review_id": "Hl5tN0PlIX", "reviewer": "Reviewer_pzYf", "summary": "The paper introduces Deconfounded Empirical Risk Minimization (DERM), a weighted extension of Empirical Risk Minimization designed to address confounding in treatment effect estimation when only high-dimensional covariates are available. DERM enables Prediction-Powered Causal Inference (PPCI), estimating treatment effects in unlabeled target experiments using predictions from pre-trained models. The method enforces a new constraint, Causal Lifting, to ensure causal validity transfers zero-shot across experiments. The authors provide both theoretical analysis and practical implementation, showing DERM outperforms standard ERM and invariant training on synthetic and real-world datasets.", "questions": "- Is “retrievable zero-shot from a pre-trained model” a predicate for the factual outcomes? What exactly is retrievable, and how?\n- What does “this problem” refer to in line 30? Why is it labeled a zero-shot problem before the pretrained model is introduced?\n- Can the authors elaborate on the source of bias in an RCT setting mentioned in lines 33–34?\n- Would Lemma 2.2 hold for any standard ATE estimator under the given assumptions, not just under \\(g\\)?\n- In the causal setting, would \\(X\\) be a parent of \\(Y\\) in a causal graph, or vice versa? Clarification would improve causal interpretation.\n- How is DERM defined given that no labels for \\(Y\\) are available? How are the weights \\(w_i\\) constructed in this case?\n- Is the definition of causal lifting introduced by the authors, or does it extend prior work?\n- Could the authors include Conditional Average Treatment Effect (CATE) results, given their relevance to ML applications?\n- Would a footnote on valid adjustment sets and identifiability improve clarity and completeness?", "limitations": "yes.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- The problem statement in Section 2 is excellent and clearly written.\n- The definition of causal lifting is neat, appealing, and well-presented.\n- The theoretical analysis, while relatively simple, is presented in a smooth and pleasant manner.\n- The empirical results are good, and the general seem to be highly relevant (I am still not getting the exact setting fully though).\n- The paper’s structure and intention become clearer as it progresses, particularly in the formal sections.\n\n## Weaknesses\n- The paper sometimes contain some overly complicated wording in my honest opinion.\n- “Retrievable zero-shot from a pre-trained model” – it's ambiguous whether this refers to factual outcomes.\n- “Powering causal inferences” – sounds awkward; “estimating treatment effects” is clearer.\n- “The interventional effect of a treatment” – overly complicated; simpler phrasing like “treatment effect” would help.\n- Lines 33–35 create confusion in my opinion, if no labels are available, how is training done at all?\n- “Conditional calibration” is mentioned multiple times without definition; to my knowledge this is not a standard term and needs at least a brief explanation before formally defining it later.\n- Lines 40–47 are dense and unclear. The discussion of domain shift and ERM bias is disconnected from the context of pretrained models.\n- Example 1 is abstract and could benefit from a more concrete illustration.\n- Example 2 uses vague terminology; more specific examples would clarify the scenario.\n- Some definitions (e.g., Definition 2.1) include conditions that may be unnecessary or confusing, such as tying identifiability of $Y$ to a property of $g$. \n- The introduction lacks the clarity and precision found in the formal sections, making it harder to follow early on.", "was_revised": true, "final_justification": "The authors addressed my questions. While some parts of the work are well presented, I still believe the paper needs more work to improve its clarity and include further results for the conditional setting to be more complete. Hence, I am keeping my assessment.", "rebuttal": "We thank reviewer @pzYf for the feedback. The methodological contribution of the paper seems properly delivered, while our **problem formulation and theoretical analysis** seems **ignored** as a contribution (no mention in reviewer’s summary and questioned Causal Lifting contribution) and **partially misinterpreted** (Zero-Shot meaning and implementation, Prediction-Powered Estimation Theorem) despite appreciated (“_excellent and clearly written_”, “_neat, appealing, and well-presented_”, “_smooth and pleasant_”, “_highly relevant_”). **No major concern** is raised, and weaknesses mostly revolve around unclear phrasing that we commit to improving. We address the raised questions in order, hoping to clarify potential misinterpretation:\n\n- Q1: _“Is ‘retrievable zero-shot from a pre-trained model’ a predicate for the factual outcomes? What exactly is retrievable, and how?”_\n\n  - Retrievable refers to the factual outcomes estimated by a predictive model. We agree it is not the best wording, and we will replace ‘‘_retrievable_” with “_predicted_”. \n\n- Q2: _“What does ‘this problem’ refer to in line 30? Why is it labeled a zero-shot problem before the pretrained model is introduced?”_\n  - Zero-shot refers to predicting the factual outcomes “_without training on the data from the target experiment_” (L35), transferring knowledge from a pre-trained model and similar training data (see Def. 2.3), while preserving the properties in the downstream causal estimation. \n\n- Q3: _“Can the authors elaborate on the source of bias in an RCT setting mentioned in lines 33–34?”_\n  - Biases in prediction-powered average treatment effect estimation can arise from different reasons, as discussed by Cadei et al. (2024). In L33-34, we refer to Lemma 3.1 from the same paper, describing how small biases in prediction (i.e., $1-\\epsilon$ accuracy with systematic errors in a subgroup) can invalidate the downstream causal estimation even in the simplest causal setting, i.e., a Randomized Controlled Trial.\n\n- Q4: _“Would Lemma 2.2 hold for any standard ATE estimator under the given assumptions, not just under (g)?”_\n  - We prove Thm. 2.2 only for AIPW, gold standard for ATE estimation on RCT (e.g., in our experiments on ISTAnt), and observational studies with suitable backdoor adjustment. It shows how the properties of a causal estimator, i.e., AIPW, are preserved by a conditional calibrated factual outcome model $g$. Similar results may be shown for other estimators. \n\n\n- Q5: _“In the causal setting, would (X) be a parent of (Y) in a causal graph, or vice versa? Clarification would improve causal interpretation.”_\n  - For ease of interpretation, we assumed the outcome ($Y$) as a cause of the high-dimensional observation ($X$), see L139. Intuitively, $Y$ is something that happens in the real world, and X is its measurement, e.g., video recording (changing $Y$ affects X, but changing the pixels in $X$ does not affect what happens in the experiment).  However, we also never explicitly rely on such an assumption, but rather on $P(Y|X)$ invariance.\n\n- Q6: _“How is DERM defined given that no labels for (Y) are available? How are the weights (w_i) constructed in this case?”_ and weakness _“lines 33–35 create confusion in my opinion,  if no labels are available, how is training done at all?”_\n  - DERM is the method we propose to enforce the Causal Lifting constraint by training (finetuning) a model on annotated data from similar experiments (see Def. 2.3), generally out-of-distribution for the non-annotated target experiment (by design). It reflects a realistic and common setup for the type of applications we target, e.g., ecologists study ants' behaviors across sequential similar experiments. In practice, it estimates the weights $w_i$ and operates on such annotated similar experiments, attempting to transfer causally valid predictions to the target experiment.\n\n- Q7: _“Is the definition of causal lifting introduced by the authors, or does it extend prior work?”_\n  - Yes, the definition is new, and it answers the challenge raised by Cadei et al. (2024) to estimate the treatment effect on the predictions of pre-trained models. Indeed, without further assumptions, a pre-trained model is generally biased, e.g., under-performing in a specific subgroup, and we introduce the expression Causal Lifting to refer to the procedure of correcting such biases. \n\n- Q8: _“Could the authors include Conditional Average Treatment Effect (CATE) results, given their relevance to ML applications?”_\n  - To the best of our knowledge, ISTAnt by Cadei et al. (2024) is still the unique benchmark for PPCI on high-dimensional measurements, and only evaluates ATE estimation. Through this paper, we have already contributed to it by introducing a similar real experiment for training (simulating a realistic PPCI pipeline). We agree on the relevance of extending PPCI benchmarks to heterogeneous effects, despite being outside the scope of this paper.\n\n- Q9: _“Would a footnote on valid adjustment sets and identifiability improve clarity and completeness?”_\n  - Fair point, we are happy to stress this further. We are indeed assuming that the causal estimand is statistically identifiable. If that is not the case, then even with ground truth Y, the causal estimation is ill-posed. In our experiment, this is ruled out by the fact that we consider randomized trials, which are one of the many possible ways to guarantee identification (one could have instruments, or feasible backdoor adjustments).  \n​​\n\nThe reviewer suggested a few **other minor wording improvements**, which are listed as the paper's weaknesses. Here is what we suggest:\n\n  - We will replace “_Retrievable zero-shot_” with “_zero-shot predictions_”.\n  - We will replace “_Powering causal inferences_” with “_prediction powered treatment effect estimation_”, which more precisely follows established Prediction Powered Inference terminology in statistics [Anastasios N. et al. 2023].\n  - We will avoid mentioning “_Conditional Calibration_” before its definition in Definition 2.2, unless to explicitly claim it as a novel contribution and point to the definition. \n  - In Definition 2.1 we assume that the causal estimand is statistically identifiable in $Y$ and define that a predictor for $Y$ is valid only if the causal estimand is also identifiable in $g(X)$. This definition ties the identifiability of $g(X)$ to that of $Y$, not viceversa. We’ll make sure this is clearer in the draft using the footnote on identifiability suggested in Q9.\n  - We thank the reviewer for the other specific suggestions where clarity can be improved. We will take this into account in the final revision and improve the presentation.\n\n[1] Cadei, Riccardo et al. “Smoke and Mirrors in Causal Downstream Tasks” NeurIPS 2024\n\n[2] Angelopoulos, Anastasios N. et al. “Prediction-Powered Inference” Science 2023"}, {"review_id": "YLnmdOdSDU", "reviewer": "Reviewer_4Lsh", "summary": "This paper addresses the problem of zero-shot causal inference from high-dimensional experimental data—such as images or videos—where ground truth outcomes are unavailable in the target experiment. The authors formalize this setting as Prediction-Powered Causal Inference (PPCI) and study conditions under which a model trained on a different but similar experiment can be used to estimate causal effects, such as the Average Treatment Effect (ATE), without further fine-tuning or annotation.\n\nTo support valid zero-shot PPCI, the paper introduces a constraint called causal lifting, which requires the learned representation phi(X) to be independent of experimental settings Z = [T, W] given the outcome Y, i.e., I(phi(X); Z | Y) = 0. The authors propose a simple, model-agnostic implementation of this constraint via Deconfounded Empirical Risk Minimization (DERM), which reweights training data to simulate a distribution where outcome and covariates are uncorrelated.\n\nEmpirical results demonstrate that DERM enables valid zero-shot ATE estimation on a new variant of the ISTAnt ecological dataset, where prior methods fail. The authors further validate their method on a controlled synthetic benchmark (CausalMNIST), showing robustness across several types of distribution shifts.", "questions": "* Would it help to give a more detailed discussion of when DERM fails, particularly in cases of limited support or latent confounding?\n* Would it be possible to connect the causal lifting constraint more directly to existing literature on conditional independence in representation learning, to clarify what is truly new?\n* Would it help to give a brief comparison or mention of adversarial methods or information bottleneck approaches for enforcing independence? This might provide useful context.", "limitations": "The authors have mentioned a possible acceleration of automated science as a potential societal side effect. They could also mention the drawback of the work being used inappropriately, for example, when attempting to apply it in a context where latent confounding is a genuine issue.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "This paper addresses a practical problem—zero-shot causal effect estimation from high-dimensional observations—by proposing a representation-level constraint (“causal lifting”) and a practical implementation (DERM). The authors demonstrate the approach on a real ecological experiment and a synthetic benchmark, showing improved generalization in settings where standard ERM and invariant training methods fail. The paper is clearly written, well-motivated, and methodologically sound.\n\nStrengths:\n* The problem of prediction-powered causal inference (PPCI) is well-framed and highly relevant for scientific domains where annotation is expensive.\n* The causal lifting constraint, while not novel in its form, is a well-chosen and effective principle for enforcing robustness.\n* The DERM procedure is simple, implementable, and grounded in sound intuitions from the representation learning and fairness literature.\n* The procedures deal effectively with scenarios where latent confounding is not an issue, but selection bias may be.\n* The experimental results are strong, including a new ISTAnt-like dataset and a successful zero-shot transfer.\n* The paper is careful in stating assumptions, and the theoretical claims are well supported.\n\nWeaknesses / Limitations:\n* The causal lifting constraint amounts to a conditional independence assumption (i.e., I(\\phi(X); Z \\mid Y) = 0), which is well-known in prior literature. The framing is helpful, but the novelty is conceptual rather than technical.\n* The approach assumes no latent confounding, which may limit its applicability in real-world settings. This assumption is acknowledged but perhaps not emphasized enough.\n* DERM depends critically on a full-support condition (i.e., \\text{supp}(Y \\mid Z=z) = \\text{supp}(Y) for all z) to simulate outcome–covariate independence. The failure of this condition can undermine the procedure, and this could be discussed more explicitly.\n* The method may be sensitive to variance in the reweighting procedure when sample sizes are small or the covariates are unbalanced.", "was_revised": true, "final_justification": "The authors have made inreads into clarity, though I was already happy with the paper. I keep my score.", "rebuttal": "We thank the reviewer @4Lsh for the curated feedback, which summary and strengths/weaknesses show the in-depth understanding and appreciation of the paper. We answer here to the raised questions and plan to integrate such clarifications in the final version of the manuscript, together with the suggested limitations emphasis.\n\n**Causal Lifting constraint: novelty and related works** (Weakness 1, Questions 2-3)\n- In this paper, we characterize which properties a (factual outcome) predictor requires to be incorporated in a causal inference pipeline without training on the target population. Alongside, we also propose a methodology for achieving them, relying on similar data. Particularly, we show that enforcing conditional independence between the measurement representation and the experimental settings, given the outcome, transfers Causal Validity among similar experiments (Thm. 3.1). The novelty lies in such formulation, which implementation happens to be a well-studied problem in Representation Learning for other reasons. We reviewed this in the Invariant Representation paragraphs in Section 4. \n- Methodologically, we chose a specific implementation of the invariance constraint, which is far simpler than adversarial [Louizos et al. 2016] or information bottleneck [Alemi et al. 2017] approaches which are known to be unstable and inefficient respectively. We are happy to expand in Section 4 on the advantages of the different approaches and why we chose DERM for our particular application, see next.\n\n\n**DERM Limitations**: full support and finite sample setting (Weaknesses 3-4, Question 1)\n- We practically designed our ad-hoc implementation, motivated by our application of interest (high-dimensional representation, with discrete and limited covariates), but other approaches could and should be considered depending on the problem characteristics. As recognised in the limitations, DERM has no guarantees beyond the full support assumption (as shown with Hard shifts on CausalMNIST experiments), and new methods, tailored for such a regime, or just a strongly unbalanced distribution, surely deserve to be investigated. \n\n**Latent confounding** (Weakness 2, Question 1)\n- Note that we are assuming that the causal estimand is statistically identifiable. If that is not the case, then even with ground truth $Y$, the causal estimation is ill-posed. In our real-world experiment, this is ruled out by the fact that we consider randomized trials, which are one of the many possible ways to guarantee identification. \n\n[1] Louizos, Christos et al. \"The Variational Fair Autoencoder\" ICLR 2016\n\n[2] Alemi, Alexander A. et al. \"Deep variational information bottleneck\" ICLR 2017"}, {"review_id": "vX0JO18IP7", "reviewer": "Reviewer_N6pk", "summary": "This paper proposes an idea of estimating causal effects from high dimensional observation without labels. Then the paper discusses the desiderata of a model to estimate causal effects. Based on the analysis, the paper proposes a method called deconfounded empirical risk minimization and shows improved performance over several baseline methods for estimation causal effects.", "questions": "Please see the weaknesses section and address those points, especially the points on the validity of assumptions and the underlying causal model.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 1, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThe idea of estimating causal effects from high dimensional measurements without explicit label for the observed data is novel and helps the community to solve the causal effect estimation task innovatively. Experimental results show the promising results of the proposed method compared to baselines.\n\nWeaknesses:\n\nThe major weakness of this paper is its readability. I had to read several times to understand the goal of the paper. \n1. Here are some issues in the introduction.  \n      (a) The first paragraph reads like the paper is about to solve an explainability problem.\n\n      (b) In L22, it is not clear what it means to \"analyzing experimental data\" by a trained model.\n\n      (c) In L29, it is not clear what \"outcome predictions\" means. outcomes and predictions are usually the same.\n\n      (d) A real-world example in the introduction would greatly help its readability.\n\n2. In section 2, having a causal graph to model the setting and assumptions is required to understand the setting clearly. \n3. $\\mathcal{I}$ is not defined in L77. When the ISTAnt task is introduced in section 2, it is not clear what the goal of the task is. The full form of AIPW is never presented.\n4. Grammatical erros and typos: L97: \"identify in a statistical estimated\", L305: \"convolutiona\"\n5. Inconsistencies: $W$ in L101 and $\\tilde{W}$ in 174 are used to denote the same thing i.e., valid adjustment set.\n6. Explaining conditional calibration and causal lifting with real-world examples would help understand the paper better.\n7. (Important) The standard causal identification assumptions rarely hold in real world data. For instance, what measures are taken to ensure that there is no hidden/unobserved confounding variables? This will subsequently influence the performance of the proposed deconfounded empirical risk minimization algorithm.", "was_revised": true, "final_justification": "Assuming that the authors will improve the paper's presentation and readability in the next revision, I am leaning towards positive score.", "rebuttal": "We thank the reviewer @N6pk for the feedback. Overall, the general objective and purpose of the paper seem delivered and appreciated (“_novel idea_”, “_promising results_”), even if the raised concerns suggest a **potential misunderstanding** of the **prediction-powered setting** and **related literature**. We clarify here.\n\n**Causal Model Assumption** (Weakness 2)\n- The PPCI formulation does _not require_ introducing a causal model, whose measurement-annotation relationship could also be controversial, as suggested by reviewer **`@7mz6`**. It simply relies on extending a well-defined treatment effect estimation problem (with its local causal assumptions), replacing the ground-truth (factual) outcomes with model predictions from high-dimensional predictive measurements.\n\n**Latent confounding** (Weakness 7)\n- Note that we are *assuming* that the causal estimand is statistically *identifiable*. If that is not the case, then even with ground truth $Y$, the causal estimation is ill-posed and so it doesn’t make much sense to investigate the Prediction-Powered extension of it. \nIn our experiment, this is ruled out by the fact that we consider randomized trials, which are one of the many possible ways to guarantee identification (one could have instruments, or feasible backdoor adjustments).  \n\nThe reviewer raised a number of valuable **minor concerns** and clarification requests that we now address, and we plan to adapt in the final version of the manuscript, hoping to **improve** its **clarity** accordingly.\n\n- *Introduction* (Weakness 1)\n  - The first paragraph in the introduction *doesn’t refer to explainability*, if not for the expression “_black-box_” referring to a model with generally no guarantees as to how it reaches its predictions, especially required in scientific applications. We are happy to make this clearer.\n  - Given the *2-stage* nature of the *problem*, i.e., factual outcome estimation and causal effect estimation, we need to specify the object of prediction.\n  - A real-world problem (zero-shot generalization to ISTAnt) is used as a *motivating example* from the introduction to the conclusion to support problem interpretation, additionally illustrated in Fig. 1-3 and Example 1-2. As suggested, we plan to extend the dataset presentation in the introduction.\n- *Typos* (Weaknesses 3a, 4 , 5). \n  - We will replace $\\mathcal{I}$ with $i$ ranging from $1$ to $n$. We thank the reviewer for spotting it, together with the *other typos*.\n- *Task Clarification* (Weakness 3b, 3c)\n  - ISTAnt *task is* explicitly *presented* in L89-90, “_ATE estimation on the ISTAnt dataset zero-shot, i.e., without behaviour annotations. Instead, we leverage our own experiment variant with different treatment and recording platform_” and additionally illustrated in Fig. 1. After extending the example presentation in the introduction, we hope this will be clearer.\n  - AIPW [Robins et al. 1994] is the *gold standard* for ATE estimation and is assumed to be known according to common practices in the field.  \n- *Clarification by Real-world Example* (Weakness 6)\n  - The Casual Lifting constraint is *already explained* by our motivating example (see L182-189), while the Conditional Calibration interpretation is self-explicatory, still interpreted several times, e.g., in Example 1 (see L149-152).\n\n[1] J.M. Robins et al, Estimation of regression coefficients when some regressors are not always observed, JASA 1994."}]

Author rebuttal:

Meta-review: This paper introduces a new framework for Prediction-Powered Causal Inference (PPCI), which uses pre-trained neural networks to estimate treatment effects from high-dimensional, unlabeled data. The core concept is to reuse models from similar experiments to infer causal relationships in new, unlabeled datasets. The authors identify conditional calibration as a crucial statistical property for these predictors. However, standard training methods often fail to achieve this due to spurious correlations in the data. To solve this, the paper proposes Causal Lifting, a constraint that prevents outcome predictors from relying on these spurious cues. They implement this constraint using DERM, a practical training method that reweights training samples to eliminate correlations between experimental settings and outcomes.

The reviews are overall positive, with all reviewers leaning toward acceptance after the author-reviewer discussion phase. However, some reviewers still found the paper difficult to read and understand, suggesting a lack of clarity. 

Given its novelty and technical contributions, I recommend its acceptance for publication at NeurIPS as a poster. I encourage the authors to further improve the clarity of presentation in the camera-ready version.
Decision: Accept (poster)