Title: Statistical Inference under Performativity
Venue: NeurIPS 2025
Accepted: yes
Keywords: statistical inference

performative prediction
Abstract: Performativity of predictions refers to the phenomenon where prediction-informed decisions influence the very targets they aim to predict—a dynamic commonly observed in policy-making, social sciences, and economics. In this paper, we initiate an end-to-end framework of statistical inference under performativity. Our contributions are twofold. First, we establish a central limit theorem for estimation and inference in the performative setting, enabling standard inferential tasks such as constructing confidence intervals and conducting hypothesis tests in policy-making contexts. Second, we leverage this central limit theorem to study prediction-powered inference (PPI) under performativity. This approach yields more precise estimates and tighter confidence regions for the model parameters (i.e., policies) of interest in performative prediction. We validate the effectiveness of our framework through numerical experiments. To the best of our knowledge, this is the first work to establish a complete statistical inference under performativity, introducing new challenges and inference settings that we believe will provide substantial value to policy-making, statistics, and machine learning.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors establish a central limit theorem for estimates of the performatively stable solution ($\\theta_{\\text{PS}}$) which are constructed using repeated risk minimization (RRM). Using this central limit theorem, the authors construct confidence regions for $\\theta_{\\text{PS}}$. Finally, the authors also consider a problem setting where prediction-powered inference (PPI) is used in the presence of performativity. The authors prove a similar central limit theorem and a corresponding estimate for the optimal weights between ``gold-standard\" labeled data and the annotated data.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n* The results on selecting optimal $\\{\\lambda_t\\}$ for performative PPI are interesting and potentially useful.\n* The paper develops asymptotic results for a generic $t \\in [T]$ as $n$ grows; this is distinct from prior work on asymptotics of performative estimates.\n\n**Weaknesses**\n\n* The authors are missing a comparison with the closely related work [1]; this work provides asymptotic results for the repeated stochastic gradient descent estimator of $\\theta_{\\text{PS}}$. The asymptotic results here are in the regime $n=1, T \\rightarrow \\infty$ so the results in the submission are distinct, but discussion of this work is still needed. For example, how are the proof techniques in this work novel to those in [1]? Also, what are the motivations for this asymptotic setting? It seems somewhat less realistic then what is considered in [1]. \n\n* The claim that 'the work initiates the study of inference in performativity' is not true; see, for example, [1], [2]. This is repeated throughout the paper and re-writes are needed to eliminate this.\n\n* The theory does not account for any optimization error in the risk minimization step of RRM\n\n* Use of the constructed confidence intervals requires a density estimation step where model $M(z, \\theta; \\psi)$ is used to estimate the performative distribution $p(z, \\theta)$. Furthermore, the theory requires an assumption of perfect convergence on the optimization over $\\psi$.\n\n* All experiments are on simulated data; I think this is an issue for the PPI experiments in particular. I am not aware of practical settings which both require the use of PPI and have performative aspects. The authors adding a data set where this is plausibly the case would greatly strengthen the results.\n\n**Minor comments**\n\n* 'Lastly,' repeated twice in the notation section\n* Line 53 'perforamtive'\n\n\n[1] Cutler et al. Stochastic Approximation with Decision-Dependent Distributions: Asymptotic Normality and Optimality. https://arxiv.org/abs/2207.04173\n\n[2] Bracale et al. Micro-foundation inference in strategic prediction. https://arxiv.org/abs/2411.08998\n\nQUESTIONS:\n* Can you provide non-synthetic experiments? How do the constructed confidence intervals perform on non-synthetic data?\n\nLIMITATIONS:\n* Consider adding a limitations section to the main text", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies how to perform statistical inference when a predictive model influences the data-generating process — a phenomenon known as *performativity*.  \nAt each round the learner deploys a model parameter \\\\( \\\\theta_t \\\\), new data \\\\( z_{t,i} \\\\sim D(\\\\theta_t) \\\\) are generated in response, and the learner updates the model by empirical risk minimisation.  \nUnder standard smoothness, convexity, and a Lipschitz condition linking \\\\( D(\\\\theta) \\\\) to \\\\( \\\\theta \\\\), the authors show that the sequence \\\\( \\\\theta_t \\\\) converges linearly to a unique performatively stable point \\\\( \\\\theta_{\\\\mathrm{PS}} \\\\).\n\nThe main theoretical contribution is a central limit theorem: for fixed \\\\( t \\\\) and sample size \\\\( n \\\\), one has  \n\\\\[\n\\\\sqrt{n}\\\\,(\\\\hat\\\\theta_t - \\\\theta_t) \\\\; \\\\xrightarrow{d} \\\\; \\\\mathcal{N}(0, V_t),\n\\\\]\nwhere the asymptotic covariance \\\\( V_t \\\\) depends on a product of Jacobians \\\\( \\\\nabla G(\\\\theta_k) \\\\) and a sandwich matrix built from the score of the unknown density \\\\( p(z, \\\\theta) \\\\).  \nBecause \\\\( p \\\\) is not observed, the authors propose estimating its score by fitting a flexible surrogate density \\\\( M(z, \\\\theta; \\\\psi) \\\\) with a score-matching objective augmented by small “policy perturbations” in \\\\( \\\\theta \\\\).  \nCombining the CLT with the geometric convergence of \\\\( \\\\theta_t \\\\) yields asymptotically exact confidence regions for the stable point \\\\( \\\\theta_{\\\\mathrm{PS}} \\\\).  \nThe framework is further extended with *prediction-powered inference*: abundant pseudo-labels are blended with scarce ground-truth labels to reduce variance, and a second CLT quantifies the resulting efficiency gains.  \nSimulations in a performative linear-regression setting confirm nominal coverage of the confidence sets and illustrate how the pseudo-label weight improves precision.\n\nSTRENGTHS AND WEAKNESSES:\nVery good results and answering important questions. Clearly presented. \nI commend the work and consider it a valuable contribution. However, I am giving a borderline accept due to concerns regarding the **precision of the variance estimates** (detailed below).  \nIn my view, this issue is central: if the estimates prove unreliable or highly sensitive in finite samples, the practical value of the results would be significantly diminished.  \nOn the other hand, if the variance estimation procedure is shown to perform well, either through theoretical guarantees or empirical validation, I would view the work as a very significant contribution to the literature and would be inclined to change my rating to **strong accept**.\n\nQUESTIONS:\n- **Meaning of \\\\( p(z, \\\\theta_k) \\\\).**  \n  I understand that \\\\( p(z, \\\\theta_k) \\\\) refers to the probability density (or mass) function of the distribution \\\\( D(\\\\theta_k) \\\\) from which the data at round \\\\( k \\\\) are drawn.  \n  A brief reminder of this definition in the main text might help readers.\n\n- **Missing of appendix.**  \n  I understand that not all assumptions and proofs can be included in the main text.  \n  However, the appendix containing these details is missing from the submission, which makes the paper difficult to understand on its own.\n\n- **Precision of the variance estimates.**  \n  Estimating \\\\( \\\\nabla G(\\\\theta_k) \\\\) and the covariance \\\\( V_t \\\\) requires approximating an intricate score function by the surrogate \\\\( M(z, \\\\theta, \\\\psi) \\\\).  \n  Because this involves sums and products over all past rounds, a discussion of finite-sample error — perhaps via non-asymptotic bounds or sensitivity analysis — would strengthen the practical relevance of the method.\n\n- **Dependence on the model class \\\\( \\\\mathcal{D}(\\\\tilde{\\\\theta}) \\\\).**  \n  The accuracy of the gradient estimates naturally depends on how well \\\\( M \\\\) can approximate the true family \\\\( \\\\{ \\\\mathcal{D}(\\\\theta) \\\\} \\\\).  \n  I would appreciate comments on how the complexity of this family affects approximation error, and whether model-selection guidance could be offered.\n\n- **Robustness to annotation error in experiments.**  \n  The numerical study assumes an annotator that is almost unbiased, with only a small mean shift.  \n  In many real applications, annotation models may be substantially misspecified.  \n  Exploring scenarios with larger mean or variance shifts, or other structural errors, would give readers a clearer sense of the method’s robustness.\n\nLIMITATIONS:\nThis is the same the last three questions in the questions \n\n- **Precision of the variance estimates.**  \n  Estimating \\\\( \\\\nabla G(\\\\theta_k) \\\\) and the covariance \\\\( V_t \\\\) requires approximating an intricate score function by the surrogate \\\\( M(z, \\\\theta, \\\\psi) \\\\).  \n  Because this involves sums and products over all past rounds, a discussion of finite-sample error — perhaps via non-asymptotic bounds or sensitivity analysis — would strengthen the practical relevance of the method.\n\n- **Dependence on the model class \\\\( \\\\mathcal{D}(\\\\tilde{\\\\theta}) \\\\).**  \n  The accuracy of the gradient estimates naturally depends on how well \\\\( M \\\\) can approximate the true family \\\\( \\\\{ \\\\mathcal{D}(\\\\theta) \\\\} \\\\).  \n  I would appreciate comments on how the complexity of this family affects approximation error, and whether model-selection guidance could be offered.\n\n- **Robustness to annotation error in experiments.**  \n  The numerical study assumes an annotator that is almost unbiased, with only a small mean shift.  \n  In many real applications, annotation models may be substantially misspecified.  \n  Exploring scenarios with larger mean or variance shifts, or other structural errors, would give readers a clearer sense of the method’s robustness.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper initiates the study of statistical inference under performativity, where prediction-informed decisions influence the outcome variable, leading to a feedback loop between predictive models and the data-generating process. The authors focus on the performative stable point and propose a framework for valid inference in this dynamic setting. The main contributions are (1) the derivation of a central limit theorem for estimators obtained through repeated risk minimization (RRM), enabling confidence region construction for the performative stable parameter, and (2) an extension of prediction-powered inference (PPI) methods to performative settings, leveraging unlabeled data and machine learning predictions to obtain more efficient inference. Theoretical results are validated through synthetic experiments.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe paper tackles the statistical inference in performative prediction, which is relevant for domains where model-informed decisions recursively shape the data (e.g., economics, policy, and social sciences). The theory is solid.\n\nWeaknesses:\n\n1. Motivation: The motivation of this paper is unclear to me. A more realistic example where the methods could work will be helpful. Otherwise, it seems to be a simple combination of PPI and Performative prediction.\n\n2. Empirical Scope: The experimental validation is currently limited to synthetic linear models. Application to more complex or realistic settings (e.g., non-linear models or real-world feedback systems) would further strengthen the practical significance of the proposed methods. Real data application will also be important.\n\n3. Practical Implementation: The paper introduces novel estimation strategies (e.g., policy perturbation for score matching), but more details or discussion on computational aspects, tuning, or diagnostics for these methods would make the framework more accessible for practitioners.\n\nQUESTIONS:\n1. Would the framework extend to non-convex loss functions or to settings where D(θ) exhibits more complex dependence on θ?\n2. Are there diagnostics that practitioners can use to check whether the model is correctly capturing performativity effects in practice?\n\nLIMITATIONS:\nSee weakness.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}