Title: Performative Risk Control: Calibrating Models for Reliable Deployment under Performativity
Venue: NeurIPS 2025
Accepted: yes
Keywords: risk control

performative prediction

statistical learning theory
Abstract: Calibrating blackbox machine learning models to achieve risk control is crucial to ensure reliable decision-making. A rich line of literature has been studying how to calibrate a model so that its predictions satisfy explicit finite-sample statistical guarantees under a _fixed_, _static_, and unknown data-generating distribution. However, prediction-supported decisions may influence the outcome they aim to predict, a phenomenon named _performativity_ of predictions, which is commonly seen in social science and economics. In this paper, we introduce _Performative Risk Control_, a framework to calibrate models to achieve risk control under performativity with provable theoretical guarantees. Specifically, we provide an iteratively refined calibration process, where we ensure the predictions are improved and risk-controlled throughout the process. We also study different types of risk measures and choices of tail bounds. Lastly, we demonstrate the effectiveness of our framework by numerical experiments on the task of predicting credit default risk. To the best of our knowledge, this work is the first one to study statistically rigorous risk control under performativity, which will serve as an important safeguard against a wide range of strategic manipulation in decision-making processes.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new framework to calibrate blackbox prediction models in environments where predictions influence future data distributions—a setting known as performativity. Building on recent work in risk control and conformal prediction, the authors extend these methods to account for performativity. They develop an iterative algorithm that selects a sequence of decision thresholds $\\lambda_t$, guaranteeing both finite-sample statistical risk control and eventual tightness around a user-specified target risk level $\\ Delta\\alpha$, despite the distributional shifts induced by previous deployments. The approach is demonstrated through experiments in credit risk scoring, under both expected risk and quantile-based risk measures like CVaR.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths:  \n\nThe paper addresses a crucial gap in the risk control literature—how to ensure safe deployment in settings where predictions causally influence future data. This is particularly relevant in real-world socio-technical systems like credit scoring or policy-making. The extension to quantile-based risk measures like CVaR is also valuable for applications that care about tail risk rather than average-case loss. \n\n\n## Weaknesses:\n\nThe major weakness of the paper is the monotonicity assumption which greatly simplify the complexity of the problem. The paper mentions a trade-off between “safety” and “utility” but doesn't clearly define what utility means or how to measure it in practice. The tumor segmentation example used to motivate the method is difficult to interpret in a performative setting. It’s unclear how prediction set size (controlled by $\\lambda$) influences future data distribution. In addition, it's a bit unclear how the method is specifically designed to account for \"performativity\" -- the proposed method seems to work for any case where the distribution is smooth and doesn't change dramatically? See more details in the question section.\n\nQUESTIONS:\n1. Monotonicity Assumption: The assumption that $\\ell(z;λ)$ is non-increasing in $\\lambda$ seems restrictive and significantly simplifies the problem. It effectively encodes a unidirectional notion of risk, implying that the decision-maker is only penalized for type II errors (e.g., false negatives), and that increasing conservativeness always reduces loss. This overlooks settings where there are different types of errors where non-monotonic losses arise, and make the search for an optimal threshold $\\lambda$ much easier. \n\n2. Motivation for loosening the threshold (Section 2.2): The rationale for choosing a less conservative (i.e., smaller) threshold $\\lambda$ is stated in terms of \"preserving utility\" and \"useful class information,\"  (line 97) but these terms are not clearly defined. It would be helpful to clarify what utility means in this context -- what constitutes \"useful class information\" (which seems to be not captured by the loss $\\ell(z; \\lambda)$?) More broadly, a more detailed discussion of the trade-off between safety (risk control) and utility would strengthen the motivation behind relaxing the threshold.\n\n3. Tumor segmentation example: I find this example difficult to interpret. My understanding is that $\\lambda$ represents a deployment parameter that influences both (a) the prediction set size (e.g., the number of pixels labeled as tumor), and (b) the data distribution in the next round via performative feedback. However, it is unclear how the induced distribution $D(\\lambda)$ should be interpreted in this context. Does a larger $\\lambda$ simply produce more inclusive prediction sets, or does it also affect how future data (e.g., future tumor scans or labels) are generated? A more detailed explanation of how performativity manifests in the tumor segmentation setting would help clarify this.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces performative risk control, a framework that post processes model predictions to achieve provable risk control under performativity. The proposed procedure is an iterative calibration process, and is shown to be safe at anytime and tight when the performative distribution $D(\\lambda)$ is Lipschitz continuous wrt to $\\lambda$ and a valid confidence width $c(n,\\delta)$ (which can be obtained via e.g., Bernstein's inequality or CLT) is provided. Experiments on the Kaggle credit scoring dataset demonstrate the effectiveness of the proposed framework in predicting credit default risk.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is well-written. The assumptions and theorems are clearly stated and the consequences and limitation are discussed. This is the first work that proposes a provable risk control method when the data exhibit performativity. \n\nOn the other hand, the proposed method seems to be a straightforward generalization of that in \\[1] for i.i.d. data to the setting where performativity exists, since for each $\\widehat\\lambda_t$, the authors assume access to $n$ i.i.d. samples from $D(\\widehat\\lambda_t)$. The technical contribution is less clear.\n\n[1]. Bates, S., Angelopoulos, A., Lei, L., Malik, J., & Jordan, M. (2021). Distribution-free, risk-controlling prediction sets. Journal of the ACM (JACM), 68(6), 1-34.\n\nQUESTIONS:\n1. In the experiments on the Kaggle credit scoring dataset, the authors assume the performativity---dependent on $\\lambda$ ---affects the data distribution through impacting the prediction function $f(x(\\lambda))$ in a specific way given at the bottom of page 8. I wonder if the theoretical guarantees hold only when the Kaggle dataset satisfies this assumption, and what would happen if the assumption is violated.\n\nLIMITATIONS:\n1. The PRC method requires access to i.i.d. samples from $D(\\widehat{\\lambda}_t)$ for each $t \\in [T]$. In practice, collecting such data can be expensive and time-consuming, as it involves deploying $T$ models sequentially and gathering $n$ data points from each model before proceeding to the next. The paper would benefit from a discussion of this limitation and potential strategies for mitigation.\n\n2. As mentioned before, it would be beneficial for the authors to clarify the technical novelty of this work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a procedure for risk control in performative settings. The basic idea is to combine the model and assumptions from performative prediction with conformal risk control to obtain a method that controls risk while allowing for performative distribution shifts. The procedure is shown to be valid and it is evaluated in a semi-synthetic credit scoring experiment.\n\nSTRENGTHS AND WEAKNESSES:\nStudying risk control under performativity is interesting and novel to the best of my knowledge. The paper gives well-motivated example setting, such as the lending problem. Given that this is a new problem setting, I wish there was more high-level discussion on what are the desiderata in this problem (e.g. we probably care about what the induced distribution looks like). In general, more conceptual discussions would help the exposition. The procedure (Alg. 1) makes sense, though it would be nice to give some instantiations where some steps simplify (e.g. step 2).\n\nLine 102: This should be with high probability.\nLine 118: You say: \"Notice that the calibration dataset ... in Def. 1.1 is the union of...\" This is not stated in Def. 1.1.\n\nQUESTIONS:\nSuppose $D(\\lambda)$ comes from some known parametric model, or we have some other knowledge of the distribution shifts. How would your procedure change in that case? Can you give a more powerful version of the method?\n\nHow can you ensure $\\Delta\\alpha$ tightness for any value? I'm thinking of a setting where the risk function is discontinuous and has sharp jumps. In that case you shouldn't be able to get arbitrarily close to $\\alpha$?\n\nCan you comment more on the role of $\\tilde T$? What would happen if we didn't have the condition $\\Delta\\lambda \\geq (\\lambda_{\\mathrm{safe}} - \\lambda_{\\min}) / \\tilde T$?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies risk control in the performative setting, e.g. where individuals/organizations/datapoints will strategically attempt to manipulate their features so as to influence risk control variables such as the size of the output set that the model assigns to them. Specifically, the authors adapt a standard iterative approach to performative prediction for risk control, which they refer to as Performative Risk Control. The algorithm follows a common iterative approach to performative prediction: in this case, starting with a conservative threshold, collecting new data from the distribution induced by the previous threshold, building an upper confidence bound on the new threshold (accounting for potential distribution shift, and assuming smoothness as is standard), and repeating. The proof of the main theorem arguing the convergence of the method (which is entirely deferred to the Appendix without even a sketch) implements the usual performative prediction proof approach by adapting UCB to select updated thresholds; the convergence rates and assumptions match exactly what one would expect from standard performative prediction results.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The authors argue for an interesting problem and while the paper could better motivate performative risk control, it's not hard to see why this may be of interest (e.g. why a suspect may hope that a judge forms a wider uncertainty set for guilt).\n- The actual theoretical result (which is the main contribution) largely follows a standard recipe for adapting a learning problem to the performative setting but its nice to see the details worked through; the form of the algorithm, assumption, and rates appear reasonable.\n\nWeaknesses:\n- The current discussion for motivating performative risk control leaves something to be desired; see questions. This is important because, while the theorem proof isn't very involved and mainly just follows a standard recipe, the paper could still have a meaningful contribution in arguing for the study of performative prediction + conformal provided the problem is better motivated.\n- Writing can be generally improved, especially in technical parts. For example, the statement of Theorem 3.3 is a hard to read: delta alpha isn't defined, you need to scan up to L163 to read it instead; same thing with Algorithm 1 (esp. Line 2). I don't believe that the lack of clarity is unavoidable (the algorithm is not too complicated, though the authors may find it cleaner to reparameterize $\\Delta \\alpha$ and $c(\\cdot)$ ?), just that the writing just needs more polishing.\n\nComments:\n- It may be worth editing Figure 2 so that tau = 0.001 is labeled as the non-performative algorithm (or more precisely to include one where you exactly ablate the performative aspect with tau = 0). Given the way the experiments are setup (distribution shift is synthetically added exactly so that these types of performative prediction algorithms work well), the result of the experiment seems somewhat expected, but it's still nice to have experimental validation that the paper's implementation for risk control does work.\n- While the existing related work discussion covers performative prediction fairly thoroughly, the authors may consider also including discussion on the connections to Stackelberg literature (though this is a point of broader friction between the two communities rather than a shortcoming of this paper).\n\nQUESTIONS:\nQuestions:\n- Addressing the weaknesses would be helpful in making me feel comfortable with a higher score.\n- Normally, performative prediction is motivated by individuals/organizations that want to bias their own prediction in a certain direction. However, risk control is normally applied to higher order statistics, most commonly uncertainty set size. The degree to which individuals/organizations are incentivized to manipulate features to influence such stats is less clear; I don't doubt that such a motivation exists, but what are some concrete examples?\n- What does poly mean on L25?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-06"}