Revisiting the Predictability of Performative, Social Events
Authors: Juan Carlos Perdomo
Venue: ICML 2025 poster (2025)
Keywords: performative prediction
online learning
multicalibration
outcome indistinguishability
Primary area: theory->learning_theory
TL;DR: we shed new light on old debates regarding the predictability of social events and performative outcomes

Abstract: Social predictions do not passively describe the future; they actively shape it. They inform actions and change individual expectations in ways that influence the likelihood of the predicted outcome. Given these dynamics, to what extent can social events be predicted? This question was discussed throughout the 20th century by authors like Merton, Morgenstern, Simon, and others who considered it a central issue in social science methodology. In this work, we provide a modern answer to this old problem. Using recent ideas from performative prediction and outcome indistinguishability, we establish that one can always efficiently predict social events accurately, regardless of how predictions influence data. While achievable, we also show that these predictions are often undesirable, highlighting the limitations of previous desiderata. We end with a discussion of various avenues forward.

Revisiting the Predictability of Performative, Social Events
Juan Carlos Perdomo
Abstract
the predictability of social events. Motivated by these issues, we revisit the following question:
Social predictions do not passively describe the future; they actively shape it. They inform actions and change individual expectations in ways that influence the likelihood of the predicted outcome. Given these dynamics, what extent can social events be predicted? This question was discussed throughout the 20th century by authors like Mer- ton, Morgenstern, Simon, and others who consid- ered it central issue in social science methodol- ogy. In this work, we provide a modern answer to this old problem. Using recent ideas from per- formative prediction and outcome indistinguisha- bility, we establish that one can always efficiently predict social events accurately, regardless of how predictions influence data. While achievable, we also show that these predictions are often unde- sirable, highlighting the limitations of previous desiderata. We end with a discussion of various avenues forward.
Is it possible to efficiently find predictor that actively influences the likelihood of observed events, yet still produces valid predictions?
During the 20th century, multiple researchers recognized that social predictions shape social patterns and posed this question in their work. To give few examples, in his 1928 thesis, Oskar Morgenstern argued that accurate economic forecasts are generally impossible since public predictions can be self-negating (Morgenstern, 1928). Later, Simon (1954), as well as Grunberg & Modigliani (1954), used recently popularized fixed point theorems from topology to make progress on this problem and proved the existence of predictions that influence outcomes and simultancously calibrated. However, they did not address the al algorithmic question of how one might actually find these predictors efficiently (both computationally and statistically speaking).
In this work, we revisit this question using modern math- ematical tools. Framing our analysis in the language of performative prediction a recent learning-theoretie frame- work introduced by Perdomo et al. (2020) that formalizes the causal aspects of social prediction we establish the following result. One can always efficiently find predic- tions that actively shape the data distribution over outcomes and simultaneously satisfy rigorous validity guarantees like calti-calibratiion (Hébert-Johnson et al., 2018) or outcome indistinguishabiiity (Dwork et al., 2021). Moreover, the statistical and computational complexity of finding these predictors is, in many cases, just as easy in supervised learning, where the distribution is static.
1. Introduction
Social predictions do not passively describe the future; they actively shape it. They inform actions and change expec- tations ways that influence the likelihood predicted events. For instance, economic forecasts influence market prices, election predictions influence voter turnout, and cli- mate forecasts shape policies that impact weather patterns.
This dynamic, where predictions are performative and shape data distributions. is pervasive throughout social prediction. is also a feature, not a bug. The overarching goal of building systems that make predictions about people to inform actions that drive positive changes in the world. For example, we predict health outcomes to cure disease and poverty alleviate it. While desirable, this dynamic also introduces methodological challenges and casts doubt on
However, while these rigorously calibrated predictions are always achievable, they are not always desirable. The fact that predictions are performative and influence the likeli- hood of future events invalidates traditional solution con- cepts inherited from supervised learning like calibration. Through simple examples, we show that these prediction rules can lead to poor equilibria where predictions exactly match the conditional distribution over outcomes and hence are perfectly multicalibrated while simultancously explaining none of the variance in outcomes.
'Departeent of Computer Science, Harvard University, Boston MA, USA. Correspondence to: Juan Carlos Perdomo <jeper- domo@ hdd...
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Revisiting the Predictabiliiyy of Performative, Social Events
1.1. Overview of Technical Results
overall. If C consists of all bounded, measurable functions, then f(x) must be equal to the conditional distribution over outcomes, f(x) = [xe]]lzz for all z. We can interpolate between these two extremes by varying C.
In this work, we study the problem of finding prediction rules that actively shape the distribution of observed out- comes y, yet still produce a valid forecast.
While the guarantee in Definition 1.1 is known to be achiev- able in supervised learning where D(f) = D( for all f and f', the situation substantially more complicated in this performative setting. Predictions can, in general, be self- negaring, and outcomes can move away from the published prediction p f(x). Given this possibility, it is not imme- diately obvious that performatively calibrated f even exist (that is f such that [f = Ep(r, [v) without mak- ing strong regularity assumptions on D(-). The situation is further complicated because the distribution map D(-) is unknown the learner. They can only deploy a predictor f and observe the induced samples (z.P. v) D(f).
We cast this problem in the language of performative pre- diction (Perdomo et al., 2020), a recent learning theoretic framework that formalizes how data in social contexts is not static, but rather function of the published predictor f- The key conceptual device of the framework is the notion of the distribution map D(.), a function which maps predictors f to distributions over feature, outcome pairs (z, y). It cap- tures how deploying different predictors induces different distributions.
In this work, we focus on the outcome performative case, in which predictions shape the distribution over outcomes y but not features x. In particular, given randomized predictor f, we write (z. v) D(f) as shorthand for
Our first result shows that this guarantee Definition 1.1 is. in fact, efficiently achievable both statistically and compu- tationally, thereby providing a modern learning- theoretic answer to the question posed by Morgenstern, Simon, and others. Unlike the bulk of work in performative prediction (Perdomo et al., 2020; Mendler-Dünner al., 2020), this result requires smoothness or continuity assumptions on the outcome performative distribution map D(-) other than the condition that outcomes y lie in a bounded range.
D.. [(x), and pz, The distribution D,r over x is static, a prediction p is sampled from f(z), and the conditional distribution over outcomes y is any function of ar and the forecaster's revealed prediction p E R. This setup exactly captures the prediction dynamics present in domains like health or education (see e.g. Perdomo et al. (2023) for real world example). The learner makes prediction p regarding someone's individual outcome (future heart disease) using historical features (age, smoking history). The prediction influences treatments and individual behavior in ways that shape the likelihood of the predicted outcome (disease), but not the historical realization of features.
Theorem (Informal). Let A be any online algorithm which is guaranteed to produce predictions pt such that for any adversarially chosen sequence {( Dt)
T Σ> i=t
pa)l < Regret(T) for all c c.C. (1)
We assert that forecast is valid if it is performatively malti- calibrated, synonymously, indistinguishabbe. Intuitively, predictor is multicalibrated if f (xx) equals (in expecta- tion), not just overall, but also when we condition on infor- (x)).
Then, given n draws from D(-), the outputs of A can be con- verted into randomized predictor SA s.t. with probability
mation about the individual x and the prediction A predictor is outcome indistinguishable if its correctness cannot be efficiently falsified, on the basis of the observed data and pre-specified collection of computational tests. We use the terms multicalibration and (outcome) indistin- guishability interchangeably since, interestingly, both con- ditions are formally equivalent (Dwork et al., 2021). We formally define these concepts for our performative setting below:
1 - ô over the randomness of the draws from D(-).
((CE [c(2, p)(u - -P)l Regret(n)
log((C)) - log(1/8)
for all c E C.
n
Furthermore, if A runs in time(t) at round t, then fa can be computed in time O(n - -imme(n)).
The result establishes an efficient reduction from our main problem to an online multicalibration problem for which nu- merous algorithms now exist (Vovk, 2007; Foster Kakade, 2006; Dwork et al., 2025; Gupta et al., 2022: Okoroafor et al., 2025). In particular, these algorithms can efficiently
Definition 1.1. A randomized predictor í mapping fea- tures to predictions p is e performatively multicalibrated (indistinguishable) with respect to C C {X' x RR- + R} if
Bee Soa)lee (x.)(
p)]l < e for all c C.
'Recall that (z, y) ~ D(f) is shorthand for the sampling process x ~ Da, ~ f(x), y ~ D, We abuse notation and write (x. ~ D(f) if the forecast p does not appear in the expectation, but data (z, y) is still that induced by f. For deterministic functions h, we write (z, y) D(h) as shorthand for x ~ Dz: y ~ Dy h(z)).
Here, C parametrizes the degree to which predictions match outcomes. C consists of just the constant 1 function, we recover the guarantee considered in Simon (1954) and Grun- berg& & Modigliani (1954) that E [yl = Exon
2

---

Revisiting the Predictabiliity of Performative, Social Events
achieve the online guarantee in Equation (1) for various rich classes of functions C such as low-degree polynomials, decision trees, or, more generally, any C. that is (weak ag- nostically) learnable or that belongs to reproducing kernel Hilbert space. Moreover, these algorithms have vT regret, implying that f achieves the optimal problem. The reduction is conceptually simple, yields tight bounds on the generalization error, and requires no Lips- chitzness conditions on the distribution map D(.). That is, the conditional distribution over outcomes D, (z; p) can be any discontinuous, non-smooth function of the forecast p. We only need to assume that outcomes are bounded.
However, there exist a distribution map D(-) such that / can be performatively multicalibrated with respect to all bounded functions c(x,p), yet simultaneously maximize the performative risk for any H.
B B(.,,,))) -(xx)² 2 max B(z,g) - -()))))(((((())) h(x))² hER
rate for this
The theorem states that any predictor that is performatively multicalibrated with respect to H. is also performatively sta- ble with respect to H. A predictor f is performatively stable if it induces a distribution D(f) such that no model h E H has a lower loss over D(J). This is Equation (2). If we judge the performance of a performatively stable predictor f (and the alternatives h) purely based on the distribution Dff), there is no reason to switch to alternative model in H since these have higher loss over D(J).
Taking a step back, the theorem states that there are learning algorithms that can efficiently cope with social feedback. They make predictions that actively shape future events while still providing an honest, calibrated signal of the out- come. The fact that these prediction rules exist is fascinating and motivates us to ask further questions. Are these pre- diction rules truly desirable? What does it mean to make a good prediction if the prediction itself influences outcomes? Part of the answer to this question involves rethinking what the goals of prediction should be and how these goals are re- flected in our choice of loss function. See (Kim & Perdomo, 2023) and (Miller et al., 2021) for further discussion.
However, stability ignores the fact that different predictors induce different distributions. Note that the expectation in the right hand side of the stability condition, Equation (2), is taken over (f) not ((h). Yet, the true measure of performance for a predictor h in performative contexts is its performative risk the expected loss over its own induced distribution - formally defined as ED( (U h(z))² Note that the predictor inside the loss function and the predictor in the distribution map D(-) are now the same. A model f is performatively optimal with respect to a class H if it has lower performative risk than any model in h E H. Equation (3) states that it is possible for predictor to be performatively multicalibrated with respect to all functions c(x, p) while simultaneously maximizing the performative risk.' 3
However, leaving the choice of loss function aside and re- stricting ourselves to predictive accuracy as the main cri- terion, prediction rules satisfying the historical desiderata formalized in Definition 1.1 can be arbitrarily poor. We show that it possible for a predictor f to be performatively multicalibrated with respect to all (continuous) functions c(z,p) while simultaneously explaining none of the vari- ance in outcomes. We state the following result in terms of the core solution concepts from the performative predic- tion framework performative stability and performative optimality - whose definitions we review below.?
Note that our intuition from supervised learning is exactly reversed. In supervised learning, where (xx, y) ~ D, if f(z) = Ely x] for all x. then ED - fx))² < Ed(y- h(z))² for any function h. In performative prediction, where f influences the distribution over (z, y). the result shows it is possible for f(x ED(r)(B æ] for every x, yet Ed( (y - fx)))² > ED - h(x))² hx())) The inequality is now flipped because of performativity.
Theorem (Informal). Assume that y is binary and let H C (X' [0, 1]} be a benchmark class. Any predictor ff that is performatively multicalibrated with respect to H (satisfies Definition 1.1 with C = H) is also performatively stable with respect to H and hence satisfies:
As a whole, our results advance the mechanies and algorith- mic foundations of social prediction, explaining how and why one can find predictors that dynamically shape yet also rigorously predict social outcomes. In doing so, we shed new light on old questions previously considered by leading researchers of the 20th century. Lastly, by re-evaluating historical desiderata using a modern lens, we highlight ways in which previous debates fall short and inspire new debate regarding what the broader goals of prediction in the social world should be.
E ((. p) < min Bee h()))²
NCH
(2)
"The solution concepts of performative stability and optimality are defined with respect to general loss functions {(x, y; h) that can capture a wide variety of higher level objectives. For instance, we choose to have losses which encourage forecasts to match outcomes, f(zz, y; h) (h(x) - 2) or losses which encour- age the induced distributions D(f) towards particular outcome f(x,, y; h) = -y. See (Kim & Perdomo, 2023). Since the goal here is to understand the relationship between from Definition 1.1 and predictive accuracy in performative contexts, we specialize the definitions to the case of squared loss for simplicity.
"There is a non-trivial gap for this problem, ming EDoa h(x))² & max; Eags) (y- - h(x))² 2 See Section 5.
3

---

Revisiting the Predictabilityy of Performative, Social Events
1.2. Related Work
á belonging to finite set. Our setup generalizes theirs since both outcomes y and forecasts p can be real-valued.
Social Science. As discussed in the introduction, several authors across the social sciences have studied the ques- tion at the heart of our work regarding the predictability of social events. In addition to Morgenstern (1928), Simon (1954) states this exact problem in the introduction to his paper, where he remarks that he learned about the issue from Hayek (1944). The existence, but not computation, of a predictor solving this problem is also considered in Grunberg & Modigliani (1954). Their paper inspired our choice of title.
2. Preliminaries
Before presenting our main results, we review some techni- cal preliminaries and provide some background and motiva- tion behind our core solution concepts.
The Distribution Map. We assume throughout that data (r. v. D() is generated according to the following process. The learner publishes a predictor f X + A 1]) mapping features x to a distribution f(x) over the unit interval [0, 1]. Features x are drawn i.i.d from a fixed distribution Dr over an arbitrary set X, predictions p E [o, 1] are sampled from f. p f(z). Outcomes y are sampled from Dy(a,,) p) where (2 is any distribu- tion over the unit interval [o. 1]. Our results apply to any (2 p) supported on bounded interval by rescaling.
Our problem also lies at the heart of the famous Lucas critique (Lucas, 1976), which marked a turning point in macroeconomic theory. Outside of economics, this problem has been extensively studied in sociology dating back to work by Merton (1948) and more recently by MacKenzie (2008). In philosophy, it is studied in Buck (1963). Our results complement prior work by advancing our algorith- mic understanding of the problem and establishing how these prediction problems can be efficiently solved both statistically and computationally.
Performative Multicalibration. Calibration is the sine qua non definition of validity for a probabilistic forecast (Dawid, 1985). For binary y. a predictor calibrated if con- ditional on f(x) o for 0 E [0, 1]. the outcome y occurs v fraction of the time, Pr(y calibration (Hébert-Johnson et al., 2018) is strengthening of calibration, requiring that f(x) = y, not just overall, but also once we condition on æ belonging to any set G in a col- lection G: Pr-y = 1 f(x) = u, x E G] = v for all G E G. This condition is equivalent, up to a normalization factor of Pr(f(z) = u, x E G]. to the one we wrote in Definition 1.1 since letting (z, 1(x E G.p v). we can write:
Performative Prediction. The area of performative predic- tion was initiated by Perdomo al. (2020), who proposed formal framework to study predictions that shape data distributions. We cannot cover all the work in this grow- ing field, but we point the reader to the excellent survey by Hardt & Mendler-Dünnner (2025) for a broader overview. Within this literature, our results are most closely related to the literature on finding performatively stable points (Mendler-Düinner et al., 2020; Drusvyatskiy & Xiao, 2023; Mofakhami et al., 2023; Oesterheld et al., 2023; Mofakhami, 2024; Taori& & Hashimoto, 2023; Khorsandi et al., 2024). Relative to these analyses, our results differ since we make no smoothness assumptions on the way forecasts influence outcomes and restrict ourselves to the outcome performa- tive setting where predictions only influence the distribution over the outcomes, but not features.
1 [ fax) = v] = u. Multi-
E Bcc.:))--P((jp--tttccc.... () p)] = (Pr(y = 1 f = p.v G v) Pr[ v, E G).
Conversely, a predictor f is outcome indistinguishable (or OI) (Dwork et al., 2021) if it establishes a genera- tive model that cannot be falsified on the basis of a pre- specified collection of tests or distinguishers
E F S {X' x [0, 1]x [0, 1] + RJ. Simplifying our discussion to the case of binary outcome, each distinguisher takes features x, predictions p. and an outcome y and outputs I or 0 (i.e. is this a real outcome/prediction for x). A predictor is Ol if all the distinguishers in the collection F. behave the same when given the true outcome ~ D(f) versus an outcome sampled from the model ÿ ~ Ber(p) p ~ fxx) Extended to the performative context, f is outcome indistinguishable with respect F if for all A E F
Multicalibration & Outcome Indistinguishabiiity. Our results build the recent lines of work on multicalibration HHébert-Johnson et al., 2018) and outcome indistinguishabil- ity (Dwork et al., 2021), further extending the foundations of these ideas to non-supervised learning settings. Our proofs heavily rely on new algorithmic insights from online cali- bration, pioneered by Foster & Vohra (1998), and extended by (Foster& & Kakade, 2006; Kakade & Foster, 2008; Vovk, 2007: Gupta et al., 2022) and others. Within this broad liter- ature, our work is most closely related to Kim & Perdomo (2023), who used these tools developed in Gopalan et al. (2023) to compute performatively optimal (not stable) pre- dictors for a restricted version of the outcome performativity setting. In their setup, outcomes are binary and their condi- tional distribution is a function of x and a discrete decision
E B(. A
A(z, p. (4)
For binary y. this guarantee is, in fact, equivalent to our performative multicalibration guarantee from Defini- tion 1.1 since this above equation is true if and only
4

---

Revisiting the Predictabiliity of Performative, Social Events
if ((..p.)-D(nncd,, p)ly P)] O for ca(x,) 1(2,,,1) A(x, p. 0), as seen in Dwork et al. (2021) for the supervised learning case.
we assume that the learner moves first and commits to function ft at every round before seeing 2 (instead of mov- ing second and producing prediction Pr after seeing the features x²). However, this is just a difference in style, not substance. Most online algorithms in the literature also commit to a prediction rule ft : X + [0, 1] at every round before seeing the features z4. Because fa is revealed, Nature knows P: f(())) for every 2, and can use this informa- tion (potentially adversarially) when choosing 24 and yt- Moving on, our results rely on algorithms that achieve the following guarantee in online prediction.
We feel that this rewriting is particularly insightful. Note that outcomes on the left of Equation (4) are performative: They are sampled from D(f). However, outcomes on the right hand side are sampled according to f. The predictor f is actively influencing the distribution in such way that outcomes behave as if they were sampled according to its own predicted distribution. By observing samples (Ir, v) ~ P(f). we might seemingly believe that ff is not influencing the data at all! It's just a great predietor.
Definition 3.2 (Online Multicalibration). Let C S {x x [0, 1] + } be class of functions. An algorithm A guaran- tees online multicalibration with respect to set C at a rate bounded by R if it always generates a sequence of functions ft for the Learner that, no matter Nature's 's strategy in the online protocol (Definition 3.1), will yield a transcript {( Mr. fo) satisfying.
However, this intuition is false. A predictor / can pass all bounded tests F and be multicalibrated with respect to any collection C while simultaneously explaining none of the variance in y. We present this construction in Section 5.
3. Algorithmic Results
This section presents our core algorithmic results, illustrat- ing how one can find prediction rules that influence data and are indistinguishable from the true outcomes. These algorithmic procedures are conceptually simple, computa- tionally efficient, and near statistically optimal.
Σ ddt, - --)1 S Regret, (T),
for all e E where Regret :N R20 is o(T).
Algorithms that achieve this guarantee date back to the work Vovk (2007); Vovk et al. (2005a) and Foster & Kakade (2006) (they refer to it under different names like resolution or just calibration). Following the work of Hébert-Johnson et al. (2018) introducing multicalibration, there has been a flurry of recent papers introducing new algorithms for this problem (Dwork et al., 2025; Gupta et al., 2022: Garg et al., 2024; Okoroafor et al., 2025). We will make use of these procedures when instantiating our general results. As final preliminary step, we define the online-to-bated procedure we use in our analysis:
Technical Overview. Recall that the goal is to find pre- diction rule f satisfying the following indistinguishabiliyy guarantee from Definition I.I. For all c E C,
B(.,)))) - p)c(2, p S..
(5)
Rather than solving this problem directly, we reduce it to a seemingly harder, online problem and then perform an online-to-batch conversion. That is, we show that any online algorithm A that (deterministically) produces predictions Pt (3:) such that Σ'= c(I2, Pt) (y) pt)l S o(T), for any adversarial sequence of {z 2) )= can be con- verted to batch predictor ʃ satisfying the indistiguishability
Definition 3.3 (Batch Converstion). Fix an outcome per- formative distribution map D(-). Let (( Vt, ft, ) be the transcript generated in the online prediction proto- col (Definition 3.1) where at every time step 4, the Learner chooses ft according to A, and Nature selects features It and outcomes y/. by sampling them from the distribution map: x2 ~ Dz, yt+ ~ D(re: Pt) where p:- = fx())).
guarantee from Equation (5). We begin by formally defining the online protocol used in our reduction.
Definition 3.1 (Online Prediction). Online prediction is a sequential, two-player game between a Learner and Nature. At every round t, the Learner moves first and sclects a function ft= : X + j0, 1), deterministically mapping E X to predictions Pt in [o, 1]. as a function of the history up until time f. Nature moves second and selects (æte y:) E X x y with knowledge of 2, and P.- = fe(x)). We refer to the sequence {(xx Me. ft. Pt) PT as the transcript of the game.
Detine the T -rund batch version of A. fA X + A(OO, 1)) to be the randomized predictor that given x, selects fo (J,...., frr from the transcript uniformly at random, and then predicts p = f((x).
This style of online-to-beid -atch conversion, where one uni- formly randomizes over all previous predictors, is standard in the online learning literature and is the typical starting point when converting online algorithms to batch learners (see, e.g., Gupta et al. (2022); Okoroafor et al. (2025)). The main difference in this construction that samples (Ir, yt)
Online prediction is a classical problem in statistics, game theory, and machine learning for which numerous algo- rithms have been developed (Cesa-Biand) Bianchi & Lugosi, 2006; Foster& & Vohra, 1998; Kakade& & Foster, 2008). Our defi- nition differs slightly from traditional presentations since
5

---

Revisiting the Predictabiliity of Performative, Social Events
are not drawn i.i.d from a fixed distribution D, but rather from the distribution (It, yt) D(f+) induced by the pre- dictions. Note that while each of the individual functions fe E ffi....,fa f in the transcript are deterministic, the batch predictor is randomized since it first mixes over the choice of ft. With these definitions out of the way, we can now state the main theorem for this section:
at time step I, then the online-to-batch conversion takes time (n time(n)). Therefore, if A runs in polynomial time, so does the batch version fA-
Example Instantiations. we now illustrate how it can be instantiated using existing algorithms to guarantee multicalibration with respect to rich classes of functions C. Since there are by now many dif- ferent online algorithms that satisfy Definition 3.2, these examples are by no means meant to be exhaustive. We simply wish to illustrate some interesting cases with the understanding that there are numerous alternatives. Our end-to-end results can be improved as the community devel- ops online algorithms with sharper regret bounds or better runtimes that can be plugged into Theorem 3.4.
Having introduced the main result,
be a class of functions and let A be an algorithm that guarantees on- line multicalibration with respect to C a rate bounded by Regret (Definition 3.2). Define LA to be the batch rounds of interaction (see Defi-
Theorem 3.4. Let C S {X x
predictor of A trained nition 3.3). Then, with probability 1 - 8 over the samples drawn from for all c C.C.
E (...))- (p - U)
In particular, we instantiate our general reduction using the K29 algorithm from (Vovk et al., 2005a;b). This algorithm is simple, kernel-based procedure which can efficiently guarantee calibration with respect to functions in a Repro- ducing Kernel Hilbert Space. We present a self-contained description and analysis of the algorithm in Appendix A. The following corollary summarizes some of its implica- tions.
Regret A (n) /
log((C)) + log(1/6)
Proof Sketch. The proof follows the standard template for online-to-batch conversions. Since outcomes are a function of x and specific prediction p we can use the definition of fa to decompose the left hand side as:
Corollary 3.5. The following statements are true.
Be D)(D- y)
(a) Any Finite Collection. Let C S. (X' x [0, [-1,1]} be any finite set of functions c(z, P) that are continuous in the forecast p. Then, there exists a choice of kernel, such that the K29 algorithm is online multicalibrated with respect to C at a rate bounded by VT(C). Consequently, the batch version f trained on n rounds of interaction satisfies:
(x.p((z.))
E a~D
[d(x v) SA fi]. ((..))
Then, viewing the transcript {(T) ye, for PT as a stochastic process, we use a Martingale argument and ap- ply the Azuma-Hoeffding inequality to establish high- probability upper bound on the RHS.
v) S
[ 44/ log((C) + log(1/5)
for all c E C.
As we mentioned, Theorem 3.4 is a general reduction. It states that any algorithm that is online multicalibrated with respect to class of functions C yields (batch) prediction rule fA that is performatively multicalibrated with respect to C. The reduction, moreover, is statistically efficient: the error decreases at the optimal (n-1/22 rate and the de- pendence on the failure probability d is logarithmic." 4 The log(IC)) dependence comes from standard union bound argument and can be sharpened using well-known tech- niques. The reduction is also computationally efficient. If the runtime of the online algorithm A is bounded by time(t)
Furthermore, the per-round run-time is Ö(t (CI. Hence, f runs in time O(n2
(b) Linear Functions. Define X= {I Cin = x |10]|2 S 1}, and let C be funite subset of Cin- Then, there exists a choice of kernel such that the K29 algorithm is online multicalibrated with respect to Ciin at a rate bounded by V2T. Consequently, the batch version f of this procedure trained on n rounds of interaction satisfies:
R || S 1}.
- v)
"The dependence n cannot be improved without further assumptions. In particular, take the case where C only contains the constant one function, y is hinary, and (-)) is not performative so that (z. 3) ~ D, where D. is a fixed distribution. In this case, the problem becomes mean estimation for a Bernoulli random variable, which has a well-known 2(n -1/3) lower bound (see e.g. Anthony& & Bartlett (2009))
2 V 4 log((C)) + 1o8(1/8)
for all c E C.
Furthermore, the per-round run-time is bounded by Ö(d).
Hence, f runs in time Ö(n d).
'One can, under further assumptions, replace log((C)) with norm-hased bound Bosa-Bianchi et al., 2004).
(c) Low-Degree Boolean Functions. Define X = 10, and CLowDog C {X + [-1,1]] to be the set of Boolean
6

---

Revisiting the Predictabiliity of Performative, Social Events
fincciins of degree 8. That is, those that can be written as,
On the other hand, a predictor fpo is performatively optimal with respect the class H if,
c(z) -
Σ
as II a
E D(,. ((p y) S min E ((n.w))-Dn)((((())) y)
RES
kCH
for some coefficients {as) E R. Let C be any finite subset of CLowDeg U (c(x,)) = p}- Then, there exists a choice of kernel such that the K29 algorithm online multi- calibrated with respect to CLowDeg U {c(zz p) = p) at rate bounded by 10 .T
E..
Lastly, we refer to (x))))))(()(((()())) y) as the performa- tive risk of a predictor
Intuitively, a predictor f is performatively stable if its op- timality cannot be refuted on the basis of the data that it induces. If we evaluate the loss of any alternative predictor h over the distribution D(f)) we find it will have higher loss than f itself. However, this ignores the fact that differ- ent models induce distributions. Performatively optimality embraces this observation. A model f is performatively optimal if it minimizes the performative risk.
Consequently, the batch version f of this procedure trained on n rounds of interaction satisfies:
E ((xx))=(()))---(()) p)(p - -v)l
d= n AV
log (I) 1og(1/0)
10
for all c E C
As stated, our definitions slight generalizations of those initially proposed by Perdomo et al. (2020) since we allow f to be randomized and do not require that it be member of the class H. Furthermore, our definition holds for any possibly non-parametric class H. However, if H = Fe is a parametric class and we impose that fps Fe. we recover the previous definition,
Lastly, the per-round run-time of the algorithm is Ö( ds). Hence, f runs in time (ds n²).
The corollary illustrates how one can efficiently find pre- diction rules If that are performatively multicalibrated with respect to common classes of functions, for instance, linear functions or low-degree polynomials. One can even effi- ciently guarantee indistinguishability with respect to any polynomially sized collection (as long as the functions c C are each efficiently computable). This, in particu- lar, implies performative indistinguishability with respect to rich classes of functions, like deep neural networks. These hold without making any Lipschitzness assumptions on the distribution map (--). Furthermore, the runtime of the pro- cedures is, up to small polynomial factors, no different than that of algorithms achieving solving the analogous guaran- tee in supervised learning settings.
Pps E arg min (. AEO E ((x,,))D(0,,) e(Jo (x), S min E Dle.
BEE
The following theorem is the main result of this section, showing how performatively multicalibrated predictors as per Definition 1.1 are also performatively stable with respect to standard losses like squared error. To simplify our pre- sentation, we make the further assumption that y is binary.
Theorem 4.2. Assume that outcomes y are binary. Let H X - [0 be benchmark class and let E be the squared loss, É(p, v) = { - y)². If f is e-performatiell multicalibrated with respect to the functions C, defined as, E H),
4. Structural Results
In the previous section, we illustrated how one can find predictors f that actively shape the data (z. y) ~ D(f), yet still make predictions that are computationally indistin- guishable from the true outcomes. Here, we take a step further and ask: are these computationally indistinguishable predictors useful (in risk minimization sense)? What is the relationship between calibration and loss minimization in this outcome performative context?
C ==(c(, p) = p- p-11 1/11////ccsxpp p) = h(x)-1/22 then f is Pe-performativel stable with respect to H.
The proof of this result follows from two lemmas. These make use of recent tools pioneered by Gopalan et al. (2023) for the supervised learning setting, showing how outcome indistinguishable predictors are loss-minimizing.
Lemma 4.3. Assume that outcomes y are binary and let é: [0.1] x {0.1) > R be a proper scoring rule. Fix a parameter e > O. If a function I I - [0, 1] satisfies the following inequalities for all E H.
To answer these, we analyze the relationship between the previous historical desiderata (Definition 1.1) and the core solutions in performative prediction. We review these below. Definition 4.1 (Performative Stability and Optimality). Let l be a loss function an H [ (X - [0, 1]} a benchmark class. A predictor fps is performatively stable with respect to the class H if,
B(s 9) S Eu-De C(p v) + (6) g)BBr((p)
((( S B( v) + y-Ber(p))
E ((. ((p, v) < min ((e.)) ((h(z), v).
MEH
then, f is 2e-performatiivll stable with respect to H.
7

---

Revisiting the Predictabiliity of Performative, Social Events
Proof. The proof follows immediately from the indistin- guishability conditions and the assumption that is proper loss. In particular, from the first condition we have that:
Lemma 4.4. If E is the squared loss v( 2/(1 - -p)²
then
E ((e.,))- BEWDe.p (p, v)1 S BrBBe((p)
((x,,,))D()) (p, y) S Eg
Cp. y)
Bew DeDDa ((((1)))) - BLE ((b(x)))))))))))))))))()))) v) Se yr-Ber(p)
Next, the definition of proper loss is that if y Ber(p), every other h: : X + [0. 1] must have at least as high loss,
for all E H if and only if I is e-performatively multi- calibrated with respect to the functions, C {c(r,p)
Dz x~ f(x) ((z)) ((p, v) S
p- 1/2) U (c(x,p) = h(x) 1/2 H).
min Eu pr s2)(()))))))))))))) (z)))r (h(()),)) v)- hEH
Theorem 4.2 follows directly from the last two lemmas. Moreover, it also follows from Corollary 3.5 that one can statistically and computationally efficiently find predictors f that are performatively stable for important classes of benchmark classes H by via an online-to-batch conversion.
The result then follows from using the second assumption which guarantees that for any h E H.
(h(x))))
(7)
B().))-D()((((())) (h(()))))))))))) +E.
These results strengthen our previous understanding of per- formative stability. In the initial work on performative prediction, stable predictors were only known to be effi- ciently computable under strong regularity conditions on the distribution map D(-) (Perdomo et al., 2020; Mendler- Dünner et al., 2020; Drusvyatskiy & Xiao, 2023). Specifi- cally, Dy (2,, p) is assumed to be sufficiently Lipschitz in the forecast p. This restriction rules out common settings where decision makers choose actions that influence outcomes if the forecast p is above or below some threshold. In educa- tion, for instance, counselors at school often assign extra attention to students by examining whether their predicted probabilities are below some fixed threshold (Perdomo et al., 2023). This thresholding implies that the distribution map is not Lipschitz.
Chaining all inequalities together, we get the 2e bound.
The inequalities in the assumptions of the lemma are best understood as a particular kind of loss outcome indistin- guishability conditions (Gopalan et al., 2023). Recalling our discussion from Section 2, one can think of the losses {(h(x),)) as a type of distinguisher As(xr, h(x)). The lemma shows that if f is a generative model of outcomes that passes class of tests defined by l and H. then f is performatively stable.
A similar condition had also been considered in the per- formative prediction literature by Kim & Perdomo (2023). The key conceptual difference is that (Kim & Perdomo, 2023) consider learning a model D.y (a. p) that is indistin- guishable from the true distribution map (2(,,p)) p) from the the set
To the best of our knowledge, all subsequent work in this area also imposed some continuity restriction on D(-) to prove that algorithms converged to stable points (e.g. (Khor- sandi et al., 2024; Taori & Hashimoto, 2023)). 6 Except for the fact that y is binary, we establish convergence to stabil- ity under essentially the weakest possible conditions for the outcome performative setting since Dy, (x. p) is unrestricted.
perspective of a set of tests Ae,. (that depend of benchmark functions h and loss function E). Informally, (x, (())) PA,, (2 h(x)) for all (h, E) in some set.
However, the indistinguishability conditions in Lemma 4.3 are with respect to a predictor f X + ([O,, 1)), not a model of the distribution map Dy : X x [0, 1] + A(Oo, 1)). They relate the performative risk to the expected risk where outcomes are self-confirming and sampled from the model proposed by f. Note that p fx) and y Ber(p) on the LHS of Equation (7). Stated again informally, these con- ditions require that (x,, (x)) AA,, fxx). Whereas the indistinguishability criteria from (Kim & Perdomo, 2023) yield performative optimality, ours yield performative sta- bility. Hence, the takeaway message from this discussion is that stability (not optimality) is the natural limit of deploying online calibration algorithms in performative contexts.
5. Suboptimality of Perfectly Calibrated Performative Predictions
We end our work by showing how validity desiderata devel- oped in supervised learning contexts (like calibration) fall short in performative settings. For predictor to be good, it's not enough for it to forecast outcomes accurately; it needs to actively steer the data and make use of the fact that f shapes D(f). In more detail, we establish the following result.
The next result finishes the proof of Theorem 4.2. It es- tablishes that the loss Ol condition from Lemma 4.3 is equivalent to performative multicalibration if we fix to be the squared loss.
An exception to this is the recent paper by (Wang et al., 2025). They show convergence to performative stability for specific multi-agent performative prediction problem.
8

---

Revisiting the Predictability of Performative, Social Events
Prly = 1 f()) = ppl
By letting a go to O, ((e)))D(.)) ((p)(y p)] 0 since lima-so c(1/2 a) c1/2).. This predictor fr which uniformly mixes between 1/2 and 1/2 e also satisfies
E By-D - - 1/4 O(a)
since y is either or O and p is always nearly 1/2. The performative risk for this problem is a quadratic function in p that is maximized at p = 1 1/2. Yet the performatively optimal solutions are to predict either 0 or 1. For these, we would have the best possible performative risk of .01. O
p
Figure 1. The distribution map in Theorem 5.1. The solid black lines describe the probability that y = 1 given that the prediction is equal to p. The blue dotted line indicates the probability that y one deploys the randomized predictor fr that mixes hetween 1/2 with probability Ả and 1/2 + with probability 1 A for the whole range of À E [o, 1]. Fixed points, Epcn [b] [x)) are those that cross the dotted, y x, diagonal line.
This construction complements insights from previous result by Miller et al. (2021), who also showed how perfor- matively stable models can maximize the performative risk. Our result provides a different perspective by connecting these notions of loss minimization (stability and optimality) with notions of computational indistinguishabiiity. Lastly, we presented this construction where there are no features for the sake of simplicity. However, one could extend it to include features by redoing a similar conditional distribution pointwise for every x E X.
Theorem 5.1. There exists a distribution map D()), such that for any O. There exists randomized f such that for all bounded functions e(2, p) that are continuous in p.
6. Discussion and Future Work
B(pp)))-(plc( p) - p)] s.
Prediction algorithms are commonplace in important social domains and performativity abounds. By shaping our mecision-making, these prediction algorithms influence the outcomes we see. Drawing on a long intellectual history, our work revisits core methodological questions regarding the design and evaluation of predictors in these domains.
Furthermore, for any function, h:X + [0.
((EPa)) u)² B(ax)~D(h)(( h(x)))² O(e).
Hence, for any class functions H C (X > 1]::
B(..)))>D(((( - -
On the design side, we introduced new algorithms that can overcome social feedback and produce rigorously calibrated predictions of social events. These results resolve algorith- mic questions left open by (Simon, 1954) and (Grunberg & Modigliani, 1954). On the evaluation side, our contributions are mostly conceptual. We illustrate how traditional desider- ata imported from supervised learning calibration and its different variants mean something quite different in these performative contexts. These insights call into question the utility and significance of public forecasts of social events that stake their validity on calibration (Silver, 2019).
max E (e2)) (y -(s))
NEH
Proof. Consider the case where there are no features, and the outcome y is binary. Since there are no features, we define the distribution map be Dy p) Pr(y = 1 pl (() where g(p) p .01 if I 5 and p - -01 if p > 5.. We depict the distribution map visually in Figure 1.
Note that there is no determistic predictor fp = p such that EDu,,lvi = =EDU, [fp] since EDU,, [bv] = =p)) and there does not exist p E (0, 1] such that g(p) = p. However, the predictor fr that uniformly mixes between p1 = 1/2 and P2 = 1/2 + a for some small a satisfies (Ey=D(f-))u Ep-/- f- [p] for any a < .5. Hence, if we don't 't impose continu- ity assumptions on D, we need predictors to be randomized to find solutions such that ED - f(x)] O.Moreovr, given any continuous e: [0, 1] + [-1, 1].
There are a number of interesting directions for future work in this area. For one, we prove our results for the simplest case of performative prediction: (stateless) outcome per- formativity. It would be interesting to consider whether these results carry over to richer, stateful domains where the outcomes we see don't just depend on the predictions we make today, but also on the predictions we' ve made in the past (Brown et al., 2022). would also be valuable to fully understand whethere these results carry over to the setting where both feature and outcomes are performative.
By-D(s)e (clp) p)] = .01 (c(1/2) c(1/2 a))

---

Revisiting the Predictability of Performative, Social Events
Acknowledgements
Garg, S., Jung, C., Reingold, O., and Roth, A. Oracle efficient online multicalibration and omniprediction. In Symposium on Discrete Algorithms, 2024.
would like to thank Aaron Roth, Tijana Zrnic, and the anonymous ICML reviewers for helpful comments and dis- cussion. JCP was supported by the Harvard Center for Research Computation and Society well by Alfred P. Sloan Foundation grant G-2020-1394)
Gopalan, P., Hu, L., Kim, M. P., Reingold, O., and Wieder, U. Loss minimization through the lens of outcome in- distinguishability. Innovations Theoretical Computer Science, 2023.
Impact Statement
Grunberg, E. and Modigliani, F. The predictability of social events. Journal of Political Economy, 1954.
This paper presents work whose goal advance the field of Machine Learning. There are many potential societal consequences of our work, none which feel must be specifically highlighted here.
Gupta, V., Jung, C., Noarov, G., Pai, M. M., and Roth, Online multivalid learning: Means, moments, and prediction intervals. Innovations in Theoretical Computer Science, 2022.
References
Hardt, M. and Mendler-Dünner, C. Performative prediction: Past and future. Statistical Science, 2025.
Anthony, M. and Bartlett, P. L. Neural network learning: Theoretical foundations. Cambridge University Press, 2009.
Hayek, v. Scientism and the study of society, part iii. Economica, 1944.
Brown, G., Hod, S., and Kalemaj. I. Performative predic- tion in stateful world. In International Conference Artificial Intelligence and Statistics, 2022.
Hébert-Johnson, U., Kim, M., Reingold, O., and Rothblum, G. Multicalibration: Calibration for the (computationally- identifiable) masses. In International Conference on Ma- chine Learning. 2018.
Buck, R. C. Reflexive predictions. Philosophy of Science, 1963.
Kakade, S. M. and Foster, P. Deterministic calibration and nash equilibrium. Journal of Computer and System Sciences, 2008
Cesa-Bianchi, N. and Lugosi, G. Prediction, learning, and games. Cambridge university press, 2006.
Cesa-Bianchi, N., Conconi, A., and Gentile, C. the gen- eralization ability of on-line learning algorithms. IEEE Transactions Information Theory, 2004.
Khorsandi, P., Gupta, R., Mofakhami, M., Lacoste-Julien, S., and Gidel, G. Tight lower bounds and improved convergence performative prediction. arXiv preprint, 2024.
Dawid, A. P. Calibration- based empirical probability. The Annals of Statistics, 1985.
Kim, M. and Perdomo, J. C. Making decisions under outcome performativity. Innovations in Theoretical Com- puter Science, 2023.
Drusvyatskiy, D. and Xiao, L. Stochastic optimization with decision-dependeen distributions. Mathematics of Opera- tions Research, 2023.
Lucas, R. E. Econometric policy evaluation: A critique. In Cameeii- conference series on public policy, 1976.
Dwork, C., Kim, M. P., Reingold, O., Rothblum, G. N., and Yona, G. Outcome indistinguishabiiity Symposiun on Theory of Computing, 2021.
MacKenzie, D. An engine, not camera: How financial models shape markets. MIT Press, 2008.
Dwork, C., Hays, C., Immorlica, N., Perdomo, J. C., and Tankala, From faimess to infinity: Outcome- indistinguishable (omni) prediction in evolving graphs. Conference Learning Theory, 2025.
Mendler- Mendler-Dünner, C., Perdomo, J., Zmnc, T., and Hardt, M.
Stochastic optimization for performative prediction. Ad- vances in Neural Information Processing Systems, 2020.
Merton, R. K. The self-fulilling prophecy. The Antioch Review, 1948.
Foster, D. and Kakade, Calibration via regression. IEEE Information Theory Workshop, 2006.
Miller, J. P., Perdomo, C., and Zmic T. Outside the echo chamber: Optimizing the performative risk. In International Conference on Machine Learning, 2021.
Foster, D. and Vohra, R. V. Asymptotic calibration. Biometrika, 1998.
10

---

Revisiting the Predictability of Performative, Social Events
Mofakhami, M. Performative prediction: expanding theo- retical horizons. 2024.
Mofakhami, M., Mitliagkas, I., and Gidel, G. Performa- tive prediction with neural networks. In International Conference Artificial Intelligence and Statistics, 2023.
Morgenstemn, O. Wirtschafssroonnose: Eine Untersuchung ihrer Voranssetzungen und Möglichkeiten. Springer, 1928.
Oesterheld, C., Treutlein, J., Cooper, E., and Hudson, R. Incentivizing honest performative predictions with proper scoring rules. In Uncertainty in Artificial Intelligence.
PMLR, 2023.
Okoroafor, P., Kleinberg, R., and Kim, M. P. Near-optimal algorithms for omniprediction. arXiv preprint, 2025.
Perdomo, J. C., Zrnic, T., Mendler-Dünner, C., and Hardt, M. Performative prediction. International Conference on Machine Learning, 2020.
Perdomo, J. C., Britton, T., Hardt, M., and Abebe, R. Diffi- cult lessons on social prediction from wisconsin public schools. arXiv preprint, 2023.
Shawe-Taylor, J. and Cristianini, N. Basic kernels and kernel types, pp. 291-326. Cambridge University Press, 2004.
Silver, N. When we say 70 percent, it re- ally means 70 percent, 2019. URL https:
//fivethirtyeight. com/ features/ 1 perroettee.e.....
wwwp:eeaaa-------e---e-eee------ Accessed: 2025-05-28.
Simon, H. A. Bandwagon and underdog effects and the pos- sibility of election predictions. Public Opinion Quarterly, 1954.
Taori, R. and Hashimoto, T. Data feedback loops: Model- driven amplification of dataset biases. In International Conference on Machine Learning. 2023.
Vovk, V. Non-asymptotic calibration and resolution. Theo- retical computer science, 2007.
Vovk, V., Nouretdinov, I., Takemura, A., and Shafer, G. De- fensive forecasting for linear protocols. In International Conference on Algorithmic Learning Theory, 2005a.
Vovk, V., Takemura, A., and Shafer, G. Defensive forecast- ing. In International Workshop on Artificial Intelligence and Statistics, 2005b.
Wang, G., Acharya, K., Lakshmikanthan, L., Muthukumar, V., and Ziani, Multi-agent performative prediction beyond the insensitivity assumption: A case study for mortgage competition. arXiv preprint, 2025.

---

Revisiting the Predictability of Performative, Social Events
A. The K29 Algorithm
The K29 Algorithm
Input: kernel : (X' x 1])2 + R
For 1,2,
1. Given history {Tr Ps, y=) and current features za define Sa [0, 1] - R as
5:(p) = Σ k((x),)) (2:, (o), - -P:).
2. If (((1 2 predict pt 1. Else (s( predict pt O.
3. Else, run binary search to return pt E [0, 1] such that (p:e) = 0
Figure 2. At every round, the algorithm implicitly publishes function ft X -+ [0. 1] where predictions pt f(x,,) are chosen by solving simple optimization problem defined with respect to the history {(x; Pi, ys |= For simplicity, we state the algorithm with exact root finding. The main regret bound is still true, however, if one finds approximate root [S) S 1/poly(t). It degrades by an additive constant. The per round run time in ( time(()) where time(k) is an upper bound on kernel evaluation.
For the sake of completeness, we provide a short overview of the K29 algorithm from (Vovk et al., 2005a;b). It is a simple, kernel-based procedure that is hyperparameter free. It guarantees online multicalibration with respect to functions f in a Reproducing Kernel Hilbert Space F. We summarize its main guarantee below:
Proposition A.1 ((Vovk et al., 2005a)). Let k((2,, p), (x',p') be kernel that is continuous in p and let F be its associated RKHS F with norm || ||.- With probability 1, the predictions produced by the K29 algorithm in the online protocol (Definition 3.1) result in a transcript {(II Pt, Vt) satisfying,
fam. pcYl Ilfllr M(a,,))(a,, - p.)²
Proof. The proof uses basic facts about RKHS. By the reproducing property, function evaluation in the RKHS can be written as an inner product, (z = (f. (zz, p)),, where & x [o, 1] + an element F. Using linearity inner products and Cauchy-Schwarz,
the feature map for the RKHS and f is
T Σ (1 - p.)|= Σ (E) -p))
t=1
KIf Σ
2 (x p1) po))=l < ||
Σ 2 p.) Pe)lF.
(8)
We now focus bounding E D( (1)r, pc) 1.) By construction, the K29 algorithm always chooses a prediction
Pt such that:
sup ((v - pc) (t) Pt), Σ ((r -p:)))= sup (y - (p.) < vE(0.1]
vE[o.
Here, we used the fact that kernels represent inner products, k((x, p), (2',p)) = (1 p). (5 p')) to do the rewriting. To see why Equation (9) holds, note that S,(1) 2 then (y 1 (,) < for all E [0, 1]. An analogous fact holds for the
case where S,(0) S If neither of these true, then S,(1) and 5, have opposite signs, and binary search returns a pt
12

---

Revisiting the Predictability of Performative, Social Events
such that Se(e)) 0 (which exists by continuity of the kernel). Next, show that,
(z), - -PO - ( - -pt)
-p-)²
(10)
t=1
This follows by induction on the partial sums Vi EE 0ill for is e Pi) -p)),
Ve+1 Σ Us + Ue+ | =
Σ 0u + |1u4+111 +
Σ Us V(41)= S Vi 1|02+1113
Here, we used Equation (9) to upper bound the cross term above. The result follows by taking square roots either side of Equation (10) and plugging the upper bound into Equation (8).
To apply the algorithm to specific function classes H. the only remaining todo is to show how to construct kernels k with corresponding RKHS F such that H C F and to show that both kernel evaluations k((1, p). (z,p)) p)) and function norms ||ハ|] are uniformly bounded. This yields the (vVT) regret bound. See (Dwork et al., 2025) for examples on how to do this for common function classes.
B. Deferred Proofs
B.1. Proof of Theorem 3.4
Proof. The proof follows the typical template of online to batch conversions, with the modification that we now draw samples from D--) rather than a static distribution D.
Let {ri, Vi. fi. pi be the sequence of random variables generated in the n-round interaction (online to batch conversion) where Ja is chosen by the online algorithm A as a function of (æ', ys: Sa, p.)s)ss) and the data at round is generated from sampling process o; Dz, P; = f((z)) y (y pt). By definition of fA and the outcome performavity assumption on D(-).
Beeps)) v)
Σ Pan - v) ff fil -r|ff fil
f(( v) f-= = fll. n y (((..(())
E
(11)
where f; is again the predictor chosen by the online learning algorithm at round (which is measurable with respect to TT-1 ((1,, Pj, f,) 1=) and p; = (x) Now, consider the following stochastic process (V:)
V = Vi-1 + ((appa))-pE)le(e, ()(p v) fa = fi, mi c(x;, Pp) ys).
This is martingale since Vi is function of and
E[c(a; pp: v1) T:-i] ((...)) v) IA Sú, m-)
By assumption, Ic(æ, (p 2)1 < 1 for any v.)). Hence, the increments lie in [-2,2]. Using the Azuma-Hoefiding inequality, with probability 1 8:
7
V.l
Σ E B.aa) (p v) f f-1 c(r,, 91) S /8n log(2/6).
The above holds for specific and can have it hold for all E C via union bound. Now using the triangle inequality and our identity from Equation (11) we get that:
(ew.)--ptssr-reddd...l.l w)1 s _ Σ c(a,, w:)l 4v
log( Cl) + (/6)
for all c C. The result follows by upper bounding the first term on the right hand side by the regret bound on the online algorithm.
13

---

Revisiting the Predictability of Performative, Social Events
B.2. Proof of Corollary 3.5 Proof of a) Consider the kernel,
p((z,p), (z',p') (x),))))
Σ CEC
cl2, c(z,p))(x'.p))
For any p), k((x, p), ()) S |C] since c(r,p) [-1,1]. Furthermore, the kernel is continuous in p since all the functions are assumed to be continuous in p. By the Moose-Aronszajn theorem, k has an RKHS F such that JF for all E C. Furthermore, ||e]] < 1. Therefore, by Proposition A. the K29 algorithm guarantees online multicalibration with respect to all E C with regret bounded by /T. C.. Note that the kernel can be evaluated in time O(([).
Proof of b) It well-known fact that the linear kernel k((r, p), (x',p') (x. x²) + ppl has an reproducing kernel Hilbert space F& S {X x [o, 1] }} containing all linear functions c(x,p) (z, 0) + b p. Moreover, the squared norm of these functions in the RKHS is equal to ||01|2 + p.
Therefore, Proposition A.I guarantees that the K29 algorithm instantiated with this linear kernel is online multicalibrated with respect to C S FR at rate /2T. Since the kernel takes ((d) time to evaluate, the per-round runtime of the algorithm is at most (t dd.
Proof of c) The regret bound follows from the analysis in Corollary 3.3 from (Dwork al., 2025) which provides an explicit choice of kernel such that the AnyKer or K29 algorithms are guarantee online outcome indistinguishable with respect to all degree polynomial functions C at rate bounded by 10vd T. The (ANOVA) kernel in this construction can be computed in time at most Odd s) Thaaw--Tayoo & Cristianini, 2004), hence the bound on the runtime.
B.3. Proof of Lemma 4.4
Proof. Expanding out the first term the left-hand side,
E ((pp))
pr flt)) D.pp + E(p,0).
Furthermore,
E De p(.. EuwDa.p (e))(((p {(p, (e pl E(p, 0)]
Ber(p)(p)
=E f(r [[((p, 1) - E(p, 0))p ((p,
Combining these two equations with the observation that for squared loss, E(p, 1) - E(p, 0) 1/2 p. get that:
Bxn.D. - En-DD (pp = Pp (e(p. C(p. 0)) p)]
((p.v)) y-Ber(pp)
-BL-D 1/2) - p)).
An identical argument shows that, given any h X + [0, [0,11]
EzwD..p ((2) ((e))(())))))))))))))))))))))))) ((((x))))) y) B(e.))-() (h(xx,,))))))))))))))))))))))))) u) = +ir
p)]
BerBe)
Taking absolute values, we see that these conditions are exactly equal to the requirement that f is performatively multi- calibrated with respect the functions c(x.p) = p- 1/2 and (a, p) h(x) 1/2.
14


Reviews:
[{"review_id": "vjNJOKVIRf", "reviewer": "Reviewer_kMPD", "summary": "The paper at hand claims to explore the predictability of socil events. Social predictions do not merely describe the future. They also influence it. Such predictions can affect market prices, voter behavior, and policy outcomes. This interaction complicates the ability to forecast accurately. Early theorists, including Morgenstern and Simon, discussed these challenges extensively. This paper addresses these questions anew, using recent concepts from machine learning. Specifically, it utilizes the framework of \"performative prediction,\" where predictions themselves shape outcomes. The authors establish that accurate forecasting of binary social events remains computationally feasible despite these dynamics. A key contribution is demonstrating the existence of predictors that remain accurate even as they actively shape events. These predictors satisfy strong conditions like multicalibration and outcome indistinguishability. Algorithms to achieve these conditions are presented, which are efficient both statistically and computationally.\n\nHowever, the paper also identifies critical limitations. Although accurate predictions are always achievable, they might not always lead to desirable outcomes. Calibrated predictions may sometimes create poor social equilibria. Such predictions could, paradoxically, maximize prediction error measured in terms of performative risk. Thus, the paper shows a tension between accuracy and social desirability in forecasting social events. The authors conclude by suggesting a reconsideration of historical forecasting methods. They call for further research into what goals predictions in social contexts should ultimately serve.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to read and provide detailed feedback on our manuscript. It is very appreciated.\n\nRe: Brown et al. 2022. This is an excellent paper. We mistakenly did not include it but will happily discuss it in the revision.\n\nRe: Conceptual question. Thanks for raising this. Lipschitzness of the distribution map D() and strong convexity of the loss are distinct conditions. One does not imply the other. Also, as per the example in Miller et al ICML 2021, the bound from Thm 4.3 in Perdomo et al. can be vacuous in the sense that the other parameters (Lipschitz constant of the loss) involved make it so the bound is just the diameter of the space. Hence, even if the distribution map is lipschitz, it does not imply that stable and optimal classifiers are close in parameter space. Of course, none of these arguments apply in our setting since we make no regularity assumptions on D(). We thank you for raising it and will add further discussion on this point in the updated version.\n\nRe: Randomized predictions. Randomization is, in general, necessary to guarantee performative calibration as per the example in Section 5. In particular, no deterministic prediction p in [0,1] has the property that E_{p}[Y] = p. One needs to randomize between different forecasts to achieve on average calibration. Our comment about near determinism was regarding the per-time-step predictions on the online algorithms. We see how this can be confusing and will clarify it. Thanks for bringing it up. \n\nRe: Explanation. We will happily clarify what we meant regarding how martingale arguments developed in online to batch conversions for supervised learning settings are close to those we develop for the performative case. \n\nRe: Stateful world. This is a great question that we had not considered. We don’t believe our results apply directly to the stateful case. However, it is an interesting question for future work.", "claims_and_evidence": "All technical claims are proven. My only concern is with the general, conceptual claim of the paper as stated in the title \"Revisiting the Predictability of Social Events\". I understand the authors want to relate to the famous article by Grunberg and Modigliani, but I think the title does oversell the paper a little bit. The paper does not answer the questions whether social events are predictable *per se*. The paper answers the question whether social events can be predictable if these predictions have performative effects on the population. The framework is the very specific (although without the strong Lipschitz-condition on the distributions mapping) original performative prediction setup (without e.g. more realisitic stateful world as in https://proceedings.mlr.press/v151/brown22a/brown22a.pdf)", "methods_and_evaluation_criteria": "This is a theoretical paper, so no experiments are needed.", "theoretical_claims": "I did spend only 1-2 hours checking the proofs, but I did not find any errors.", "experimental_designs_or_analyses": "no experiments, see above.", "relation_to_broader_scientific_literature": "see above", "essential_references_not_discussed": "Discussion of https://proceedings.mlr.press/v151/brown22a/brown22a.pdf would be nice, since I consider this extension a much more realistic model of reality than the original performative prediction setup, see question below. But it is not strictly essential.", "other_strengths_and_weaknesses": "none", "other_comments_or_suggestions": "typos: \n\"Converstion\" → \"Conversion\"\n\"instatiating\" → \"instantiating\"\n\"indistiguishability\" → \"indistinguishability\"\n \"ℓ(p,y)=½(1−p)²\" should be \"ℓ(p,y)=½(y−p)²\"", "questions_for_authors": "1. conceptual question: I understand the results presented in this paper require much weaker conditions (e.g. no Lipschitzness of distribut. mapping, which in turn induces e.g. strong convexity of loss if I remember correctly?) than in the original performative prediction setup (2020 paper). So that's certainly a valid and substantial contributions of its own. In terms of conceptual results, however, I must admit that I do not find it surprising for a classifier to be multi-calibrated in a setup where it was shown before that it converges to a classifier that is close (in parameter space) to the performatively optimal one (thm. 4.3. in https://arxiv.org/pdf/2002.06673). Can the authors comment in more details on this relation?\n\n2. Can the authors provide some intuitions why the results require *randomized* predictors, while readers are \"encouraged to think of them as deterministic\"?\n\n3. For the main results (Thm. 3.4) the authors write: \"The main difference here is that samples (xt, yt) are not drawn i.i.d from a fixed distribution D, but rather from the distribution (xt, yt) ∼ D(ft) induced by the predictions. Despite these differences, a similar strategy suffices.\" The authors then procede by presenting the results. Can they explain WHY this strategy suffices? Generally, I think the paper could benefit from some more motivation/explanation\n\n4. Does Thm. 3.4 also hold for performative prediction in a stateful world (see reference above, i.e., in a setup where the distributions map is bivariate mapping from both predictions AND previous distributions)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8aJabafyD7", "reviewer": "Reviewer_vAMA", "summary": "* This paper investigates multicalibration problems in performative settings.\n* The models assume the performative prediction framework, where data distribution depends on the deployed model $(x,y)\\\\sim \\\\mathcal{D}(f)$.  \n* The main result is a convergence bound for the performative multicalibration loss, achieved through an online-to-batch reduction (Theorem 3.4). The result is modular, converting online learning algorithms with multicalibration guarantees into batch algorithms with performative multicalibration guarantees.   \n* Section 4 shows that predictors which are approximately performatively multicalibrated are also approximately performatively stable for the quadratic loss function. Finally, Section 5 presents a construction showing that perfectly multicalibrated classifiers can have the worst possible performance with respect to the quadratic loss.\n\n## Update after rebuttal\nThank you for the clarifications, and I look forward to seeing the improvements in the next revision of the paper.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate you taking the time to read and carefully critique our work. These are great questions. \n\nStateful world. We had not considered this possibility. We don’t believe it applies directly to the stateful case, but this is an interesting, open question for future work. \n\nAlso, given the additional space that comes with the revision, we’d happily include further discussion and comments to explain the main ideas behind the results and make the paper more accessible. We will also consider potential comparisons to previous algorithms and practical examples to illustrate how our ideas compare to previous approaches, with the caveat that previous approaches lack theoretical guarantees in our setting.", "claims_and_evidence": "Claims seem to be supported by theoretical evidence.", "methods_and_evaluation_criteria": "The paper does not contain an empirical evaluation.", "theoretical_claims": "The proofs were checked at a high level. While the high-level structure appears sound, a meticulous examination can help provide additional verification.", "experimental_designs_or_analyses": "The paper does not include any empirical experiments. Although this is common in theory-focused work, a simulated example or a practically motivated case study could help demonstrate the utility of the proposed framework.", "relation_to_broader_scientific_literature": "The paper positions itself against classic results in the performative prediction literature, which typically rely on strong Lipschitz conditions. Here, the authors show that performative multicalibration is achievable under milder assumptions on the distribution map $\\\\mathcal{D}$.", "essential_references_not_discussed": "Key results in this area seem to be discussed.", "other_strengths_and_weaknesses": "Additional strenghts:\n* The paper offers a novel convergence bound for performative multicalibration loss, providing a modular framework to bridge online and batch settings.\n\nAdditional weaknesses:\n* The exposition is challenging to follow in parts, which may hinder accessibility.\n* The absence of a practically motivated example or empirical demonstration limits the intuition behind the theoretical results, and the practical applicability of the findings.", "other_comments_or_suggestions": "* Including a practical example or simulation would help illustrate the value of the proposed method compared to existing work, and build intuition about its theoretical and practical behavior.\n* In addition, I wonder if the paper can benefit from an empirical demonstration of the results.", "questions_for_authors": "* Would the theoretical results still hold in a stateful performative prediction setting, such as the one considered in Brown et al.'s “Performative Prediction in a Stateful World” (AISTATS 2022)?\n* Can you provide an example or simulation that demonstrates the practical advantages of your approach over traditional methods under the performative prediction framework?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OgrJtWWHce", "reviewer": "Reviewer_Y8H3", "summary": "This paper formalizes the question of whether predictions can remain accurate when the act of predicting affects the state of the world. The authors address this question theoretically and show that a predictor can maintain some bounded level of calibration/validity. The paper provides a bound on how far a predictor might be from accuracy and show that high quality predictions can be found in polynomial time. The paper ends by showing that while the calibration criteria which has been used through the paper has very bad worst-case performance.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to carefully read our manuscript and provide comments. We’re delighted you find the direction interesting and look forward to more work in this area. \n\nWe will happily add more clarification and intuition on the distinctions between accuracy and calibration, the notation we use, and, more generally, the main mathematical tools used in our work. The camera-ready allows a page of more space, and we will use it for this purpose. We will also add a discussion section that succinctly summarizes the implications of our work and fix these typos and color issues in the figure.", "claims_and_evidence": "I have not found errors in the claims of the paper, however, I find the paper to be quite lacking in clarity. While the introduction does an excellent job of setting out some intuition as to the problem domain I find the explanation within the paper highly devoid of both (1) intuitive explanation of each idea being used (e.g. an example of what good [multi-]calibration means in practice, contrasted with something like accuracy would help my understanding), and (2) a clear explanation of all notation: there is some assumption of understanding around certain notation standards which are then not introduced in the paper. Basic definitions, such as the distinction between p and y needs to be spelled out more clearly to firmly establish a solid grounding that a reader will use for the rest of the paper.", "methods_and_evaluation_criteria": "The theoretical framework applied to the problem seems quite fitting.", "theoretical_claims": "Most proofs are not in the main paper but instead included in an appendix; I did not thoroughly review them.", "experimental_designs_or_analyses": "N/A", "relation_to_broader_scientific_literature": "There is discussion around the work most directly related to this topic. The paper does not include a concluding discussion section which might add some broader connection to other areas of the scientific community.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "In general, I find the topic of this paper to be excellent and in a direction that I truly hope is developed much further. However, I find this paper to be quite inaccessible due to a lack of both thorough explanation of the basics of the paper, and a lack of examples/intuition around how to understand the concepts.\n\nThe paper suffers as a result, both in terms of clarity and any comment I can add on the significance of the paper. A section more clearly discussing the implications of your results would help me to much better understand where this paper fits in and how important it is to the literature.", "other_comments_or_suggestions": "Some minor notes:\n\n- typo in Definition 1: \"to a distributions over\"\n- the blue line in Figure 1 is not at all friendly to the colour-blind (or any reader that prints out papers in black and white, like myself). A different line style for that segment may be worth the effort", "questions_for_authors": "I rarely update my review based on a response but you are welcome to respond to any portion of my review as you wish.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vypiDRkh9V", "reviewer": "Reviewer_QnUX", "summary": "The authors consider a classic problem in social science--how can we make accurate predictions about the world if our predictions affect the world--from a learning theory perspective. In this setting, there  are features $x$, a binary outcome $y$, and we wish to make probabilistic predictions $f(x)$ to predict $y$. However, once we pick $f$, the world changes so that $(x, y) \\sim \\mathcal D(f)$, and outcomes now depend on our predictions in an arbitrary way (the performative prediction setting). Classic work using fixed-point analysis shows that there exist predictors $f$ such that $f(x)$ predicts $y$ when $(x, y)\\sim \\mathcal D(f)$, but does not provide a method of computing such predictors. Here, the authors adapt online multicalibrated algorithms to performative prediction. They show that it is indeed possible to efficiently learn a predictor $f(x)$ that is (multi)calibrated (on whichever subsets of $x$ we care about) with the induced outcomes drawn from $\\mathcal D(f)$. Moreover, they show a connection to outcome indistinguishability, so that their predictor's outputs are computationally indistinguishable from the world outcomes. On the other hand, through a construction, the authors also demonstrate that the learned predictor can utterly fail to explain the variance in $y$, despite being perfect calibrated. Intuitively, they show that in some cases, the only way to achieve calibration is to steer the world into higher variance outcomes, where are predictions are essentially useless (despite being calibrated). In contrast, if we were willing to accept some small bias in our predictions in this construction, we could steer the world into highly predictable outcomes (consistently y=0 or y=1).\n\n### Update after rebuttal\nI continue to think this is a very nice paper and I'm glad the other reviews agree on acceptance.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for all of the insightful comments on our work! We’re glad you found the paper interesting. \n\nWe will certainly add intuition behind the construction in section 5 to the introduction. Your comments in the summary of your review are very helpful in this regard. We appreciate it!\n\nAnd yes, thanks for catching those typos; we will fix them.", "claims_and_evidence": "The theoretical claims are thoroughly supported by proofs, including sketches in the main text and extended proofs in the appendix.", "methods_and_evaluation_criteria": "N/A", "theoretical_claims": "I am confident that the analysis in section 5 is correct. The other proofs sound reasonable, although I have not extensively checked the details.", "experimental_designs_or_analyses": "N/A", "relation_to_broader_scientific_literature": "The key contribution is a method for constructing a calibrated performative predictor, and analysis of why this might not be the best measure for social prediction.", "essential_references_not_discussed": "None that I am aware of, although this is not my primary area of expertise. However, the paper in many places acknowledges where the ideas it draws from originated, so my impression is that the authors are very familiar with the relevant related work.", "other_strengths_and_weaknesses": "Strengths:\n1. The quality of the paper and writing are extremely high.\n2. The problem is a very important one, and from my understanding this paper provides a very nice contribution to the literature \n\nWeaknesses:\n1. In some cases, I found the subtle differences between equations and definitions difficult to follow, so I think there could be just a bit more intuition provided (see suggestion below for the main one)", "other_comments_or_suggestions": "1. The construction from section 5 is very nice and easy to understand, and makes the distinction between performative stability and performative optimality much clearer than the definitions and Eqs (1) and (2). I would recommend describing the intuition behind this construction in 1.1, so the reader doesn't have my experience: thinking \"how can this possibly be true, I don't understand how (1) and (2) lead to such drastic differences\" until they reach the last page of the paper and it clicks.", "questions_for_authors": "1. Some places have $(x, y)\\sim \\mathcal D (f)$ (e.g., line 417) while others have $y \\sim \\mathcal D (f)$ (e.g., line 421), and yet others have $x \\sim \\mathcal D (f)$ (e.g., line 381). All of those cases seem to use both $x$ and $y$, so surely they should all say $(x, y)\\sim \\mathcal D (f)$, right?\n2. Along the same lines, some places say $p \\sim f$ (line 417) and others say $p \\sim f(x)$. Shouldn't it always be $p \\sim f(x)$?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper studies the predictability of social event within a modern performative prediction setting. The paper theoretically establishes positive results for the binary case within this setting, while highlighting its limitations. Authors were generally positive about the paper, were appreciative of the aim of recreating classic results in contemporary settings, and found the theoretical results to be sound and interesting. Some reviewers encourage the authors to better frame their contribution in the precise setting in which results hold, namely stateful performative prediction, rather than a general thesis about the predictability of social events in general. Statefulness in particular seems to be a common simplifying assumption in the performative prediction literature, but that does not necessary apply broadly. Several reviewers found clarity to be improvable on several fronts – the authors are encourage to take these into account in their next revision. Adding a final discussion section is also advisable as it is the norm for almost all ICML papers.
Decision: Accept (poster)