SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Wenjing Yan Xuanyu Cao *
Department of Electronic and Computer Engineering
wj.yan@connect.ust.hk, eexcao@ust.hk
Performative prediction is a recently proposed framework where predictions guide decision-making and hence influence future data distributions. Such performative phenomena are ubiquitous in various areas, such as transportation, finance, public policy, and recommendation systems. To date, work on performative prediction has only focused on unconstrained scenarios, neglecting the fact that many real-world learning problems are subject to constraints. This paper bridges this gap by studying performative prediction under inequality constraints. Unlike most existing work that provides only performative stable points, we aim to find the optimal solutions. Anticipating performative gradients is a challenging task, due to the agnostic performative effect on data distributions. To address this issue, we first develop a robust primal-dual framework that requires only approximate gradients up to a certain accuracy, yet delivers the same order of performance as the stochastic primal-dual algorithm without performativity. Based on this framework, we then propose an adaptive primal-dual algorithm for location family. Our analysis demonstrates that the proposed adaptive primal-dual algorithm attains  $\mathcal{O}(\sqrt{T})$  regret and constraint violations, using only  $\sqrt{T} + 2T$  samples, where  $T$  is the time horizon. To our best knowledge, this is the first study and analysis on the optimality of the performative prediction problem under inequality constraints. Finally, we validate the effectiveness of our algorithm and theoretical results through numerical simulations.
Stochastic optimization plays a critical role in statistical sciences and data-driven computing, where the goal is to learn decision rules (e.g., classifiers) based on limited samples that generalize well to the entire population. Most prior studies on stochastic optimization [Heyman and Sobel, 2004; Karimi et al., 2019; Powell, 2019] rely on the assumption that the data of the entire population follows a static distribution. This assumption, however, does not hold in applications where the data distributions change dynamically in response to decision-makers' actions [Hardt et al., 2016; Dong et al., 2018]. For instance, in transportation, travel time estimates [Mori et al., 2015] influence routing decisions, resulting in realized travel times; in banking, credit evaluation criteria [Abdou and Pointon, 2011] guide borrowers' behaviors and subsequently their credit scores; and in advertising, recommendations [García-Sánchez et al., 2020] shape customer preferences, leading to consumption patterns. Such interplay between decision-making and data distribution arises widely in various areas, such as transportation, finance, public policy, and recommendation systems.
The seminal work [Perdomo et al., 2020] formalized the phenomenon as performative prediction, which represents the strategic responses of data distributions to the taken decisions via decision-
dependent distribution maps [Quinonero-Candela et al., 2008]. Since then, an increasing body of research has been dedicated to performative prediction problems. Most existing studies are focused on identifying performative stable points [Li and Wai, 2022; Li et al., 2022; Drusvyatskiy and Xiao, 2022; Brown et al., 2022; Mendler-Dünner et al., 2020; Wood et al., 2021; Ray et al., 2022], given the complexities of the decision-induced distribution shifts and the unknown decision-dependent distributions. The proposed algorithms typically iteratively retrain the deployed models until convergence. However, performative stability generally does not imply performative optimality. Aiming to achieve optimal performance, a few recent works designed effective algorithms by leveraging rich performative feedback [Jagadeesan et al., 2022], or by making some parametric assumptions on the underlying distribution maps. For instance, the distribution maps belong to the location family with linear structure [Miller et al., 2021] or are from the exponential family [Izzo et al., 2021].
All the aforementioned work on performative prediction is focused on unconstrained learning problems. However, in the real world, many performative prediction applications are subject to constraints [Detassis et al., 2021; Wood and Dall'Anese, 2022a]. Constraints can be used to ensure the satisfaction of desired properties, such as fairness, safety, and diversity. Examples include safety and efficiency constraints in transportation [Metz, 2021], relevance and diversity constraints in advertising [Khamis, 2020], and risk tolerance and portfolio constraints in financial trading [Föllmer and Schied, 2002], etc. In addition, constraints can serve as side information to enhance the learning outcomes, e.g., by narrowing the scope of exploration or by incorporating prior knowledge Serafini and Garcez [2016]; Wu et al. [2018]. As performative shifts can rarely be analyzed offline, incorporating constraints on what constitutes safe exploration [Turchetta et al., 2019] or facilitates optimization [Wood and Dall'Anese, 2022b] is of crucial importance.
Despite its importance, research on performative prediction under constraints has been neglected so far. Although some work [Izzo et al., 2021; Piliouras and Yu, 2022] restricted decision variables to certain regions, this feasible set restriction was simply handled by projections. This paper bridges this gap by studying the performative prediction problem under inequality constraints, for which simple projection is inadequate to handle. Unlike most existing work that provides only performative stable points, we aim to find the optimal solutions. As aforementioned, finding performative optima is a challenging task because we now need to anticipate performative effect actively rather than simply retrain models in a myopic manner.
However, the performative effect on distribution maps is unknown, which hinders the computation of exact performative gradient. To solve this problem, we develop a robust primal-dual framework that admits inexact gradients. We ask the following questions: How does the gradient approximation error affect the performance of the primal-dual framework? Under what accuracy can the approximate gradients maintain the performance order of the stochastic primal-dual algorithm without performativity? How to construct effective gradient approximations that attain the desired accuracy? We answer the above thoroughly. Our idea hinges on enhancing gradient approximation with the parametric knowledge of distribution maps. In particular, we follow existing studies [Miller et al., 2021; Jagadeesan et al., 2022] and focus on the family of location maps. Location family exhibits a favorable linear structure for algorithm development while maintaining broad generality to model many real-world applications. Distribution maps of this type are ubiquitous throughout the performative prediction literature, such as strategic classification [Hardt et al., 2016; Perdomo et al., 2020], linear regression [Miller et al., 2021], email spam classification [Li et al., 2022], ride-share [Narang et al., 2022], among others. Nevertheless, we emphasize that our robust primal-dual framework is applicable to other forms of distributions with effective gradient approximation methods.
To our best knowledge, this paper provides the first study and analysis on the optimality of performative prediction problems under inequality constraints. We highlight the following key contributions:
- We develop a robust primal-dual framework that requires only approximate gradients up to an accuracy of  $\mathcal{O}(\sqrt{T})$ , yet delivers the same order of performance as the stochastic primal-dual algorithm without performativity, where  $T$  is the time horizon. Notably, the robust primal-dual framework does not restrict the approximate gradients to be unbiased and hence offers more flexibility to the design of gradient approximation.
- Based on this framework, we propose an adaptive primal-dual algorithm for location family which consists of an online stochastic approximation and an offline parameter estimation for the performative gradient approximation. Our analysis demonstrates that the proposed algorithm achieves  $\mathcal{O}(\sqrt{T})$  regret and constraint violations, using only  $\sqrt{T} + 2T$  samples
Finally, we conduct experiments on two examples: multi-task linear regression and multi-asset portfolio. The numerical results validate the effectiveness of our algorithm and theoretical analysis.
The study on performative prediction was initiated in [Perdomo et al., 2020], where the authors defined the notion of performative stability and demonstrated how performative stable points can be found through repeated risk minimization and stochastic gradient methods. Since then, substantial efforts have been dedicated to identifying performative stable points in various settings, such as single-agent [Mendler-Dünner et al., 2020; Drusvyatskiy and Xiao, 2022; Brown et al., 2022], multiagent [Li et al., 2022; Piliouras and Yu, 2022], games [Narang et al., 2022], reinforcement learning [Mandal et al., 2023], and online learning [Wood et al., 2021; Wood and Dall'Anese, 2022a].
A few recent works aimed to achieve performative optimality, a more stringent solution concept than performative stability. In [Miller et al., 2021], the authors evaluated the conditions under which the performative problem is convex and proposed a two-stage algorithm to find the performative optima for distribution maps in location family. Another paper on performative optimality is [Izzo et al., 2021], which proposed a PerfGD algorithm by exploiting the exponential structure of the underlying distribution maps. Both works took advantage of parametric assumptions on the distribution maps. Alternatively, [Jagadeesan et al., 2022] proposed a performative confidence bounds algorithm by leveraging rich performative feedback, where the key idea is to exhaustively explore the feasible region with an efficient discarding mechanism.
A closely related work is [Wood and Dall'Anese, 2022b], which studied stochastic saddle-point problems with decision-dependent distributions. The paper focused on performative stable points (equilibrium points), whereas we aim at the performative optima, which is more challenging. Another difference is that [Wood and Dall'Anese, 2022b] only demonstrated the convergence of the proposed primal-dual algorithm in the limit, without providing an explicit finite-time convergence rate. In contrast, we provide  $\mathcal{O}(\sqrt{T})$  regret and  $\mathcal{O}(\sqrt{T})$  constraint violation bounds for the proposed algorithm in this paper.
We study a performative prediction problem with loss function  $\ell(\theta; Z)$ , where  $\theta \in \Theta$  is the decision variable,  $Z \in \mathbb{R}^k$  is an instance, and  $\Theta \in \mathbb{R}^d$  is the set of available decisions. Different from in stationary stochastic optimization where distributions of instances are fixed, in performative prediction, the distribution of  $Z$  varies with the decision variable  $\theta$ , represented by  $Z \sim \mathcal{D}(\theta)$ . In this paper, we consider that the decision variable  $\theta$  is subject to a constraint  $\mathbf{g}(\theta) \preceq \mathbf{0}$ , where  $\mathbf{g}(\cdot): \Theta \to \mathbb{R}^m$ . The constraint  $\mathbf{g}(\cdot)$  can be imposed on  $\theta$  to ensure certain properties, such as fairness, safety, and diversity, or to incorporate prior knowledge. We assume that  $\mathbf{g}(\cdot)$  is available at the decision-maker in advance of the optimization. Ideally, the goal of the decision-maker is to solve the following stochastic problem:
$$
\min  _ {\boldsymbol {\theta} \in \boldsymbol {\Theta}} \quad \mathbb {E} _ {Z \sim \mathcal {D} (\boldsymbol {\theta})} \ell (\boldsymbol {\theta}; Z) \quad \text {s . t .} \quad \mathbf {g} (\boldsymbol {\theta}) \preceq \mathbf {0}, \tag {1}
$$
where  $\mathbb{E}_{Z\sim \mathcal{D}(\pmb {\theta})}\ell (\pmb {\theta};Z)$  is referred to as performative risk, denoted by  $\mathrm{PR}(\pmb {\theta})$
Problem (1) is, however, impossible to be solved offline because the distribution map  $\mathcal{D}(\pmb{\theta})$  is unknown. Instead, the decision-maker needs to interact with the environment by making decisions to explore the underlying distributions. Given the online nature of this task, we measure the loss of a sequence of chosen decisions  $\pmb{\theta}_1,\dots ,\pmb{\theta}_T$  by performative regret, defined as
$$
\operatorname {R e g} (T) := \sum_ {t = 1} ^ {T} \left(\mathbb {E} \left[ \operatorname {P R} \left(\boldsymbol {\theta} _ {t}\right) \right] - \operatorname {P R} \left(\boldsymbol {\theta} _ {\mathrm {P O}}\right)\right),
$$
where the expectation is taken over the possible randomness in the choice of  $\{\pmb{\theta}_t\}_{t=1}^T$ , and  $\pmb{\theta}_{\mathrm{PO}}$  is the performative optimum, defined as
$$
\boldsymbol {\theta} _ {\mathrm {P O}} \in \arg \min  _ {\boldsymbol {\theta} \in \boldsymbol {\Theta}} \quad \mathbb {E} _ {Z \sim \mathcal {D} (\boldsymbol {\theta})} \ell (\boldsymbol {\theta}; Z) \quad \text {s . t .} \quad \mathbf {g} (\boldsymbol {\theta}) \preceq \mathbf {0}.
$$
Performative regret measures the suboptimality of the chosen decisions relative to the performative optima. Another performance metric for problem (1) on evaluating the decision sequence  $\{\pmb{\theta}_t\}_{t=1}^T$  is
constraint violation, given by
$$
\operatorname {V i o} _ {i} (T) := \sum_ {t = 1} ^ {T} \mathbb {E} \left[ g _ {i} \left(\boldsymbol {\theta} _ {t}\right) \right], \forall i \in [ m ],
$$
where we use the symbol  $[m]$  to represent the integer set  $\{1, \dots, m\}$  throughout this paper.
Applications pertaining to problem (1) are ubiquitous. Next is an example.
Example 1 (Multi-Asset Portfolio). Consider a scenario where an investor wants to allocate his/her investment across a set of  $l$  assets, such as stocks, bonds, and commodities. The objective is to maximize the expected return subject to certain constraints, including liquidity, diversity, and risk tolerance. Let  $z_{i}$  denote the rate of return of the  $i$ th asset and  $\theta_{i}$  denote its weight of allocation,  $\forall i \in [l]$ . The investment can affect the future rates of return of the assets and, consequently, the overall expected return of the portfolio. For example, excessive investment in a particular asset may lead to a decline in the rate of return of other assets. Let  $\mathbf{z} = [z_{1},\dots ,z_{l}]^{\top}$  and  $\pmb {\theta} = [\theta_1,\dots ,\theta_l]^{\top}$ . Then, the expected return of the portfolio is  $\mathbb{E}[r_p] \coloneqq \mathbb{E}_{\mathbf{z}\sim \mathcal{D}(\pmb {\theta})}\mathbf{z}^{\top}\pmb{\theta}$ . Typically, the risk of the portfolio is measured by the variance of its returns, given by  $\pmb{\theta}^{\mathrm{T}}\pmb{\Psi}\pmb{\theta}$ , where  $\Psi$  is the covariance matrix of  $\mathbf{z}$ . One common approach to model liquidity is using the bid-ask spread, which measures the gap between the highest price a buyer is willing to pay (the bid) and the lowest price a seller is willing to accept (the ask) for a particular asset. Denote the vector of the bid-ask spread of the  $l$  assets by  $\mathbf{s} = [s_1,\dots ,s_l]^{\top}$ . Then, a liquidity constraint on the portfolio can be defined as  $\mathbf{s}^{\top}\pmb{\theta} \leq S$ , where  $S$  is the maximum allowable bid-ask spread. The multi-asset portfolio problem can be formulated as:
$$
\min  _ {\boldsymbol {\theta}} - \mathbb {E} _ {\mathbf {z} \sim \mathcal {D} (\boldsymbol {\theta})} \mathbf {z} ^ {\top} \boldsymbol {\theta} \quad \text {s . t .} \sum_ {i = 1} ^ {l} \theta_ {i} \leq 1, \mathbf {0} \preceq \boldsymbol {\theta} \preceq \epsilon \cdot \mathbf {1}, \mathbf {s} ^ {\top} \boldsymbol {\theta} \leq S, a n d \boldsymbol {\theta} ^ {\mathrm {T}} \boldsymbol {\Psi} \boldsymbol {\theta} \leq \rho ,
$$
where  $\epsilon$  restricts the maximum amount of investment to one asset, and  $\rho$  is the risk tolerance threshold.
In this paper, our goal is to design an online algorithm that achieves both sublinear regret and sublinear constraint violations with respect to the time horizon  $T$ , i.e.,  $\mathrm{Reg}(T) \leq o(T)$  and  $\mathrm{Vio}_i(T) \leq o(T)$ , for all  $i \in [m]$ . Then, the time-average regret satisfies  $\mathrm{Reg}(T) / T \leq o(1)$ , and the time-average constraint violations satisfy  $\mathrm{Vio}_i(T) / T \leq o(1)$ , for all  $i \in [m]$ . Both asymptotically go to zero as  $T$  goes to infinity. Therefore, the performance of the decision sequence  $\{\pmb{\theta}_t\}_{t=1}^T$  generated by the algorithm approaches that of the performative optimum  $\pmb{\theta}_{\mathrm{PO}}$  as  $T$  goes to infinity.
3 Adaptive Primal-Dual Algorithm
3.1 Robust Primal-Dual Framework
In this subsection, we develop a robust primal-dual framework for the performative prediction problem under inequality constraints. Our approach involves finding a saddle point for the regularized Lagrangian of problem (1). The Lagrangian, denoted by  $\mathcal{L}(\theta ,\lambda)$ , is defined as
$$
\mathcal {L} (\boldsymbol {\theta}, \boldsymbol {\lambda}) := \operatorname {P R} (\boldsymbol {\theta}) + \boldsymbol {\lambda} ^ {\top} \mathbf {g} (\boldsymbol {\theta}) - \frac {\delta \eta}{2} \| \boldsymbol {\lambda} \| _ {2} ^ {2}, \tag {2}
$$
where  $\theta$  is the primal variable (decision),  $\lambda$  is the dual variable (multiplier),  $\eta > 0$  is the stepsize of the algorithm, and  $\delta > 0$  is a control parameter. In (2), we add the regularizer  $-\frac{\delta\eta}{2}\|\lambda\|_2^2$  to suppress the growth of the multiplier  $\lambda$ , so as to improve the stability of the algorithm.
To find the saddle point of the Lagrangian  $\mathcal{L}(\theta, \lambda)$ , we utilize alternating gradient update on the primal variable  $\theta$  and the dual variable  $\lambda$ . The gradients of  $\mathcal{L}(\theta, \lambda)$  with respect to  $\theta$  and  $\lambda$  are respectively given by
$$
\nabla_ {\boldsymbol {\theta}} \mathcal {L} (\boldsymbol {\theta}, \boldsymbol {\lambda}) = \nabla_ {\boldsymbol {\theta}} \operatorname {P R} (\boldsymbol {\theta}) + \nabla_ {\boldsymbol {\theta}} \mathbf {g} (\boldsymbol {\theta}) ^ {\top} \boldsymbol {\lambda}, \tag {3}
$$
$$
\nabla_ {\boldsymbol {\lambda}} \mathcal {L} (\boldsymbol {\theta}, \boldsymbol {\lambda}) = \mathbf {g} (\boldsymbol {\theta}) - \delta \eta \boldsymbol {\lambda},
$$
where  $\nabla_{\theta}\mathbf{g}(\pmb {\theta})$  is the Jacobian matrix of  $\mathbf{g}(\cdot)$ . In (3),  $\nabla_{\theta}\mathrm{PR}(\pmb {\theta})$  is the gradient of the performative risk  $\mathrm{PR}(\pmb {\theta})$ , given by
$$
\nabla_ {\boldsymbol {\theta}} \Pr (\boldsymbol {\theta}) = \mathbb {E} _ {Z \sim \mathcal {D} (\boldsymbol {\theta})} \nabla_ {\boldsymbol {\theta}} \ell (\boldsymbol {\theta}; Z) + \mathbb {E} _ {Z \sim \mathcal {D} (\boldsymbol {\theta})} \ell (\boldsymbol {\theta}; Z) \nabla_ {\boldsymbol {\theta}} \log p _ {\boldsymbol {\theta}} (Z), \tag {4}
$$
where  $p_{\pmb{\theta}}(Z)$  is the density of  $\mathcal{D}(\pmb{\theta})$
Since the data distribution  $\mathcal{D}(\pmb{\theta})$  is unknown, the exact gradient of the performative risk  $\mathrm{PR}(\pmb{\theta})$  is unavailable, posing a significant challenge to the algorithm design. In this paper, we tackle this
issue using a robust primal-dual framework. The main idea is to construct gradient approximations from data and then perform alternating gradient updates based on the inexact gradients. Denote by  $\nabla_{\theta}\widehat{\mathrm{PR}}_t(\pmb {\theta})$  the approximation of the gradient  $\nabla_{\theta}\mathrm{PR}(\pmb {\theta})$  at the  $t$ th iteration. Correspondingly, an approximation for the Lagrangian gradient  $\nabla_{\theta}\mathcal{L}(\pmb {\theta},\pmb {\lambda})$  at the  $t$ th iteration is given by
$$
\nabla_ {\boldsymbol {\theta}} \widehat {\mathcal {L}} _ {t} (\boldsymbol {\theta}, \boldsymbol {\lambda}) := \nabla_ {\boldsymbol {\theta}} \widehat {\operatorname {P R}} _ {t} (\boldsymbol {\theta}) + \nabla_ {\boldsymbol {\theta}} \mathbf {g} (\boldsymbol {\theta}) ^ {\top} \boldsymbol {\lambda}, \forall t \in [ T ].
$$
The robust alternating gradient update is then performed as
$$
\boldsymbol {\theta} _ {t + 1} = \Pi_ {\boldsymbol {\Theta}} \left(\boldsymbol {\theta} _ {t} - \eta \nabla_ {\boldsymbol {\theta}} \widehat {\mathcal {L}} _ {t} \left(\boldsymbol {\theta} _ {t}, \boldsymbol {\lambda} _ {t}\right)\right), \tag {5}
$$
$$
\boldsymbol {\lambda} _ {t + 1} = \left[ \boldsymbol {\lambda} _ {t} + \eta \nabla_ {\boldsymbol {\lambda}} \mathcal {L} _ {t} \left(\boldsymbol {\theta} _ {t}, \boldsymbol {\lambda} _ {t}\right) \right] ^ {+}. \tag {6}
$$
Then, the next question is how to construct effective gradient approximations that achieve satisfactory performance.
By (4), the expectation over  $\mathcal{D}(\theta)$  in the gradient  $\nabla_{\theta}\mathrm{PR}(\theta)$  can be approximated by samples, while the unknown probability density  $p_{\theta}(Z)$  presents the main challenge. Most existing research circumvented this problem by omitting the second term in  $\nabla_{\theta}\mathrm{PR}(\theta)$ . This essentially gives a performative stable point. However, as pointed out in [Miller et al., 2021], performative stable points can be arbitrarily sub-optimal, leading to vacuous solutions. Instead, if we have further knowledge about the parametric structure of  $p_{\theta}(Z)$ , the complexity of gradient approximation can be greatly reduced. In this regard, [Miller et al., 2021] and [Jagadeesan et al., 2022] exploited the linear structure of location family, and [Izzo et al., 2021] considered distribution maps within exponential family. Following [Miller et al., 2021; Jagadeesan et al., 2022], we focus on the family of location maps in this paper because it exhibits a favorable linear structure for algorithm development while maintaining broad generality to various applications. Next, we develop an adaptive algorithm for problem (1) with location family distribution maps based on the above robust primal-dual framework.
3.2 Algorithm Design for Location family
In the setting of location family, the distribution map depends on  $\theta$  via a linear shift, i.e.
$$
Z \sim \mathcal {D} (\boldsymbol {\theta}) \Leftrightarrow Z \stackrel {d} {=} Z _ {0} + \mathbf {A} \boldsymbol {\theta}, \tag {7}
$$
where  $Z_0 \sim \mathcal{D}_0$  is a base component representing the data without performativity,  $\mathbf{A} \in \mathbb{R}^{k \times d}$  captures the performative effect of decisions, and  $\frac{d}{\theta}$  means equal in distribution. Denote by  $\pmb{\Sigma}$  the covariance matrix of the base distribution  $\mathcal{D}_0$ . Note that  $\mathcal{D}_0$  is still unknown. Plugging the distribution definition (7) into (4), we obtain a more explicit expression for  $\nabla_{\theta} \mathrm{PR}(\pmb{\theta})$  as
$$
\nabla_ {\boldsymbol {\theta}} \operatorname {P R} (\boldsymbol {\theta}) = \mathbb {E} _ {Z _ {0} \sim \mathcal {D} _ {0}} \left[ \nabla_ {\boldsymbol {\theta}} \ell (\boldsymbol {\theta}; Z _ {0} + \mathbf {A} \boldsymbol {\theta}) + \mathbf {A} ^ {\top} \nabla_ {Z} \ell (\boldsymbol {\theta}; Z _ {0} + \mathbf {A} \boldsymbol {\theta}) \right].
$$
To compute  $\nabla_{\theta}\mathrm{PR}(\pmb {\theta})$  , we still need to address two problems: the unknown base distribution  $\mathcal{D}_0$  and the unknown performative parameter A. We tackle them as follows.
Offline Stochastic Approximation: We approximate the base distribution  $\mathcal{D}_0$  offline by sample average approximation [Kleywegt et al., 2002]. Specifically, before the start of the alternating gradient update, we first draw  $n$  samples  $\{Z_{0,i}\}_{i = 1}^{n}$  from  $\mathcal{D}(\mathbf{0})$ . These samples are used to approximate the expectation over  $Z_{0}$  throughout the algorithm iteration. Hence, the sample complexity from this expectation approximation is fixed at  $n$ .