# Noncooperative Games with Prospect Theoretic Preferences

Marta Fochesato, Frédy Pokou, Hélène Le Cadre, John Lygeros

► To cite this version:

Marta Fochesato, Frédy Pokou, Hélène Le Cadre, John Lygeros. Noncooperative Games with Prospect Theoretic Preferences. IEEE Control Systems Letters, 2025, 9 (70), pp.7125-7140. hal-04866629v5

HAL Id: hal-04866629

https://hal.science/hal-04866629v5

Submitted on 13 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Noncooperative Games with Prospect Theoretic Preferences

Extended version

Marta Fochesato†, Frédy Pokou‡, Hélène Le Cadre‡, and John Lygeros†

†Automatic Control Laboratory, ETH Zürich, 8092 Zürich, Switzerland,
{mfochesato,jlygeros}@ethz.ch

‡Inria, CNRS, University of Lille, Centrale Lille, UMR 9189 CRISTAL, F-59000 Lille, France,
{fredy-vale-manuel.pokou, helene.le-cadre}@inria.fr.

## Abstract

We study noncooperative games with uncertain payoffs, where agents display irrational behaviors in response to underlying risk factors. Our formulation incorporates prospect theory, a behavioral model used to describe agents’ risk attitude, into the equilibrium theory of noncooperative *N*-agent games. We show that the resulting nonconvex nonsmooth game admits equilibria and provide convergence guarantees for their computation. Further, the concept of “Price of Irrationality” is introduced to quantify the suboptimality induced by irrational behaviors. Finally, we provide bounds on the performance of a class of prospect theoretic aggregative games and illustrate our results on an electricity market game involving strategic end users exposed to risk.

## 1 Introduction

Game theory has been used in many fields to model and analyze the outcomes of conflicts involving strategic agents in competitive scenarios [1]. When the outcomes of the game are inherently uncertain, Expected Utility Theory (EUT) [2] has served as a framework for decision-making under uncertainty, with the agent choosing the action that maximizes the expectation across all possible outcomes. While conceptually and often mathematically simple, this *rational* behavior has been found lacking by simple experiments in the case of humans as decision makers [3]. Considering the growing trend towards integrated socio-techno-economic networks and human-in-the-loop systems, incorporating *irrational* behaviors – arising from the risk preferences of the agents – into game-theoretic models, is therefore highly relevant. Ultimately, understanding how agents with different risk preferences choose their actions in multi-agent settings can better inform mechanism designers, guiding them on how to effectively steer agents towards improved collective and individual outcomes.

To this end, Prospect Theory (PT), an empirically-grounded model initiated by [3] to describe how irrational agents assign values to different risky options, offers a toolkit to equip game theory with behavioral insights. According to PT, individuals measure loss or gain of utility relative to a reference, and exhibit risk-seeking behavior under losses and risk-averse behavior under gains – a behavior known as “diminishing sensitivity”. Furthermore, the decrease in utility under a loss is typically greater than the increase in utility under a gain of the same magnitude – a behavior known as “loss aversion”. As a result, prospect theoretic utility functions account for systematic (and experimentally observed) deviations in human behavior from the predictions of the classical EUT framework, i.e., biases.

This work seeks to incorporate prospect theory into the equilibrium theory of noncooperative *N*-agent games. While prospect theoretic preferences strictly generalize EUT preferences, working with PT-based game theoretical formulations is challenging due to the possible lack of convexity and smoothness of the PT profit functions. As a consequence, many results regarding equilibrium definitions, existence, and algorithm design are inapplicable [4]. Here, we focus on PT-based games where each agent’s reference is exogenously given. Our contribution is threefold:
1. **Equilibria characterization and computation.** Based on a novel notion of equilibrium, we establish conditions for the existence and computation of equilibrium for PT games, relying on a recently introduced generalization of the Clarke Jacobian. Notably, this result is of independent interest and extends beyond the prospect theoretic setting to the general class of continuous games with nonconvex nonsmooth utility functions.

2. **Effect of irrationality.** Drawing inspiration from the popular price of anarchy, we introduce the concept of Price of Irrationality (PoI) to quantify how the efficiency of the system degrades due to the irrational behavior of the agents.

3. **Application to aggregative games.** We analytically characterize the impact of prospect theoretic preferences on the class of aggregative games with separable profit functions. Further, we corroborate our findings on an electricity market case study involving strategic end users exposed to a two-part tariff.

## 1.1 Related Works

*Prospect-theoretic preferences in games.* The theoretical analysis of prospect theory to model decision-making in multi-agent settings is relatively recent. Prospect theoretic preferences in noncooperative games first appeared in the economics literature, with [5–7] introducing new equilibrium concepts for finite games in normal form. The approach however does not carry over to the continuous N-agent setting considered in this work. In particular, [5] introduced “loss aversion equilibrium” in strategic games where agents exhibit endogenous references and loss aversion behaviors. More recently, [8] introduced several notions of equilibria in strategic games with finite action sets when players have prospect theoretic risk preferences, showing that the resulting equilibrium concept often explains observed experimental behaviors more closely than its EUT counterpart. Building up from these seminal works, several articles have examined the effect of prospect theoretic preferences in specific classes of game outside of the setting considered here. For example, [9] studies prospect theoretic preferences in interdependent security games, [10] considers online pay-per-bid auctions, [11] focuses on coercive bargaining games, and [12] analyses a game arising from the sharing of common failure-prone resources. More closely related to our game setting, we recall [13–16]. However, these works lack a theoretical framework to analyze the impact of irrational behaviors, as well as theoretical guarantees regarding equilibrium existence and algorithmic convergence. In particular, [13] constrains the prospect theoretic parameters to a specific set ensuring that the overall convexity of the game is maintained; [14] only provides results for a two-player setting, while [15, 16] disregard any algorithmic convergence consideration.

*Modeling irrationality in games.* Concurrently, other frameworks have been proposed in the literature to model bounded rationality in strategic multi-agent settings. While prospect theory primarily targets risk perceptions towards an exogenous uncertain parameter, other frameworks, such as conjectural variations and strategically robust games, consider irrationality stemming from other players’ actions. Specifically, according to conjectural variations, a player conjectures a reaction of its rivals in response to a modification of its own actions, and acts according to this belief [17]. Generic interpretations of conjectural variations have been recently proposed in [18, 19]; however, the endogenization of the conjectures requires a consistency notion which makes them difficult to deploy to real-world problems [20]. In practice, behavioral models in use are limited to discrete and softmax-style choice rules while accounting for irrationality through the addition of noise [21]. However, these simplistic models are not expressive enough to correctly model the complexities of human behavioral biases.

## 1.2 Notations and Basic Preliminaries

### Basic notation and Convex analysis

$\mathbb{N}^*, \mathbb{R}^n, \mathbb{R}_{\ge 0}^n$ is the set of nonnegative natural, real and nonnegative real, respectively. For a set-valued operator $\mathcal{F}: \mathbb{R}^n \rightarrow \mathbb{R}^n$, its graph is defined as $\text{gra}(\mathcal{F}) := \{(x, u) | u \in \mathcal{F}(x)\}$. For $\mathcal{A} \subset \mathbb{R}^n$, $\text{conv} \mathcal{A}$ denotes the convex hull of $\mathcal{A}$, while $\mathcal{N}_{\mathcal{A}}: \mathcal{A} \hookrightarrow \mathbb{R}^n: x \mapsto \{v \in \mathbb{R}^n | \sup_{z \in \mathcal{A}} v^\top(z-x) \le 0\}$
denotes its normal cone. Further, $\text{proj}_A(\cdot)$ denotes the projection onto the set $A$. Given two sets $A, B \subseteq \mathbb{R}^n$, their Cartesian product is defined as $A \times B := \{(a, b) : a \in A, b \in B\}$. Given $x \in A$ and $\epsilon > 0$, denote $B(x, \epsilon) := \{z \in A \mid \|z - x\| < \epsilon\}$, with $\| \cdot \|$ the Euclidean norm. Furthermore, $1_{x \in A} = 1$ if $x \in A$, 0 otherwise. Finally, the point-to-set distance between a point $x$ and a set $A$ is defined as $\text{dist}(x, A) := \inf\{\|x - z\| \mid z \in A\}$.

## Nonsmooth analysis

In the following sections, we repeatedly deal with the task of maximizing a nonsmooth nonconcave function. We introduce the concept of conservative Jacobians, as a generalization of the notion of gradient to functions that are almost everywhere differentiable.

Let $f: \mathbb{R}^n \to \mathbb{R}^m$ be a locally Lipschitz mapping. By Rademacher's theorem, it is almost everywhere differentiable (in the sense of Lebesgue measure). A path is an absolutely continuous function $x: [0, 1] \to \mathbb{R}^n$ which admits a derivative $\dot{x}$ for almost every $t \in [0, 1]$, and for which $x(t) - x(0)$ is the Lebesgue integral of $\dot{x}$ between 0 and $t$ for all $t \in [0, 1]$.

**Definition 1.1** A locally Lipschitz function $f: \mathbb{R}^n \to \mathbb{R}^m$ admits $\mathcal{J}_f: \mathbb{R}^n \rightrightarrows \mathbb{R}^{m \times n}$ as a conservative Jacobian, if $\mathcal{J}_f$ is nonempty-valued, outer semicontinuous, locally bounded, and for all paths $x: [0, 1] \to \mathbb{R}^n$ and almost all $t \in [0, 1]$

$$ \frac{d}{dt} f(x(t)) = \langle v, \dot{x}(t) \rangle, \quad \forall v \in \mathcal{J}_f(x(t)). $$

A locally Lipschitz function $f$ that admits a conservative Jacobian $\mathcal{J}_f$ is called path-differentiable.

When $f: \mathbb{R}^n \times \mathbb{R}^p \to \mathbb{R}^m$ is a function of two arguments $x$ and $y$, we use $\mathcal{J}_{f,x}(\tilde{x}, \tilde{y})$ (resp. $\mathcal{J}_{f,y}(\tilde{x}, \tilde{y})$) to denote the conservative Jacobian of $f$ as a function of $x$ (resp. $y$) and evaluated at $(\tilde{x}, \tilde{y})$. For scalar-valued functions $f: \mathbb{R}^n \to \mathbb{R}$, we equivalently talk about gradients and use the plain notation $\mathcal{J}_f$.

Unfortunately, not every locally Lipschitz function is path-differentiable. For this reason, we will restrict our attention to definable functions, as defined next [22, Definitions 1.4 and 1.5].

**Definition 1.2 (Definable function)** A collection $\mathcal{O} = (\mathcal{O}_n)_{n \in \mathbb{N}^*}$, where each $\mathcal{O}_n$ contains subsets of $\mathbb{R}^n$, is an $o$-minimal structure on $(\mathbb{R}, +, \cdot)$ if

1. all semialgebraic subsets of $\mathbb{R}^n$ belong to $\mathcal{O}_n$;

2. the elements of $\mathcal{O}_1$ are precisely the finite unions of points and intervals;

3. $\mathcal{O}_n$ is a boolean subalgebra of the powerset of $\mathbb{R}^n$;

4. if $A \in \mathcal{O}_n$ and $B \in \mathcal{O}_m$ then $A \times B \in \mathcal{O}_{n+m}$;

5. if $A \in \mathcal{O}_{n+1}$, then the set containing the elements of $A$ projected onto the first $n$ coordinates belongs to $\mathcal{O}_n$.

A subset of $\mathbb{R}^n$ which belongs to $\mathcal{O}$ is said to be definable (in an $o$-minimal structure). A function $f$ is definable if its graph $\{(x, v) \mid v = f(x)\}$ is definable.

Definable functions encompass several functions of practical engineering relevance, such as semialgebraic functions, log and exp functions, and globally subanalytic functions [22]. In particular, we will focus on the set of globally subanalytic functions, which notably includes analytic functions restricted to semialgebraic compact domains (contained in their natural open domain) and piecewise analytics functions that are real-analytic everywhere except on a set of measure zero.

Finally, definable functions possess the following useful property [23, Proposition 2].

**Lemma 1.3** All locally Lipschitz definable functions are path-differentiable.
<figure><img src="image_2.png" /><figcaption>Figure 1: Prospect theoretic ingredients.</figcaption></figure>

# 2 Background and Problem Formulation

## 2.1 Prospect Theory

Prospect theory captures four biases of human psychology: (i) Reference points: outcomes of a random event are evaluated relatively to an expected baseline; (ii) Loss aversion: humans dislike incurring losses more than they like gains of the same magnitude; (iii) Diminished sensitivity: humans value incremental improvements less as the magnitude of the amount in question increases; (iv) Probability distortion: humans overestimate the probabilities of rare events. To describe them, prospect theory introduces three main ingredients, which we describe next.

### 2.1.1 Value function

Let $\xi: \Xi \to \mathbb{R}$ be a random variable defined on the probability space $(\Xi, \mathcal{F}, \mathcal{P})$, which has to be intended as a profit, that is: the higher, the better. Thus, the distribution of $\xi$ is given by the pushforward distribution $\mathbb{P} = \mathcal{P} \circ \xi^{-1}$ of $\mathcal{P}$ w.r.t. $\xi$. We let the support of $\mathbb{P}$ be supp($\mathbb{P}$) = $\Xi$, thus $\xi: \Omega \to \Xi$ almost surely. Let $R \in \mathbb{R}$ be a reference point representing the baseline against which the agent perceives gains ($\xi \ge R$) and losses ($\xi < R$)$^1$. We assume $R$ to be exogenously fixed for each agent. The utility of the individual is of the form

$$S(\xi, R) = s_{-}(\xi, R)1_{\xi-R<0} + \kappa s_{+}(\xi, R)1_{R-\xi \le 0}, \quad (1)$$

where $S: \Xi \times \mathbb{R} \to \mathbb{R}$ is called value function and $\kappa \in \mathbb{R}$ is a parameter capturing the agent's loss aversion, i.e., its sensitivity towards losses compared to gains. Traditionally, $s_{-}(\xi, R)$ is convex in $(R-\xi)$ indicating a risk-seeking behavior in losses; conversely, $s_{+}(\xi, R)$ is concave in $(\xi-R)$ indicating a risk-averse behavior in gains. One popular example, as originally reported in [3] and represented in Figure 1a, is

$$S(\xi, R) = \begin{cases} (\xi - R)^{\alpha} & \text{when } \xi \ge R, \\ -\kappa(R - \xi)^{\beta} & \text{otherwise,} \end{cases} \quad (2)$$

with $\alpha, \beta > 0$. Smaller values of $\alpha$ and $\beta$ result in greater sensitivity to gains and losses of small magnitude compared to those of large magnitude. When $\alpha = \beta$, if $k > 1$ the agent weights its losses more than gains ("loss averse" behavior); conversely, when $0 \le k \le 1$ the agent weights gains more than losses ("gain seeking" behavior) [24]. Different descriptions of the value function have followed in the literature [25].

<footnote n="1">¹With a slight abuse of notation, we denote as $\xi$ both the random variable and its realizations.</footnote>
### 2.1.2 Probability weighting function

Further, [3] observed that, unlike in the EUT framework, individuals weight the utility of an uncertain outcome using weights that differ from the actual probability of the outcome. Traditionally, prospect theory has considered a discrete setting, where the uncertain outcome $\xi$ takes on a finite number $K < +\infty$ of values $\{\xi_1, \dots, \xi_K\}$ with probability $\{p_1, \dots, p_K\}$ such that $\sum_{k=1}^K p_k = 1, p_k \ge 0$. In this setting, the distorted probability is simply obtained by filtering each weight $p_k$ through a suitable<fnref n="2" /> function $\omega: [0, 1] \to [0, 1]$, such as [25] (see also Fig. 1b)

$$ \omega(p_k) = \frac{p_k^{\gamma}}{(p_k^{\gamma} + (1 - p_k^{\gamma}))^{1/\gamma}}, \quad \forall k = 1, \dots, K, \qquad (3) $$

with $\gamma \in (0, 1)$. Smaller values of $\gamma$ result in sharper increases of $\omega(p_k)$ when $p_k$ is close to 0, capturing a greater exacerbation of rare events. While conceptually easy, discrete distributions often fail to model the complexity of real-world phenomena. In the following, we extend this concept to continuous probabilities with a density to broaden the applicability of our method.

Let $\mathbb{P}$ be absolutely continuous with respect to the Lebesgue measure, with density $p(x)$ and cumulative density function (CDF) $F_{\mathbb{P}}(x)$. We introduce the notion of distortion function next.

**Definition 2.1** A continuous function $\omega: [0, 1] \to [0, 1]$ is a distortion function if it is non-decreasing, it satisfies $\omega(0) = \omega(0+) = 0$ and $\omega(1) = \omega(1-) = 1$, and it is differentiable almost everywhere.

The distorted measure $\mathbb{Q}$ can then be defined via the following transformation:

$$ F_{\mathbb{Q}}(x) = \omega(F_{\mathbb{P}}(x)), \qquad (4) $$

where $F_{\mathbb{Q}}(x)$ is the CDF of $\mathbb{Q}$.

**Lemma 2.2** Let $\mathbb{P}$ be absolutely continuous with respect to the Lebesgue measure, with density function $p(x)$. Further, let the distortion function $\omega$ satisfy Definition 2.1. Then, the distorted measure $\mathbb{Q}$ induced via (4) is absolutely continuous with density $q(x) = \omega'(F_{\mathbb{P}}(x))p(x)$.

**Proof:** Since $\omega$ is differentiable, $F_{\mathbb{Q}}(x)$ is differentiable almost everywhere. The density of $\mathbb{Q}$, denoted as $q(x)$, can then be obtained by the chain rule as

$$ F'_{\mathbb{Q}}(x) = \omega'(F_{\mathbb{P}}(x)) F'_{\mathbb{P}}(x) \implies q(x) = \omega'(F_{\mathbb{P}}(x)) p(x). \qquad (5) $$

Since $\omega$ is non-decreasing, it follows that $\omega'(u) \ge 0$ for all $u \in [0, 1]$; hence, $\omega'(F_{\mathbb{P}}(x)) \ge 0$ for almost all $x$. Next, in order for $q(x)$ to be a valid density function, we require $\int_{-\infty}^{+\infty} \omega'(F_{\mathbb{P}}(x))p(x)dx = 1$. Using the change of variable, $u = F_{\mathbb{P}}(x)$, we express equivalently $\int_{-\infty}^{+\infty} \omega'(F_{\mathbb{P}}(x))p(x)dx = \int_{0}^{1} \omega'(u)du$. Now, in order to satisfy the condition $\int_{0}^{1} \omega'(u)du = 1$, we would require $\omega(1) - \omega(0) = 1$, which is true by the definition of weighting function. As a result, $\mathbb{Q}$ is absolutely continuous with density $q(x)$ as defined in (5), completing the proof. □

It is then possible to “measure the distance”, e.g., by the Kullback-Leibler divergence (for simplicity we will assume that $F_{\mathbb{P}}(x)$ is increasing and continuous):

$$ D(\mathbb{P}\|\mathbb{Q}) = \int_{-\infty}^{+\infty} p(x) \log \frac{p(x)}{q(x)} dx = \int_{-\infty}^{+\infty} p(x) \log \frac{p(x)}{\omega'(F_{\mathbb{P}}(x))p(x)} dx = - \int_{-\infty}^{+\infty} p(x) \log \omega'(F_{\mathbb{P}}(x)) dx. $$

After substitution $y = F_{\mathbb{P}}(x) \implies dy = p(x)dx \iff dx = \frac{1}{p(x)}dy$, we get

$$ D(\mathbb{P}\|\mathbb{Q}) = - \int_{0}^{1} \log \omega'(y) dy. $$

Likewise: $D(\mathbb{Q}\|\mathbb{P}) = \int_{0}^{1} \omega'(y) \log \omega'(y) dy$. The divergence quantifies the “distribution shift” induced by the distortion function $\omega$ and can then be used to assess the probability distortion level of an individual.

The following result is an immediate consequence of the closure properties of definable functions [22].

<footnote n="2">²We require $\omega$ to satisfy certain properties such as i) non-negativity $\omega(p_k) \ge 0$, ii) normalization $\sum_{k=1}^K \omega(p_k) = 1$.</footnote>
**Lemma 2.3** If $p(\cdot), \omega(\cdot)$ are globally subanalytic, then $q(\cdot)$ is also globally subanalytic.

We enforce the following assumption.

**Assumption 1** $\mathbb{P}$ is absolutely continuous w.r.t. Lebesgue with global subanalytic density $p(x)$.

In the remainder, we limit ourselves to an absolutely continuous distribution for the sake of simplicity. Since definability (and consequently, the Sard's property [23] which we will employ in the following) are preserved by finite sum, all our results naturally apply to discrete distributions, as well as to settings where $\mathbb{P}$ is the joint distribution between a discrete and a continuous random variable.

### 2.1.3 Subjective vs. objective prospect

The subjective (i.e., perceived) expected utility of the individual reads

$$ \int_{\Xi} S(\xi, R)d\mathbb{Q}, \qquad (6) $$

as opposed to the (objective) EUT-based utility $\int_{\Xi} \xi d\mathbb{P}$. Note that for (6) to be well-defined we require $S(\xi, R)$ to be $\mathbb{Q}$-integrable in the variable $\xi$. This is equivalent to require $\int_{-\infty}^{+\infty} |S(\xi, R)|d\mathbb{Q} = \int_{-\infty}^{+\infty} |S(\xi, R)|\omega'(F_{\mathbb{P}}(\xi)) p(\xi)d\xi < +\infty$, or, equivalently, the product $|S(\xi, R)|\omega'(F_{\mathbb{P}}(\xi))$ is $\mathbb{P}$-integrable. In general, the satisfaction of this condition requires studying the interplay between $p(\xi)$, $S$ and $\omega'(F_{\mathbb{P}}(\xi))$: densities with rapidly decaying tails (such as sub-Gaussians and sub-exponentials) can more easily counteract the growth of $|S(\xi, R)|\omega'(F_{\mathbb{P}}(\xi))$, hence imposing less restrictive limitations to the growth of the integrand. In the remainder of the paper, we shall assume the integral in (6) to be well-defined.

**Remark 2.1 (On parameter selection)** The successful deployment of prospect theory to model irrationality depends on the careful choice of the parameters employed in the value and weighting functions. Ultimately, parameters selection is problem-specific and often requires datasets obtained through surveys and behavioral studies. We refer the interested reader to [26], where a comparative analysis of different statistical and parsimony-based methods to select the most appropriate model to explain behavioral biases is presented. In the absence of problem-specific datasets, the practitioner can refer to literature data, such as from [27].

## 2.2 Problem Formulation

We consider a noncooperative game with uncertain payoffs<fnref n="3" /> between $N$ self-interested agents, indexed by $i \in \mathcal{I} := \{1, ..., N\}$. Each agent $i \in \mathcal{I}$ selects a strategy $x_i$ from a feasible set $\mathcal{X}_i \subseteq \mathbb{R}^{m_i}$, with $m_i \in \mathbb{N}^*$. Let $\boldsymbol{x} = \text{col}(x_1, x_2, ..., x_N)$ be the collection of all players' strategies and $\mathcal{X} := \prod_i \mathcal{X}_i \subseteq \mathbb{R}^m$ be the joint feasible strategy set, with $m := \sum_i m_i$. The goal of each agent is to maximize a profit determined by a Borel function $f_i(x_i, \boldsymbol{x}_{-i}, \xi)$, $f_i: \mathbb{R}^m \times \Xi \to \mathbb{R}$ that depends on its own strategy, the other agents' strategies $\boldsymbol{x}_{-i} \in \mathcal{X}_{-i} := \prod_{j \in \mathcal{I} \setminus \{i\}} \mathcal{X}_j$, and a $\xi$ defined on the probability space $(\Xi, \mathcal{F}, \mathbb{P})$ which is shared among all agents.

In the presence of irrational agents (e.g., humans), we formulate a PT-based game by exploiting the ingredients introduced in Subsection 2.1. We equip each agent with an exogenous reference point $R_i$, a value function $S_i$ and a weighting function $\omega_i$ resulting in the distorted measure $\mathbb{Q}_i$.

Further, we enforce the following standing assumptions throughout the paper.

**Assumption 2** We assume the following<fnref n="4" />:

(i) The support $\Xi$ is given by $\Xi = \{G\xi \le g\}$, for $G, g$ of appropriate dimensions.

(ii) For all $i \in \mathcal{I}$, $\mathcal{X}_i = \{G_i\xi \le g_i\}$, for $G_i, g_i$ of appropriate dimensions.

<footnote n="3">³We do not use the term stochastic games as it traditionally denotes games involving Markov Decision Processes.</footnote>

<footnote n="4">⁴The upcoming results can be extended to the more general case of $\Xi, \mathcal{X}_i$ being compact (and $\mathcal{X}_i$ also convex).</footnote>
(iii) Each $f_i(\cdot, \mathbf{x}_{-i}, \xi)$ and $S_i(\cdot, R_i)$ are globally subanalytic functions for almost all $\xi \in \Xi$ and $\mathbf{x}_{-i} \in \mathcal{X}_{-i}$, and locally Lipschitz.

(iv) Each $\omega_i(\cdot)$ is globally subanalytic.

Assumptions 2 encompass almost all cases of practical relevance in engineering applications. Further, we remark that global subanalytic utility functions represent a broader class than the concave and/or differentiable profit functions traditionally considered in game theoretic formulations (see for example [28]).

We define a PT game as a tuple $\Gamma_{\text{PT}} := (\mathcal{X}, \{S_i(f_i(x_i, \mathbf{x}_{-i}, \xi), R_i)\}_{i \in \mathcal{I}}, \mathbb{Q}_i)$ specifying a set of interdependent optimization problems of the form: $\forall i \in \mathcal{I}$

$$ \max_{\mathbf{x}_i \in \mathcal{X}_i} \mathcal{O}_i := \int_{\Xi} S_i(f_i(\mathbf{x}_i, \mathbf{x}_{-i}, \xi), R_i) d\mathbb{Q}_i. \quad (7) $$

To ease the notation, in the remainder, we shall use $\tilde{f}_i(\cdot, \mathbf{x}_{-i}, \xi) \equiv S_i(f_i(\mathbf{x}_i, \mathbf{x}_{-i}, \xi), R_i)$. Unfortunately, $\tilde{f}_i(\cdot, \mathbf{x}_{-i}, \xi)$ yields a nonconvex nonsmooth problem, rendering standard results (see [29]) inapplicable. Indeed, even if $f_i$ is “well-behaved” (e.g., convex and differentiable), the composition $\tilde{f}_i(\cdot, \mathbf{x}_{-i}, \xi)$ is generally not, due to $S_i$. As global solutions of (7) are therefore generally inaccessible, the classical concept of Nash equilibrium is not practically applicable. As a result, we introduce a different (weaker) solution concept suitable for stochastic nonconvex nonsmooth games as (7).

**Definition 2.4** A critical Nash equilibrium for (7) is a strategy profile $\mathbf{x}^*$ that satisfies

$$ \mathbf{0} \in \mathcal{J}_{\mathcal{O}_i, x_i}(x_i^*, \mathbf{x}_{-i}^*) - \mathcal{N}_{\mathcal{X}_i}(x_i^*), \quad \forall i \in \mathcal{I}. \quad (8) $$

Define $\bar{F}(\mathbf{x}) := \text{col}((\mathcal{J}_{\mathcal{O}_i, x_i}(x_i, \mathbf{x}_{-i}))_{i \in \mathcal{I}})$ as an extension of the classical pseudo-gradient [29] to our setting. Then, (8) is equivalent to the following variational inequality [4, Chapter 11]: there exists $\mathbf{v} \in \bar{F}(\mathbf{x})$ such that

$$ \mathbf{v}^{\top}(\mathbf{x} - \mathbf{x}^{*}) \le 0, \quad \forall \mathbf{x} \in \mathcal{X}. \quad (9) $$

**Remark 2.2** Eq. (8) is a necessary (albeit not sufficient) condition for the following to hold at $\mathbf{x}^*$

$$ \int_{\Xi} \tilde{f}_i(x_i^*, \mathbf{x}_{-i}^*, \xi) d\mathbb{Q}_i \ge \int_{\Xi} \tilde{f}_i(z, \mathbf{x}_{-i}^*, \xi) d\mathbb{Q}_i, \quad (10) $$

for all $z \in B(x_i^*, \epsilon)$. Eq. (10) is intuitive: in contrast to the standard definition of Nash equilibrium (see [30]) which is robust against any possible unilateral deviation, in local Nash equilibria these possible deviations are constrained, i.e., a local Nash equilibrium is a strategy profile such that no player has an incentive to deviate to a close strategy. While not fully satisfactory, the characterization of the set of local maximizers in the nonsmooth regime considered here remains a challenging problem. In the reminder, we shall restrict our attention to critical Nash equilibria satisfying (8).

# 3 Theoretical Framework

In the following, we provide theoretical guarantees on the existence and computation of equilibria for (7).

## 3.1 Existence of Equilibria

We establish the existence of critical Nash equilibria for (7).

**Proposition 3.1 (Path-differentiability)** Under Assumption 2, $\tilde{f}_i(\cdot, \mathbf{x}_{-i}, \xi)$ is path-differentiable for almost all $\xi \in \Xi$ and $\mathbf{x}_{-i} \in \mathcal{X}_{-i}$.
**Proof:** By Assumption 2, $f_i$ and $S_i$ are both definable. Further, definable functions are stable under composition [31]. By [23, Prop. 2], $\tilde{f}_i(\cdot, x_{-i}, \xi)$ is path-differentiable. $\square$

As a result of Proposition 3.1, $\tilde{f}_i(\cdot, x_{-i}, \xi)$ admits a conservative gradient denoted as $D_i : \mathbb{R}^{m_i} \to \mathbb{R}^{1 \times m_i}$ by Lemma 1.3. Importantly, as conservative Jacobians admit a chain rule [23, Lem. 2], the conservative gradient for all $i \in \mathcal{I}$ can be readily computed as

$$D_i(x_i, x_{-i}, \xi) \equiv \mathcal{J}_{S_i} (f_i(x_i, x_{-i}, \xi)) \mathcal{J}_{f_i, x_i}(x_i, x_{-i}, \xi).$$

**Proposition 3.2** Let Assumptions 1 and 2 hold and further assume that each $\tilde{f}_i$ is integrable. The following holds:

1. $\int_{\Xi} \tilde{f}_i(x_i, x_{-i}, \xi) d\mathbb{Q}_i$ is path-differentiable.

2. $\int_{\Xi} D_i(x_i, x_{-i}, \xi) d\mathbb{Q}_i$ is a conservative gradient for $\int_{\Xi} \tilde{f}_i(x_i, x_{-i}, \xi) d\mathbb{Q}_i$.

**Proof:** To prove the result, we invoke [32, Theorem 4]. Hence, the remainder of the proof proceeds by showing that the assumptions in [32, Theorem 4] hold true in our setting. Measurability of $\tilde{f}_i$ follows from Assumption 1(iii). Similarly, integrability follows by assumption. Next, $\tilde{f}_i$ is locally Lipschitz and path-differentiable by Proposition 3.1, with conservative gradient given by $D_i$. By definability of $\tilde{f}_i$, the set-valued $D_i$ is definable, hence jointly measurable. Since $\mathcal{X}$ is compact, there exists an integrable function $\gamma_i : \Xi \to \mathbb{R}_+$ such that for all $(x_i, \xi) \in \mathcal{X}_i \times \Xi$, $\|D_i(x_i, x_{-i}, \xi)\| \le \gamma_i(\xi)$, where $\|D_i(x_i, x_{-i}, \xi)\| := \sup_{y \in D_i(x_i, x_{-i}, \xi)} \|y\|$. At this point, we can conclude by invoking [32, Theorem 4]. $\square$

**Theorem 3.3** Let Assumptions 1 and 2 hold, and further assume that each $\tilde{f}_i$ is integrable. Then, (7) admits a critical Nash equilibrium.

**Proof:** We decompose the proof into two parts.

Part 1: $\bar{F}(x)$ is upper semicontinuous. First, by Proposition 3.2 we exchange the conservative gradient and integral in (8) and equivalently consider $\int_{\Xi} D_i(x_i, x_{-i}, \xi) d\mathbb{Q}_i$. By [23, Lemma 3], the component-wise aggregation of conservative field is a conservative field. Hence,

$$\bar{F}(x) := \operatorname{col} \left( \left( \int_{\Xi} D_i(x_i, x_{-i}, \xi) d\mathbb{Q}_i \right)_{i \in \mathcal{I}} \right)$$

is a conservative Jacobian for the game. From the definition of conservative Jacobian, it follows that $\bar{F}$ is graph-closed, nonempty valued, and locally bounded. Thus, it is upper semicontinuous [33, Corollary 1].

Part 2: Existence of equilibria. Since $\mathcal{X}$ is compact by Assumption 2 and $\bar{F}(x)$ is upper semicontinuous by Part 1, the set $\cup_{x \in \mathcal{X}} \bar{F}(x)$ is compact. Let $E$ be its convex hull. By definition $E$ is compact and convex. Define the set-valued map $\hat{F} : \mathcal{X} \times E \to \mathcal{X} \times E$ as

$$\hat{F}(x, y) := \left[ \underset{z \in \mathcal{X}}{\operatorname{argmax}} \frac{y^\top(z - x)}{\bar{F}(x)} \right], \quad \forall (x, y) \in \mathcal{X} \times E.$$

Since $\bar{F}(x)$ is upper semicontinuous, $\hat{F}$ is nonempty, compact and convex-valued, and upper semi-continuous. Thus, by Kakutani's fixed point theorem, $\hat{F}$ has a fixed-point $(x^*, y^*) \in \mathcal{X} \times E$, i.e., $x^* \in \underset{z \in \mathcal{X}}{\operatorname{argmax}} (y^*)^\top(z - x^*)$ and $y^* \in \bar{F}(x^*)$. Combining the above expressions yields

$$\begin{align} x^* \in \underset{z \in \mathcal{X}}{\operatorname{argmax}} \bar{F}(x^*)^\top (z - x^*), \tag{11} \\ \Leftrightarrow \exists v \in \bar{F}(x^*) \text{ s.t. } v^\top (z - x^*) \le 0, \forall z \in \mathcal{X}, \nonumber \end{align}$$

which is equivalent to (9). $\square$

We refer to the set of critical Nash equilibria of (7) as SOL($\Gamma_{\mathrm{PT}}$) $\subseteq \mathcal{X}$. By Theorem 3.3, SOL($\Gamma_{\mathrm{PT}}$) $\neq \emptyset$.
## 3.2 Computation of Equilibria

Stochastic variational inequalities, as introduced in Equation (11), can be solved by resorting to Stochastic Gradient Ascent (SGA) methods. However, in the nonconvex nonsmooth regime, stochastic gradient methods may generate undesired directions resulting in sporadically erratic movements due to the mere inclusion $\tilde{F}(x) \supset F(x)$ [34, Thm. 2.7.3]. We clarify this in the following pedagogical example.

**Example 3.4** Let $\psi : (x, \xi) \mapsto \xi|x|$ and $\mathbb{P} = \frac{1}{2}\delta_{-1} + \frac{1}{2}\delta_1$ or $\mathbb{P}$ is the uniform distribution on $[-1, 1]$. Then, $\mathcal{J}_{\mathbb{E}[\psi(x, \xi)]} = 0$ but $\mathbb{E}[\mathcal{J}_{\psi(x, \xi)}(0)] = [-1, 1]$.

Nonetheless, we can exploit Proposition 3.2 to interchange the conservative gradient and the integral operator, allowing us to devise Algorithm 1 that converges to a critical Nash equilibrium for (7).

<caption>**Algorithm 1:** SGA for PT-games</caption>

Input: $x^0, \{\eta^t\}_{t \in \mathbb{N}}$
Init: $t = 0$.
while not converged do
Extract $\hat{\xi}_i^t \sim Q_i$
Determine $v^t \in \text{col} \left( \left( D_i(x_i^t, x_{-i}^t, \hat{\xi}_i^t) \right)_{i \in \mathcal{I}} \right)$
Compute $x^{t+1} = \text{proj}_{\mathcal{X}}(x^t + \eta^t v^t)$
$t \leftarrow t + 1$

**Assumption 3** The step size $\eta^t$ is chosen such that $\forall t \ge 0$, $\eta^t > 0$, $\sum_t \eta^t = +\infty$, $\sum_t (\eta^t)^2 < +\infty$.

**Theorem 3.5** Let Assumptions (1), (2) and (3) hold, and further assume that each $\tilde{f}_i$ is integrable. Then, $x^t$ in Algorithm (1) converges to a critical Nash equilibrium of (7).

**Proof:** To prove the result, we invoke [32, Theorem 5]. Hence, the remainder of the proof proceeds by showing that the assumptions in [32, Theorem 5] hold true in our setting. Since [32, Theorem 5] considers a minimization problem, we shall flip the objective function. Measurability of $\tilde{f}_i$ follows from Assumption 1(iii). Similarly, integrability follows by assumption. Next, $\tilde{f}_i$ is locally Lipschitz and path-differentiable by Proposition 3.1, with conservative gradient given by $D_i$. By definability of $\tilde{f}_i$, the set-valued $D_i$ is definable, hence jointly measurable. Assumptions 2 ensure that there exists a measurable selection function $v$ such that for all $(x, \xi)$ it holds $v^t \in \text{col} \left( \left( D_i(x_i^t, x_{-i}^t, \hat{\xi}_i^t) \right)_{i \in \mathcal{I}} \right)$ [35, Thm. 18.13]. Further, $v^t$ is definable, as definable functions and their gradients retain definability [36, Chapter 3]. Next, the boundedness of the iterates, e.g., $\sup_t \|x^t\| < \infty$ almost surely, follows from the use of a projection operator $\text{proj}_{\mathcal{X}}$ on a compact set $\mathcal{X}$. Under Assumption 1, $D_i$ is definable by [37, Theorem 1.3]. Then, by [32, Theorem 6], the set-valued map $D_{\mathcal{O}_i} = \mathbb{E}_{Q_i}[D_i]$ is definable. Since definability is preserved under componentwise aggregation, $D_{\mathcal{O}} = \text{col}(D_{\mathcal{O}_i})_{i \in \mathcal{I}}$ is definable and it is a conservative gradient for $\mathcal{O} = \text{col}(\mathcal{O}_i)_{i \in \mathcal{I}}$ by [32, Lemma 1]. By the definable conservative Sard's theorem [23], the set of $D_{\mathcal{O}}$-critical values of $\mathcal{O}$ is finite. The result then follows under Assumption 2 of Lemma 4 in [32] by invoking [32, Theorem 5]. Finally, we return to the original "max" problem and consider an ascent step rather than descent.

□

**Remark 3.1 (Algorithmic implementation)** Algorithm 1 is centralized and requires that each agent discloses its profit function and prospect theoretic parameters. Devising decentralized privacy-oriented counterparts is an interesting future direction.
### 3.3 Optimal Value and the Price of Irrationality

We are now interested in analyzing the differences between the PT game (7) and its EUT counterpart. To this end, we define the EUT game as a tuple $\Gamma_{EUT} := (\mathcal{X}, \{f_i\}_{i \in \mathcal{I}}, \mathbb{P})$ and denote its solution set as SOL($\Gamma_{EUT}$)$^5$. We provide the following result under additional regularity conditions. To streamline the presentation, we consider $R_i = 0$ and use the value function in (2); results are however generalizable.

**Proposition 3.6** Let Assumptions 2 hold and let $\mathbf{x}^* \in \text{SOL}(\Gamma_{PT}), \mathbf{y}^* \in \text{SOL}(\Gamma_{EUT})$. Let $R_i = 0$ and consider $S_i$ as in (2). Further, assume that $f_i$ is locally Lipschitz continuous with respect to the second argument and Lipschitz continuous in the third argument with constant $L_\xi$. Then,

$$|\mathbb{E}_{Q_i}[\tilde{f}_i(x_i^*, \mathbf{x}_{-i}^*, \xi)] - \mathbb{E}_{\mathbb{P}}[f_i(y_i^*, \mathbf{y}_{-i}^*, \xi)]| \le \bar{B} + L_i \|x_i^* - y_i^*\| + L_{-i} \|x_{-i}^* - y_{-i}^*\| + L_\xi C,$$

where $\bar{B}, L_i, L_{-i} \in \mathbb{R}_+$ are constants defined below, and

$$C = \begin{cases} \sqrt{\frac{1}{2} \int_{0}^{1} \omega'_{i}(y) \log \omega'_{i}(y) dy} & \text{if } \mathbb{P} \text{ is absolutely continuous with given density,} \\ \sum_{i=1}^{K} |\omega_{i}(p_{i}) - p_{i}| & \text{if } \mathbb{P} \text{ is discrete supported on K atoms.} \end{cases}$$

**Proof:** To ease the notation, let $z = f_i(x_i^*, \mathbf{x}_{-i}^*, \xi)$ and $\bar{z} = f_i(y_i^*, \mathbf{y}_{-i}^*, \xi)$. For $R_i = 0$ and $S_i$ as in (2), it holds:

$$\begin{aligned} & |\mathbb{E}_{Q_i}[z^{\alpha} \mathbf{1}_{z \ge 0} - \kappa(-z)^{\beta} \mathbf{1}_{z < 0}] - \mathbb{E}_{\mathbb{P}}[\bar{z}]| \\ & \le \underbrace{|\mathbb{E}_{Q_i}[z^{\alpha} \mathbf{1}_{z \ge 0} - \kappa(-z)^{\beta} \mathbf{1}_{z < 0}] - \mathbb{E}_{Q_i}[\bar{z}]|}_{(a)} + \underbrace{|\mathbb{E}_{Q_i}[\bar{z}] - \mathbb{E}_{Q_i}[\bar{z}]|}_{(b)} + \underbrace{|\mathbb{E}_{Q_i}[\bar{z}] - \mathbb{E}_{\mathbb{P}}[\bar{z}]|}_{(c)}. \end{aligned}$$

We next bound each term.

$$(a) \le \mathbb{E}_{Q_i} [|(z^{\alpha} \mathbf{1}_{z \ge 0} - \kappa(-z)^{\beta} \mathbf{1}_{z < 0}) - z|] \le \mathbb{E}_{Q_i} [B_i(\xi)] =: \bar{B},$$

where the first inequality follows from Jensen's inequality, while the second from the fact that since $x_i, \mathbf{x}_{-i}$ live in a compact set it is always possible to find a constant $B_i(\xi)$, generally depending on $\xi$, such that $|(z^{\alpha} \mathbf{1}_{z \ge 0} - \kappa(-z)^{\beta} \mathbf{1}_{z < 0}) - z| \le B_i(\xi)$. Since $(z^{\alpha} \mathbf{1}_{z \ge 0} - \kappa(-z)^{\beta} \mathbf{1}_{z < 0})$ and $z$ are integrable for any fixed $x, \mathbf{x}_{-i}$, it follows that the pointwise bound $B_i(\xi)$ is also integrable in $\xi$, hence $\bar{B} < +\infty$.

$$(b) \le \mathbb{E}_{Q_i} [\|z - \bar{z}\|] \le L_i \|x_i^* - y_i^*\| + L_{-i} \|\mathbf{x}_{-i}^* - \mathbf{y}_{-i}^*\|,$$

with $L_i = \sup_{q \in \mathcal{X}_i} \sup_{g \in \mathcal{J}_{f_i, x_i}(q)} \|g\|$ and $L_{-i} = \sup_{q \in \mathcal{X}_{-i}} \sup_{h \in \mathcal{J}_{f_i, x_{-i}}(q)} \|h\|$, where the first inequality follows again from Jensen's inequality, while the second from the (local) Lipschitz continuity of $f_i$ in its first and second arguments, respectively.

$$(c) \le L_\xi W_1(Q_i, \mathbb{P}) \le \begin{cases} L_\xi C \sqrt{\frac{1}{2} \int_0^1 \omega'_i(y) \log \omega'_i(y) dy} & \text{if } \mathbb{P} \text{ abs. continuous with density } p(x), \\ L_\xi \sum_{i=1}^K |\omega_i(p_i) - p_i| & \text{if } \mathbb{P} \text{ discrete supported on K atoms,} \end{cases}$$

where $W_1(Q_i, \mathbb{P})$ denotes the type-1 Wasserstein distance between $Q_i$ and $\mathbb{P}^6$, and $C = \text{diam}(\Xi) + 1$, with $\text{diam}(\Xi) = \sup\{\|\xi_1 - \xi_2\| : \xi_i \in \Xi\}$. Here, the first inequality follows from Kantorovich-Rubinstein duality [38, Thm. 5.10] and the second from: Upper case: we combine the relationships (a) $W_1 \le \rho_P$ (where $\rho_P$ denotes the Prohorov distance among $Q_i$ and $\mathbb{P}$)) from [39], (b) $\rho_P \le \rho_{TV}$ (where $\rho_{TV}$ denotes the Total Variation distance among $Q_i$ and $\mathbb{P}$)), (c) Pinsker's inequality. Lower case: it follows directly from the definition of Wasserstein distance among discrete distributions with the same atoms. $\square$

<figure><img src="image_3.png" /></figure>

Proposition 3.6 provides a qualitative understanding of the effect of prospect theory on the optimal value function of the i-th agent. To further quantify the inefficiency introduced by irrational behaviors, we introduce the *Price of Irrationality* (PoI).

<footnote>$^5$Under the stated assumptions, SOL($\Gamma_{EUT}$) $\neq \emptyset$ as the EUT game is a degenerate case of (7).</footnote>

<footnote>$^6$For two probability measures $\mu, \nu$ supported on $\Xi$, the Wasserstein distance is defined as $W_1(\mu, \nu) = \inf_{\pi \in \Pi(\mu, \nu)} \int_{\Xi \times \Xi} \|x - y\| d\pi(x, y)$, where $\| \cdot \|$ is any norm, and $\Pi(\mu, \nu)$ denotes the set of couplings with marginals $\mu$ and $\nu$, respectively.</footnote>
**Definition 3.7** For a given $\boldsymbol{x}^* \in \text{SOL}(\Gamma_{PT})$ and $\boldsymbol{y}^* \in \text{SOL}(\Gamma_{EUT})$, the Price of Irrationality is defined as

$$ \text{PoI}(\boldsymbol{x}^*, \boldsymbol{y}^*) := \frac{\int_{\Xi} (\sum_{i \in \mathcal{I}} f_i(x_i^*, \boldsymbol{x}_{-i}^*, \xi)) d\mathbb{P}}{\int_{\Xi} (\sum_{i \in \mathcal{I}} f_i(y_i^*, \boldsymbol{y}_{-i}^*, \xi)) d\mathbb{P}} $$

The semantic goes as follows: the PoI is the ratio between the expected value of the social welfare induced by the PT equilibria and the corresponding quantity induced by the EUT equilibria, both evaluated w.r.t. the *objective* prospect. Note that any welfare function $W: \mathcal{X} \to \mathbb{R}$ can be in practice used in the PoI definition.

A PoI $\ge 1$ indicates that irrationality enhances (or at least do not degrade) social welfare, due for example to the fact that risk-seeking and gain-seeking agents facilitate exploration of high-reward strategies. Conversely, when PoI $< 1$ irrationality leads to reduced social welfare, due for example to the fact that loss averse agents cause overly conservative strategies.

# 4 Prospect Theoretic Preferences in Electricity Markets

Motivated by an energy market application, we consider aggregative games enjoying additional structural properties and derive further insights.

## 4.1 Game Description

We consider an energy community comprising $\mathcal{I} = \{1, \dots, N\}$ strategic end users, operated by a supplier which proposes a two-part tariff. We assume a time-of-day tariff, with a fixed fee $\bar{P} > 0$ during the first part of the day, and a variable charge $P(\xi) = \bar{P} + c + \xi$ during the second part, where $c > 0$ is a constant offset (e.g., the peak electricity price from the wholesale market) and $\xi$ a random variable (e.g., due to the fluctuation of renewables) distributed according to a discrete distribution supported on $K$ ordered atoms, $\{\xi_1, \dots, \xi_K\}$, with weights $p_k = \mathbb{P}[\{\xi = \xi_k\}], \forall k \in \mathcal{K} = \{1, \dots, K\}$. We assume $\xi_1 < c < \xi_K$. For a given end user $i$, the action $x_i \ge 0$ defines the amount of energy purchased by $i$ during the first part of the day from the supplier at price $\bar{P}$. Moreover, we assume that $i$ provides a forecast of its consumption $d_i$ during the second part of the day. To encourage social aspects in the community management (e.g., CO2 emission reduction), the supplier imposes a penalty for deviations from a collective target. We let $d^0 := \frac{1}{N} \sum_{i=1}^N d_i$ be the community collective target, and $\sigma(\boldsymbol{x}) := \frac{1}{N} \sum_{i=1}^N x_i$ be the aggregate purchase. Then, the profit function of user $i$ takes the form

$$ f_i(x_i, \sigma(\boldsymbol{x}), \xi) = U_i(\sigma(\boldsymbol{x})) - C_i(x_i, \xi), \qquad (12) $$

where $U_i(\sigma(\boldsymbol{x})) = -a_i(\sigma(\boldsymbol{x}) - d^0)^2 + b_i$ for $a_i \in \mathbb{R}_{>0}$, $a_i = a_j, \forall j \ne i$, $b_i \in \mathbb{R}_+$, and $C_i(x_i, \xi) = \bar{P}x_i + P(\xi)(d_i - x_i)$. The first component $U_i(\sigma(\boldsymbol{x}))$ captures the social benefit from the behavior of user $i$, by penalizing any deviation of the aggregate purchase from the collective target. The second component $C_i(x_i, \xi)$ is the trading cost: if $d_i \ge x_i$, end user $i$ incurs a penalty for buying the missing quantity; else it receives a compensation for the surplus$^7$. This setting gives rise to a coordination game [40], which is a noncooperative game where players earn a higher profit when they select the same action as the other players, thus giving rise to symmetric equilibria. For the PT setting, we consider a logarithmic distortion function and the Prelec weighting function. The game satisfies the assumptions in Subsection 2.2 and Section 3, thus it admits (at least) one equilibria $\boldsymbol{x}^*$.

## 4.2 Game Analysis

We begin by analyzing the EUT game, which in this specific case has a simplified structure.

**Proposition 4.1** Let Assumptions 2 (i), (ii) hold. Then, there exists a unique symmetric NE $\boldsymbol{y}^*$ solution of $\Gamma_{EUT}$.

<footnote>$^7$More generally, the results derived in Section (4) hold beyond the presented energy market game. For profit functions of the form (12), it suffices the following to hold: (i) each $f_i(x_i, \sigma(\boldsymbol{x}), \xi)$ is strongly concave and continuously differentiable in $x_i$ for almost all $\xi$; (ii) $C_i$ is monotone decreasing; (iii) there exists an integrable function $\Phi: \Xi \to \mathbb{R}_+$ such that $\nabla_{x_i} C_i(\cdot, \xi) \ge \phi(\xi)$ for almost all $\xi$. One can easily notice that (i)-(iii) are satisfied by the considered game.</footnote>
**Proof:** In a symmetric NE, $y_i = y_j, \forall i \neq j$. The existence follows from [41], the uniqueness noticing that $f_i(\cdot, \mathbf{y}_{-i}, \xi)$ is strongly concave due to the quadratic in $U_i$ involving $\sigma(\mathbf{y})$. □

Next, we move on to the PT game.

**Lemma 4.2** For any player $i \in \mathcal{I}$ and any $\mathbf{x} \in \mathcal{X}$, there exists $\underline{M}_k^i, \overline{M}_k^i > 0$ such that $\forall k \in \mathcal{K}$

$$ \omega_i(p_k) \cdot J_{S_i}^c(f_i(x_i, \mathbf{x}_{-i}, \xi_k)) \in [\underline{M}_k^i; \overline{M}_k^i]. $$

**Proof:** By definition of Clarke subdifferential of a continuous function, it follows that $J_{S_i}^c(f_i(x_i, \mathbf{x}_{-i}, \xi_k))$ is locally bounded, i.e., $\|g\| \le L, \forall g \in J_{S_i}^c(z)$ with $z \in B_\varepsilon(f_i(x_i, \mathbf{x}_{-i}, \xi_k))$. By the compactness of $\mathcal{X}$ and $\Omega_i$ and the continuity of $f_i$, it further follows that there also exists a global upper bound $\bar{L}$. Setting $\underline{M}_k^i = -\omega_i(p_k) \cdot \bar{L}$ and $\overline{M}_k^i = \omega_i(p_k) \cdot \bar{L}$ concludes the proof. □

Recall that $\mathbf{x}^* \in \text{SOL}(\Gamma_{\text{PT}})$ if and only if it solves the variational inequality in (9). By exploiting Lemma 4.2, we then obtain

$$ -\sum_k [\nabla_{x_i} C_i(x_i, \xi_k)|_{x_i=x_i^*}]_{i \in \mathcal{I}}^{\top} \widetilde{M}_k(\mathbf{x}-\mathbf{x}^*) + \frac{1}{N} \sum_k \widetilde{M}_k \sum_i [\nabla_z U_i(z)|_{z=\sigma(\mathbf{x}^*)}]^{\top} (x_i-x_i^*) \le 0, \forall \mathbf{x} \in \mathcal{X}, \quad (13) $$

where

$$ \widetilde{M}_k := \sum_i (\overline{M}_k^i \mathbf{1}_{\nabla_{x_i} f_i(x_i, \sigma(\mathbf{x}), \xi_k)|_{\mathbf{x}=\mathbf{x}^*} < 0} + \underline{M}_k^i \mathbf{1}_{\nabla_{x_i} f_i(x_i, \sigma(\mathbf{x}), \xi_k)|_{\mathbf{x}=\mathbf{x}^*} \ge 0}), $$

with $\mathbf{1}_{z \in \mathcal{A}} = 1$ if $z \in \mathcal{A}$; 0 otherwise.

Let dist $(\mathbf{y}^*, \text{SOL}(\Gamma_{\text{PT}}))$ be the point-to-set distance between the solution of $\Gamma_{\text{EUT}}$, $\mathbf{y}^*$, and the set of solutions of $\Gamma_{\text{PT}}$, $\text{SOL}(\Gamma_{\text{PT}})$. We provide the following result.

**Theorem 4.3** For any $\mathbf{x}^* \in \text{SOL}(\Gamma_{\text{PT}})$, it holds

$$ \|\sigma(\mathbf{y}^*) - \sigma(\mathbf{x}^*)\| \le \frac{\sum_k \widetilde{M}_k \sqrt{\sum_{i=1}^N (L_U^i)^2}}{N^2 \sum_k \psi(\xi_k) \widetilde{M}_k} \text{dist}(\mathbf{y}^*, \text{SOL}(\Gamma_{\text{PT}})). $$

Hence, $\|\sigma(\mathbf{y}^*) - \sigma(\mathbf{x}^*)\| \to 0$ as $N \to +\infty$.

**Proof:** By definition, $U_i$ is $L_U^i$-locally Lipschitz. Further, define $\psi(\xi) = c - |\xi_1|$ and notice that $\psi(\xi) : \Xi \to \mathbb{R}_+$ is such that $\nabla_{x_i} C_i(\cdot, \xi) \ge \psi(\xi)$ for almost all $\xi$. Pick $\mathbf{x}^* \in \text{SOL}(\Gamma_{\text{PT}})$. Since $\mathbf{y}^* \in \mathcal{X}$, taking the norm of the inequality (13) yields

$$ \underbrace{N \sum_k \psi(\xi_k) \widetilde{M}_k \|\sigma(\mathbf{y}^*) - \sigma(\mathbf{x}^*)\|}_{(*)} \le \frac{1}{N} \|\sum_k \widetilde{M}_k \sum_i [\nabla_z U_i(z)|_{z=\sigma(\mathbf{x}^*)}]^\top (y_i^* - x_i^*)\|. \quad (14) $$

Applying Cauchy-Schwarz inequality to the right-hand side of (14), we get

$$ (*) \le \frac{1}{N^2} \sum_k \widetilde{M}_k \sqrt{\sum_{i=1}^N (L_U^i)^2} \|\mathbf{y}^* - \mathbf{x}^*\|, \forall \mathbf{x}^* \in \text{SOL}(\Gamma_{\text{PT}}), $$

from which the theorem statement follows. □

Theorem 4.3 matches the following intuition: when a limited number of agents interact, the marginal effect of one agent's decision on the overall decision is greater than when a large number of agents interact. Social norms or collective expectations may, in fact, be less present or influential in a small group, allowing irrational behaviors [42].
## 4.3 Numerical Results

The parameters used in our simulations are described in Table 1. For the sake of illustration, we consider $N = 20$ end users with different risk profiles, classified into “aggressive” (e.g., gain seeking and risk seeking in gains), “cautious” (e.g., risk averse and loss averse), and “neutral”. These classes go beyond the usual classification of Kahneman and Tversky in [3]. Code is available at https://github.com/phdPokou/PT-Game, while implementation details are reported in the Appendix.

<figure><img src="image_4.png" /><figcaption>Figure 2: Illustration on an energy community based coordination game involving irrational end users.</figcaption></figure>

Fig. 2a illustrates the bound derived in Theorem 4.3, which captures the sublinear convergence of the aggregate strategies of the PT game $\sigma(\mathbf{x}^*)$ for one generated $\mathbf{x}^* \in \text{SOL}(\Gamma_{\text{PT}})$ to the EUT game aggregate $\sigma(\mathbf{y}^*)$ as $N$ increases.

In Fig. 2b, we plot the amount of energy purchased at equilibrium as a function of the market profitability, for different risk profiles (we used here the same parameters $b_i$ and $d_i$ for ease of comparison). Under $S_i$, losses are overweighted, resulting in stagnant gains and therefore additional risk-taking would not be justified for aggressive end users. For a fixed level of market favorability, aggressive agents buy more than cautious, who buy more than rational. Under unfavorable market conditions, aggressive agents continue to expose themselves more to risk anticipating a rapid turnaround in prices, while cautious agents considerably limit the quantity of electricity purchased.

Fig. 2c shows the PoI introduced in Definition 3.7, in different combinations of risk profiles. When the agents have homogeneous risk profiles, their PoI remains lower than 1, which means that irrationality amplifies system imbalances and leads to efficiency loss compared to a situation where all agents act rationally. When the agents have heterogeneous risk profiles, there exists combinations of risk profiles leading to PoIs larger than 1. Thus, irrational behaviors are not intrinsically negative for the system efficiency. On the contrary, a well-portioned diversification of behaviors through a combination of agents with varied risk profiles can paradoxically generate higher performance than a purely rational system. This illustrates the complex nature of market dynamics, where diversity of risk profiles is an advantage, not a weakness.
# 5 Conclusions & Outlook

We considered $N$-agent noncooperative games with uncertain payoffs, where agents display irrational behaviors due to their risk perception. We showed that these behaviors can be encoded in game theoretical formulations by means of prospect theory. We retrieved theoretical guarantees on the existence and the algorithmic convergence to critical Nash equilibria of the uncertain nonconvex nonsmooth game.

We conclude with an outlook on some exciting open problems.

* **Reference-dependent preferences.** Instead of fixing the reference point $R_i$ a priori (e.g., with ad-hoc assumptions), one may consider a different formulation where reference-dependence is derived from preference conditions and a unique reference point arises endogenously.

* **Prospect theoretic preferences in mechanism design.** Several incentive design problems, such as those arising in recommendation systems and highway tolling, are naturally described via hierarchical interactions between one leader (the policy maker) and several followers (the consumers). We envision our prospect theoretic game formulation to be extended to Stackelberg games to more accurately model modern techno-socio-economical networks. Alternatively, the agents’ expected modified profit functions in Eq. (7) can be integrated into mechanism design problems via a *game augmentation*, where an external agent, e.g., a *coordinator*, is added to the game. The coordinator’s profit function $\phi(\boldsymbol{x}, \boldsymbol{\xi})$ can be freely selected (upon satisfying Assumptions 2) to encode the considered incentive design problem, e.g., steering the $N$ agents towards an operationally-safe/fair equilibrium. This framework, which is known as the co-design model (see for example [43]), represents an alternative to Stackelberg games in modeling mechanism design tasks, thus avoiding the computational complexity of the former [44].

* **Empirical validation.** The prospect theoretic formulation contains parameters describing the risk attitude of each agent, which are here randomly assigned. Consequently, we are currently working to validate the proposed framework using real-world datasets, which will allow us to better understand the risk preferences of specific populations.

# References

[1] D. Bauso, *Game theory with engineering applications*. SIAM, 2016.

[2] J. Von Neumann and O. Morgenstern, “Theory of games and economic behavior: 60th anniversary commemorative edition,” in *Theory of games and economic behavior*, Princeton university press, 2007.

[3] D. Kahneman and A. Tversky, “Prospect theory: An analysis of decision under risk,” *Econometrica*, vol. 47, pp. 263–292, 1979.

[4] Y. Cui and J.-S. Pang, *Modern nonconvex nondifferentiable optimization*. SIAM, 2021.

[5] J. Shalev, “Loss aversion equilibrium,” *International Journal of Game Theory*, vol. 29, pp. 269–287, 2000.

[6] L. P. Metzger and M. O. Rieger, “Non-cooperative games with prospect theory players and dominated strategies,” *Games and Economic Behavior*, vol. 115, pp. 396–409, 2019.

[7] K. Keskin, “Equilibrium notions for agents with cumulative prospect theory preferences,” *Decision Analysis*, vol. 13, no. 3, pp. 192–208, 2016.

[8] P. Leclerc, *Prospect theory preferences in noncooperative game theory*. Virginia Commonwealth University, 2014.

[9] A. R. Hota and S. Sundaram, “Interdependent security games on networks under behavioral probability weighting,” *IEEE Transactions on Control of Network Systems*, vol. 5, no. 1, pp. 262–273, 2016.
[10] T. Brünner, J. Reiner, M. Natter, and B. Skiera, "Prospect theory in a dynamic game: Theory and evidence from online pay-per-bid auctions," *Journal of Economic Behavior & Organization*, vol. 164, pp. 215-234, 2019.

[11] C. K. Butler, "Prospect theory and coercive bargaining," *Journal of Conflict Resolution*, vol. 51, no. 2, pp. 227-250, 2007.

[12] A. R. Hota and S. Sundaram, "Controlling human utilization of failure-prone systems via taxes," *IEEE Transactions on Automatic Control*, vol. 66, no. 12, pp. 5772-5787, 2020.

[13] G. El Rahi, S. R. Etesami, W. Saad, N. B. Mandayam, and H. V. Poor, "Managing price uncertainty in prosumer-centric energy trading: A prospect-theoretic stackelberg game approach," *IEEE Transactions on Smart Grid*, vol. 10, no. 1, pp. 702-713, 2017.

[14] T. Li and N. B. Mandayam, "When users interfere with protocols: Prospect theory in wireless networks using random access and data pricing as an example," *IEEE Transactions on Wireless Communications*, vol. 13, no. 4, pp. 1888-1907, 2014.

[15] Y. Yao, C. Gao, T. Chen, J. Yang, and S. Chen, "Distributed electric energy trading model and strategy analysis based on prospect theory," *International Journal of Electrical Power & Energy Systems*, vol. 131, p. 106865, 2021.

[16] M. Vahid-Pakdel, S. Ghaemi, B. Mohammadi-Ivatloo, J. Salehi, and P. Siano, "Modeling non-cooperative game of gencos' participation in electricity markets with prospect theory," *IEEE Transactions on Industrial Informatics*, vol. 15, no. 10, pp. 5489-5496, 2019.

[17] C. Figuières, A. Jean-Marie, N. Quérou, and M. Tidball, *Theory of Conjectural Variations*. World Scientific, Series on Mathematical Economics and Game Theory, vol. 2, 2004.

[18] D. J. Calderone, B. J. Chasnov, S. A. Burden, and L. J. Ratliff, "Consistent Conjectural Variations Equilibrium: Characterization & Stability for a Class of Continuous Games," June 2023. arXiv:2305.11866 [cs].

[19] B. Chasnov, T. Fiez, and L. Ratliff, "Opponent Anticipation via Conjectural Variations," 2019.

[20] T. Lindh, "The inconsistency of consistent conjectures: Coming back to cournot," *Journal of Economic Behavior & Organization*, vol. 18, no. 1, pp. 69-90, 1992.

[21] R. D. Luce and H. Raiffa, *Games and Decisions*. Wiley, New York, 1959.

[22] M. Coste, *An introduction to o-minimal geometry*. Istituti editoriali e poligrafici internazionali Pisa, 2000.

[23] J. Bolte and E. Pauwels, "Conservative set valued fields, automatic differentiation, stochastic gradient methods and deep learning," *Mathematical Programming*, vol. 188, pp. 19-51, 2021.

[24] P. P. Wakker, *Prospect theory: For risk and ambiguity*. Cambridge university press, 2010.

[25] A. Tversky and D. Kahneman, "Loss aversion in riskless choice: A reference-dependent model," *Oxford Journals*, vol. 106, pp. 1039-1061, 1991.

[26] W. Neilson and J. Stowe, "A further examination of cumulative prospect theory parameterizations," *Journal of risk and uncertainty*, vol. 24, pp. 31-46, 2002.

[27] S. Wu, H.-W. Huang, Y.-L. Li, H. Chen, and Y. Pan, "A novel probability weighting function model with empirical studies," *International Journal of Computational Intelligence Systems*, vol. 14, no. 1, pp. 208-227, 2021.

[28] L. J. Ratliff, S. A. Burden, and S. S. Sastry, "On the characterization of local nash equilibria in continuous games," *IEEE transactions on automatic control*, vol. 61, no. 8, pp. 2301-2307, 2016.
[29] H. H. Bauschke and P. L. Combettes, *Convex analysis and monotone operator theory in Hilbert spaces*. Springer, 2017.

[30] C. A. Holt and A. E. Roth, “The nash equilibrium: A perspective,” *Proceedings of the National Academy of Sciences*, vol. 101, no. 12, pp. 3999–4002, 2004.

[31] D. Davis, D. Drusvyatskiy, S. Kakade, and J. D. Lee, “Stochastic subgradient method converges on tame functions,” *Foundations of computational mathematics*, vol. 20, no. 1, pp. 119–154, 2020.

[32] J. Bolte, T. Le, and E. Pauwels, “Subgradient sampling for nonsmooth nonconvex minimization,” *SIAM Journal on Optimization*, vol. 33, no. 4, pp. 2542–2569, 2023.

[33] J.-P. Aubin and A. Cellina, *Differential inclusions: set-valued maps and viability theory*, vol. 264. Springer Science & Business Media, 2012.

[34] F. H. Clarke, *Optimization and nonsmooth analysis*. SIAM, 1990.

[35] C. D. Aliprantis and K. Border, “Infinite dimensional analysis, (1994),” 1994.

[36] L. Van den Dries, *Tame topology and o-minimal structures*, vol. 248. Cambridge university press, 1998.

[37] R. Cluckers and D. J. Miller, “Stability under integration of sums of products of real globally subanalytic functions and their logarithms,” 2011.

[38] C. Villani *et al.*, *Optimal transport: old and new*, vol. 338. Springer, 2009.

[39] A. L. Gibbs and F. E. Su, “On choosing and bounding probability metrics,” *International statistical review*, vol. 70, no. 3, pp. 419–435, 2002.

[40] M. J. Osborne and A. Rubinstein, *A Course in Game Theory*. Cambridge, Massachusetts: Harvard University Press, 1960.

[41] J. B. Rosen, “Existence and uniqueness of equilibrium points for concave n-person games,” *Econometrica*, pp. 520–534, 1965.

[42] J. K. Goeree and C. A. Holt, “Ten little treasures of game theory and ten intuitive contradictions,” *American Economic Review*, vol. 91, no. 5, pp. 1402–1422, 2001.

[43] F. Fabiani, A. Simonetto, and P. J. Goulart, “Personalized incentives as feedback design in generalized nash equilibrium problems,” *IEEE Transactions on Automatic Control*, vol. 68, no. 12, pp. 7724–7739, 2023.

[44] Z.-Q. Luo, J.-S. Pang, and D. Ralph, *Mathematical programs with equilibrium constraints*. Cambridge University Press, 1996.
# Appendix

## Implementation details

<table><caption>Table 1: Parameters chosen in the numerical part, $\mathcal{U}$ and $\mathcal{G}$ being respectively the Uniform and Gumbel distributions. To avoid spurious results, we run each simulation (up to a given tolerance tol) multiple times with different step sizes and initializations, and consider the best outcome.</caption><thead><tr><th colspan="2">Model parameters</th></tr></thead><tbody><tr><td>N</td><td>number of agents</td><td>20</td></tr><tr><td>a<sub>i</sub></td><td>function profit parameter</td><td>360</td></tr><tr><td>b<sub>i</sub></td><td>function profit parameter</td><td>U[0.9 &middot; 10<sup>3</sup>, 2.85 &middot; 10<sup>3</sup>]</td></tr><tr><td>d<sub>i</sub></td><td>consumption</td><td>U[24, 44]</td></tr><tr><td>P&#770;</td><td>fixed free</td><td>15.8</td></tr><tr><td>c</td><td>constant offset</td><td>0.5</td></tr><tr><td>&xi;</td><td>variable charge</td><td>G(&mu; = 0.2, &sigma; = 0.2)</td></tr><tr><td colspan="3">PT parameters</td></tr><tr><td>&gamma;<sub>a</sub></td><td>aggressive agents parameter</td><td>0.7</td></tr><tr><td>&alpha;<sub>a</sub></td><td>aggressive agents parameter</td><td>1.2</td></tr><tr><td>&beta;<sub>a</sub></td><td>aggressive agents parameter</td><td>1.035</td></tr><tr><td>&kappa;<sub>a</sub></td><td>aggressive agents parameter</td><td>U[0.14, 0.38]</td></tr><tr><td>&gamma;<sub>t</sub></td><td>cautious agents parameter</td><td>0.9</td></tr><tr><td>&alpha;<sub>t</sub></td><td>cautious agents parameter</td><td>0.124</td></tr><tr><td>&beta;<sub>t</sub></td><td>cautious agents parameter</td><td>0.3398</td></tr><tr><td>&kappa;<sub>t</sub></td><td>cautious agents parameter</td><td>U[2.4, 3.7]</td></tr><tr><td colspan="3">Algorithm parameters</td></tr><tr><td>&eta;<sup>0</sup></td><td>initial step size</td><td>0.01</td></tr><tr><td>&eta;<sup>t</sup></td><td>step size</td><td>&eta;<sup>0</sup>/(t + 1)<sup>0.51</sup></td></tr><tr><td>tol</td><td>tolerance</td><td>10<sup>-4</sup></td></tr><tr><td>x<sub>i</sub><sup>0</sup></td><td>initial guess</td><td>1</td></tr></tbody></table>