SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Kyriakos Lotidis  
Stanford University  
klotidis@stanford.edu
Panayotis Mertikopoulos  
Univ. Grenoble Alpes, CNRS, Inria, Grenoble INP, LIG 38000 Grenoble, France & Archimedes RU, NKUA panayotis.mertikopoulos@imag.fr
Nicholas Bambos  
Stanford University  
bambos@stanford.edu
Jose Blanchet  
Stanford University  
jose.blanchet@stanford.edu
In this paper, we study the problem of learning in quantum games – and other classes of semidefinite games – with scalar, payoff-based feedback. For concreteness, we focus on the widely used matrix multiplicative weights (MMW) algorithm and, instead of requiring players to have full knowledge of the game (and/or each other's chosen states), we introduce a suite of minimal-information matrix multiplicative weights (3MW) methods tailored to different information frameworks. The main difficulty to attaining convergence in this setting is that, in contrast to classical finite games, quantum games have an infinite continuum of pure states (the quantum equivalent of pure strategies), so standard importance-weighting techniques for estimating payoff vectors cannot be employed. Instead, we borrow ideas from bandit convex optimization and we design a zeroth-order gradient sampler adapted to the semidefinite geometry of the problem at hand. As a first result, we show that the 3MW method with deterministic payoff feedback retains the  $\mathcal{O}(1 / \sqrt{T})$  convergence rate of the vanilla, full information MMW algorithm in quantum min-max games, even though the players only observe a single scalar. Subsequently, we relax the algorithm's information requirements even further and we provide a 3MW method that only requires players to observe a random realization of their payoff observable, and converges to equilibrium at an  $\mathcal{O}(T^{-1/4})$  rate. Finally, going beyond zero-sum games, we show that a regularized variant of the proposed 3MW method guarantees local convergence with high probability to all equilibria that satisfy a certain first-order stability condition.
The integration of quantum information theory into computer science and machine learning [4, 51, 64] has the potential of providing faster and more efficient computing resources, new encryption and security protocols, and improved machine learning algorithms, enabling advancements in areas such as quantum cryptography, shadow tomography, quantum GANs, and adversarial learning [1, 15, 18, 37]. As a well-known example, Google's "Sycamore" 54-qubit processor recently showcased this "quantum advantage" by training an autonomous vehicle model in less than 200 seconds [4], a fact made possible by the ability of quantum computers to prepare superpositions of qubits that exceed the operational capabilities of standard Boolean gates.
Deploying such models within a multi-agent context, such as the utilization of QGANs or autonomous vehicles, leads to a significant transformation compared to classical non-cooperative environments.
Indeed, unlike classical games (where a mixed strategy is a probabilistic mixture of the underlying pure strategies), quantum games utilize mixed states, which represent probabilistic mixtures of quantum projectors. As a consequence, a mixed quantum state can yield payoffs that cannot be expressed as a convex combination of classical pure strategies.
In light of this, quantum learning has drawn significant attention in recent years [2, 27, 28, 33, 34, 60]. In a multi-agent context, the most widely used framework is the so-called matrix multiplicative weights (MMW) algorithm [2, 16, 27, 28, 35]: First introduced by Tsuda et al. [59] in the context of matrix and dictionary learning, MMW can be viewed as a semidefinite analogue of the standard Hedge / EXP3 methods for multi-armed bandits [6, 36, 61], and is a special case of the mirror descent family of algorithms [47]. Specifically, in the concrete setting of two-player, zero-sum quantum games, Jain & Watrous [27] showed that players using the MMW algorithm can learn an  $\varepsilon$ -equilibrium in  $\mathcal{O}(1/\varepsilon^2)$  iterations - or, in terms of speed of convergence after  $T$  iterations, they converge to equilibrium at a  $\mathcal{O}(1/\sqrt{T})$  rate.
To the best of our knowledge, this result remains the tightest known bound for equilibrium learning in quantum games – and the more general class of semidefinite games [26]. At this point, we highlight that we focus on classical computing algorithms for solving quantum games, unlike recent results [10, 22] that employ quantum algorithms to solve classical games and semidefinite programs. Building on [27], Jain et al. [28] studied its continuous-time analogue – the quantum replicator dynamics (QRD) – in quantum min-max games, focusing on the recurrence and volume conservation properties of the players' actual trajectory of play. Going beyond the min-max case, [38] examined the convergence of the dynamics of "follow the quantum leader" (FTQL), a class of continuous-time dynamics that includes the QRD as a special case. The main result of [38] was that the only states that are asymptotically stable under the (continuous-time) dynamics of FTQL are those that satisfy a certain first-order stationarity condition known as variational stability [44, 46]. In a similar line of work, Lin et al. [35] studied the continuous-time QRD, and discrete-time MMW in quantum potential games, utilizing a Riemannian metric to obtain a gradient flow in the spirit of [41, 42].
Our contributions in the context of previous work. All works mentioned above, in both continuous and discrete time, assume full information, i.e., players have access to their individual payoff gradients - which, among others, might imply that they have full knowledge of the game. However, this condition is rarely met in online learning environments where players only observe their in-game payoffs; this is precisely the starting point of our paper which aims to derive a convergent payoff-based, gradient-free variant of MMW algorithm for learning in quantum games.
A major roadblock in this is that standard approaches from learning in finite games fail in the quantum setup for two reasons: First and foremost, there is a continuum of pure states available to every player, unlike classical finite games where there is only a finite set of pure actions. Second, even after the realization of the pure states of the players, there is an inherent uncertainty and randomness due to the payoff-generating quantum process (an aspect that has no classical counterpart). To overcome this hurdle, we employ a continuous-action reformulation of quantum games, and we leverage techniques from bandit convex optimization for estimating the players' payoff gradients.
Our first contribution is a variant of MMW that only requires mixed payoff observations and achieves an  $\mathcal{O}(1 / \sqrt{T})$  equilibrium convergence rate in two-player zero-sum quantum games, matching the rate of the full information MMW in [27]. Then, to account for information-starved environments where players are only able to observe their in-game, realized payoff observable, we also develop a bandit variant of MMW which utilizes a single-point gradient estimation technique in the spirit of [55] and achieves an  $\mathcal{O}(T^{-1/4})$  equilibrium convergence rate. Finally, we also examine the behavior of the MMW algorithm with bandit information in general  $N$ -player games, where we show that variationally stable equilibria are locally attracting with high probability.
Importantly, the above results transfer to more general games with a semidefinite structure – such as multi-agent covariance matrix optimization in signal processing, energy efficiency maximization in multi-antenna systems, etc. [43, 45, 62]. While we do not provide a complete theory, we discuss a number of non-quantum applications that showcase how our results can be generalized further.
Notation. Given a (complex) Hilbert space  $\mathcal{H}$ , we will use Dirac's bra-ket notation and write  $|\psi\rangle$  for an element of  $\mathcal{H}$  and  $\langle\psi|$  for its adjoint; otherwise, when a specific basis is implied by the context, we will use the dagger notation “†” to denote the Hermitian transpose  $\psi^\dagger$  of  $\psi$ . We will also write
$\mathbf{H}^d$  for the space of  $d \times d$  Hermitian matrices, and  $\mathbb{H}_+^d$  for the cone of positive-semidefinite matrices in  $\mathbb{H}^d$ . Finally, we denote by  $\| \mathbf{A}\|_F = \sqrt{\operatorname{tr}[\mathbf{A}^\dagger\mathbf{A}]}$  the Frobenius norm of  $\mathbf{A}$  in  $\mathbb{H}^d$ .
We begin by reviewing some basic notions from the theory of quantum games, mainly intended to set notation and terminology; for a comprehensive introduction, see [23]. To streamline our presentation, we introduce the primitives of quantum games in a 2-player setting before treating the general case.
Quantum games. Following [20, 23], a 2-player quantum game consists of the following:
1. Each player  $i \in \mathcal{N} \coloneqq \{1,2\}$  has access to a complex Hilbert space  $\mathcal{H}_i \cong \mathbb{C}^{d_i}$  describing the set of (pure) quantum states available to the player (typically a discrete register of qubits). A quantum state is an element  $\psi_i$  of  $\mathcal{H}_i$  with unit norm, so the set of pure states is the unit sphere  $\Psi_i \coloneqq \{\psi_i \in \mathcal{H}_i : \| \psi_i \|_F = 1\}$  of  $\mathcal{H}_i$ . We will write  $\Psi \coloneqq \Psi_1 \times \Psi_2$  for the space of all ensembles  $\psi = (\psi_1, \psi_2)$  of pure states  $\psi_i \in \Psi_i$  that are independently prepared by each player.
2. The rewards that players receive are based on their individual payoff functions  $u_{i} \colon \Psi \to \mathbb{R}$ , and they are derived through a positive operator-valued measure (POVM) quantum measurement process. Following [17], this unfolds as follows: Given a finite set of measurement outcomes  $\Omega$  that a referee can observe from the players' quantum states (e.g., measure a player-prepared qubit to be "up" or "down"), each outcome  $\omega \in \Omega$  is associated to a positive semi-definite operator  $\mathbf{P}_{\omega} \colon \mathcal{H} \to \mathcal{H}$  defined on the tensor product  $\mathcal{H} \coloneqq \mathcal{H}_1 \otimes \mathcal{H}_2$  of the players' individual state spaces. We further assume that  $\sum_{\omega \in \Omega} \mathbf{P}_{\omega} = \mathbf{I}$  so the probability of observing  $\omega \in \Omega$  at state  $\psi \in \Psi$  is  $P_{\omega}(\psi) = \langle \psi_1 \otimes \psi_2 | \mathbf{P}_{\omega} | \psi_1 \otimes \psi_2 \rangle$ .
3. The payoff of each player is then generated by this measurement process via a payoff observable  $U_{i} \colon \Omega \to \mathbb{R}$ : specifically, the measurement  $\omega$  is drawn from  $\Omega$  based on the players' state profile  $\psi = (\psi_{1},\psi_{2})$ , and each player  $i \in \mathcal{N}$  receives as reward the quantity  $U_{i}(\omega)$ . Accordingly, the player's expected payoff at state  $\psi \in \Psi$  is  $u_{i}(\psi) \coloneqq \langle U_{i} \rangle \equiv \sum_{\omega} P_{\omega}(\psi) U_{i}(\omega)$ .
A quantum game is then defined as a tuple  $\mathcal{Q} \equiv \mathcal{Q}(\mathcal{N}, \Psi, u)$  with players, states, and payoff as above.
Mixed states. Apart from pure states, each player  $i \in \mathcal{N}$  may prepare probabilistic mixtures thereof, known as mixed states. These mixed states differ from mixed strategies used in classical, finite games as they do not correspond to convex combinations of their pure counterparts; instead, given a family of pure quantum states  $\psi_{i\alpha_i} \in \Psi_i$  indexed by  $\alpha_i \in \mathcal{A}_i$ , a mixed state is described by a density matrix of the form
$$
\mathbf {X} _ {i} = \sum_ {\alpha_ {i} \in \mathcal {A} _ {i}} x _ {i \alpha_ {i}} \left| \psi_ {i \alpha_ {i}} \right\rangle \left\langle \psi_ {i \alpha_ {i}} \right| \tag {1}
$$
where the mixing weights  $x_{i\alpha_i} \geq 0$  of each  $\psi_{i\alpha_i}$  are normalized so that  $\operatorname{tr} \mathbf{X}_i = 1$ . By Born's rule, this means that the probability of observing  $\omega \in \Omega$  under  $\mathbf{X} = (\mathbf{X}_1, \mathbf{X}_2)$  is
$$
P _ {\omega} (\mathbf {X}) = \sum_ {\alpha_ {1} \in \mathcal {A} _ {1}} \sum_ {\alpha_ {2} \in \mathcal {A} _ {2}} x _ {1, \alpha_ {1}} x _ {2, \alpha_ {2}} P _ {\omega} (\psi_ {\alpha}). \tag {2}
$$
where  $\psi_{\alpha} = \psi_{1,\alpha_1}\otimes \psi_{2,\alpha_2}$ . Therefore, in a slight abuse of notation, the expected payoff of player  $i\in \mathcal{N}$  under  $\mathbf{X}$  will be  $u_{i}(\mathbf{X}) = \sum_{\alpha \in \mathcal{A}}x_{\alpha}u_{i}(\psi_{\alpha})$ . which, equivalently, can be written as:
$$
u _ {i} (\mathbf {X}) = \sum_ {\omega \in \Omega} \sum_ {\alpha \in \mathcal {A}} x _ {\alpha} u _ {i} \left(\psi_ {\alpha}\right) U _ {i} (\omega). \tag {3}
$$
This gives a succinct representation of the payoff structure of  $\mathcal{Q}$  - see also Eq. (5) below.
Continuous game reformulation. In view of the above, treating a quantum game as a "tensorial" extension of a finite game can be misleading. For our purposes, it would be more suitable to treat a quantum game as a continuous game where each player  $i \in \mathcal{N}$  controls a matrix variable  $\mathbf{X}_i$  drawn from the "spectraplex" defined as  $\mathbf{X}_i = \{\mathbf{X}_i \in \mathbb{H}_+^{d_i} : \operatorname{tr} \mathbf{X}_i = 1\}$ . In this interpretation, the players' payoff functions  $u_i: \mathbf{X} \equiv \mathbf{X}_1 \times \mathbf{X}_2 \to \mathbb{R}$  are linear in each player's density matrix  $\mathbf{X}_i \in \mathbf{X}_i$ ,  $i \in \mathcal{N}$ . Since  $u_1, u_2$  are linear in  $\mathbf{X}_1$  and  $\mathbf{X}_2$ , the individual payoff gradients of each player will be given by
$$
\mathbf {V} _ {1} (\mathbf {X}) := \nabla_ {\mathbf {X} _ {1} ^ {\top}} u _ {1} (\mathbf {X}) \quad \text {a n d} \quad \mathbf {V} _ {2} (\mathbf {X}) := \nabla_ {\mathbf {X} _ {2} ^ {\top}} u _ {2} (\mathbf {X}) \tag {4}
$$
so we can further write each player's payoff function as
$$
u _ {1} (\mathbf {X}) = \operatorname {t r} \left[ \mathbf {X} _ {1} \mathbf {V} _ {1} (\mathbf {X}) \right] \quad \text {a n d} \quad u _ {2} (\mathbf {X}) = \operatorname {t r} \left[ \mathbf {X} _ {2} \mathbf {V} _ {2} (\mathbf {X}) \right] \quad \text {f o r a l l} \mathbf {X} \in \mathcal {X}. \tag {5}
$$
Since  $\mathcal{X}$  is compact and each  $u_{i}$  is multilinear in  $\mathbf{X}$ , the players' payoff functions are automatically bounded, Lipschitz continuous and Lipschitz smooth, i.e., there exist constants  $B_{i}, G_{i}$  and  $L_{i}, i \in \mathcal{N}$  such that, for all  $\mathbf{X}, \mathbf{X}' \in \mathcal{X}$ , we have:
1. Boundedness:  $|u_{i}(\mathbf{X})|\leq B_{i}$
2. Lipschitz continuity:  $|u_{i}(\mathbf{X}) - u_{i}(\mathbf{X}^{\prime})| \leq G_{i}\| \mathbf{X} - \mathbf{X}^{\prime}\|_{F}$
3. Lipschitz smoothness:  $\| \mathbf{V}_i(\mathbf{X}) - \mathbf{V}_i(\mathbf{X}^{\prime})\| _F\leq L_i\| \mathbf{X} - \mathbf{X}^{\prime}\| _F$
Nash equilibrium. The most widely used solution concept in game theory is that of a Nash equilibrium (NE). In our context, it is mixed profile  $\mathbf{X}^{*}\in \mathcal{X}$  from which no player has incentive to deviate, i.e.,  $u_{1}(\mathbf{X}^{*})\geq u_{1}(\mathbf{X}_{1};\mathbf{X}_{2}^{*})$  and  $u_{2}(\mathbf{X}^{*})\geq u_{2}(\mathbf{X}_{1}^{*};\mathbf{X}_{2})$  for all  $\mathbf{X}_1\in \mathcal{X}_1,\mathbf{X}_2\in \mathcal{X}_2$ . Since  $\mathcal{X}_i$  is convex and  $u_{i}$  linear in  $\mathbf{X}_i$ , the existence of Nash equilibria follows from the Debreu's theorem [19].
Zero-sum quantum games. In the case where  $u_{1} = -u_{2}$ , and setting  $\mathcal{L}\colon \mathcal{X}_1\times \mathcal{X}_2\to \mathbb{R}$ , the Nash equilibria of  $\mathcal{Q}$  are the saddle points of  $\mathcal{L}$ , i.e., the solutions of the minimax problem
$$
\max  _ {\mathbf {X} _ {1} \in \boldsymbol {\mathcal {X}} _ {1}} \min  _ {\mathbf {X} _ {2} \in \boldsymbol {\mathcal {X}} _ {2}} \mathcal {L} (\mathbf {X} _ {1}, \mathbf {X} _ {2}) = \min  _ {\mathbf {X} _ {2} \in \boldsymbol {\mathcal {X}} _ {2}} \max  _ {\mathbf {X} _ {1} \in \boldsymbol {\mathcal {X}} _ {1}} \mathcal {L} (\mathbf {X} _ {1}, \mathbf {X} _ {2}) \tag {6}
$$
By Sion's minimax theorem [54], the set of Nash equilibria is nonempty. Then, given a Nash equilibrium  $\mathbf{X}^*$ , we define the duality gap of  $\mathbf{X} = (\mathbf{X}_1, \mathbf{X}_2)$  as
$$
\operatorname {G a p} _ {\mathcal {L}} (\mathbf {X}) := \mathcal {L} \left(\mathbf {X} _ {1} ^ {*}, \mathbf {X} _ {2}\right) - \mathcal {L} \left(\mathbf {X} _ {1}, \mathbf {X} _ {2} ^ {*}\right) \tag {7}
$$
so  $\operatorname{Gap}_{\mathcal{L}}(\mathbf{X}) \geq 0$  with equality if and only if  $\mathbf{X}$  is itself a Nash equilibrium. In particular,  $\mathbf{X}$  is an  $\varepsilon$ -Nash equilibrium of  $\mathcal{Q}$  if and only if  $\operatorname{Gap}_{\mathcal{L}}(\mathbf{X}) \leq \varepsilon$ .
Other semidefinite games. In addition to quantum games, our framework can also be used for learning in other classes of games with a semidefinite structure as per [26, 45]. As an example, consider the problem of covariance matrix optimization in vector Gaussian multiple-access channels [9, 43, 57, 62]. In this case, there is a finite set of players indexed by  $i \in \mathcal{N} = \{1, \dots, N\}$ ; each player  $i \in \mathcal{N}$  picks a unit-trace semidefinite matrix  $\mathbf{X}_i \in \mathcal{X}_i$  and their payoff is given by the Shannon-Telatar capacity formula [57], viz.
$$
u _ {i} \left(\mathbf {X} _ {1}, \dots , \mathbf {X} _ {N}\right) = \log \det  \left(\mathbf {I} + \sum_ {j} \mathbf {H} _ {j} \mathbf {X} _ {j} \mathbf {H} _ {j} ^ {\dagger}\right) \tag {8}
$$
where each  $\mathbf{H}_i$  is a player-specific gain matrix [58]. Even though  $u_i$  is no longer multilinear in  $\mathbf{X}$ , the algorithms we derive later in the paper can be applied to this setting essentially verbatim.
3 The matrix multiplicative weights algorithm
Throughout the sequel, we will focus on equilibrium learning in quantum – and semidefinite – games. In the context of two-player, zero-sum quantum games, the state-of-the-art method is based on the so-called matrix multiplicative weights (MMW) algorithm [7, 27, 29, 59] which updates as
$$
\mathbf {Y} _ {i, t + 1} = \mathbf {Y} _ {i, t} + \gamma_ {t} \mathbf {V} _ {i} (\mathbf {X} _ {t}) \quad \mathbf {X} _ {i, t} = \frac {\exp (\mathbf {Y} _ {i , t})}{\operatorname {t r} [ \exp (\mathbf {Y} _ {i , t}) ]} \tag {MMW}
$$
In the above,  $(a)\mathbf{X}_t = (\mathbf{X}_{1,t},\mathbf{X}_{2,t})$  denotes the players' density matrix profile at each stage  $t = 1,2,\ldots$  of the process;  $(b)\mathbf{V}_i(\mathbf{X}_t)$  is the payoff gradient of player  $i\in \mathcal{N}$  under  $\mathbf{X}_t$ ;  $(c)\mathbf{Y}_t$  is an auxiliary state matrix that aggregates gradient steps over time; and  $(d)\gamma_{t} > 0,t = 1,2,\dots$  , is a learning rate (or step-size) parameter that can be freely tuned by the players.
Importantly, as stated, (MMW) requires full information at the player end: specifically, at each stage  $t = 1,2,\ldots$  of the process, each player  $i\in \mathcal{N}$  must receive their individual payoff gradient  $\mathbf{V}_i(\mathbf{X}_t)$  in order to perform the gradient update step in (MMW). Under this assumption, Jain & Watrous [27] showed that the induced empirical frequency of play
$$
\bar {\mathbf {X}} _ {T} = \frac {1}{T} \sum_ {t = 1} ^ {T} \mathbf {X} _ {t} \tag {9}
$$
converges to equilibrium at a rate of  $\mathcal{O}(1 / \sqrt{T})$  as per the formal result below:
Theorem 1 (Jain & Watrous [27]). Suppose that each player of a 2-player zero-sum game  $\mathcal{Q}$  follows (MMW) for  $T$  epochs with learning rate  $\gamma = G^{-1}\sqrt{2H / T}$  where  $H = \log (d_1d_2)$ . Then the players' empirical frequency of play enjoys the bound
$$
\operatorname {G a p} _ {\mathcal {L}} \left(\bar {\mathbf {X}} _ {T}\right) \leq G \sqrt {2 H / T} \tag {10}
$$
In particular, if (MMW) is run for  $T = \mathcal{O}(1 / \varepsilon^2)$  iterations,  $\bar{\mathbf{X}}_T$  will be an  $\varepsilon$ -Nash equilibrium of  $\mathcal{Q}$ .
To the best of our knowledge, this guarantee of Jain & Watrous [27] remains the tightest known bound for Nash equilibrium learning in 2-player zero-sum quantum games. At the same time, Theorem 1 hinges on the players having perfect access to their individual gradients – which, among others, might entail full knowledge of the game, observing the other player's density matrix, etc. Our goal in the sequel will be to relax precisely this assumption and develop a payoff-based variant of (MMW) that can be employed without stringent information and observability requirements as above.
4 Matrix learning without matrix feedback
In an online learning framework, it is more realistic to assume that players observe only the outcome of their actions – i.e., their individual payoffs. In this information-starved, payoff-based setting, our main goal will be to employ a minimal-information matrix multiplicative weights (3MW) algorithm that updates as
$$
\mathbf {Y} _ {i, t + 1} = \mathbf {Y} _ {i, t} + \gamma_ {t} \hat {\mathbf {V}} _ {i, t} \quad \mathbf {X} _ {i, t} = \frac {\exp (\mathbf {Y} _ {i , t})}{\operatorname {t r} [ \exp (\mathbf {Y} _ {i , t}) ]} \tag {3MW}
$$
where  $\hat{\mathbf{V}}_{i,t}$  is some payoff-based estimate of the payoff gradient  $\mathbf{V}_i(\mathbf{X}_t)$  of player  $i$  at  $\mathbf{X}_t$ , and all other quantities are defined as per (MMW). In this regard, the main challenge that arises is how to reconstruct each player's payoff gradient matrices when they are not accessible via an oracle.
4.1. The classical approach: Importance weighted estimators. In the context of classical, finite games and multi-armed bandits, a standard approach for reconstructing  $\hat{\mathbf{V}}_{i,t}$  is via the so-called importance weighted estimator (IWE) [12, 14, 32]. To state it in the context of finite games, assume that each player has at their disposal a finite set of pure strategies  $\alpha_{i} \in \mathcal{A}_{i}$ , and if each player plays  $\hat{\alpha}_{i} \in \mathcal{A}_{i}$ , then, in obvious notation, their individual payoff will be  $\hat{u}_i = u_i(\hat{\alpha}_i; \hat{\alpha}_{-i})$ . Then, if each player is using a mixed strategy  $x_{i} \in \Delta(\mathcal{A}_{i})$  to draw their chosen action  $\hat{\alpha}_{i}$ , the importance weighted estimator (IWE) for the payoff of the (possibly unplayed) action  $\alpha_{i} \in \mathcal{A}_{i}$  of player  $i$  is defined as
$$
\mathrm {I W E} _ {i \alpha_ {i}} = \frac {\mathbb {1} \left\{\alpha_ {i} = \hat {\alpha} _ {i} \right\}}{x _ {i \alpha_ {i}}} u _ {i} \left(\hat {\alpha} _ {i}; \hat {\alpha} _ {- i}\right) \quad \text {f o r a l l} \alpha_ {i} \in \mathcal {A} _ {i} \tag {IWE}
$$
with the assumption that  $x_{i}$  has full support, i.e., each action  $\alpha_{i} \in \mathcal{A}_{i}$  has strictly positive probability  $x_{i\alpha_i}$  of being chosen by the  $i$ -th player.
This approach has proven extremely fruitful in the context of multi-armed bandits and finite games where (IWE) is an essential ingredient of the optimal algorithms for each context [5, 12, 32, 65]. However, in our case, there are two insurmountable difficulties in extending (IWE) to a quantum context: First and foremost, the quantum regime is characterized by a continuum of pure states with highly correlated payoffs (in the sense that quantum states that are close in the Bloch sphere will have highly correlated POVM payoff observables); this comes in stark contrast to the classical regime of finite normal-form games, where players only have to contend with a finite number of actions (with no prior payoff correlations between them). Secondly, even after the realization of the pure states of the players, there is an inherent uncertainty and randomness due to the quantum measurement process that is invoked in the payoff-generating process; as such, the players' payoffs are also affected by an exogenous source of randomness which is altogether absent from (IWE).