SlowGuess/NeurIPS_2023_OCR huggingface dataset:
To that end, our point of departure is the observation that the Lyapunov property (6) of the energy function  $E(\theta)$  is precisely the key feature that enables convergence of (NHGD). Thus, assuming that control variables are mapped to latent ones via a general - but possibly highly coupled - representation map  $\chi \colon \Theta \to \mathcal{X}$ , we will consider an abstract preconditioning scheme of the form
$$
\dot {\theta} _ {i} = - \mathbf {P} _ {i} (\theta_ {i}) v _ {i} (\theta) \tag {7}
$$
$$
v _ {i} (\theta) := \nabla_ {\theta_ {i}} \ell_ {i} (\theta) \tag {8}
$$
denotes the individual loss gradient of player  $i$ , while the preconditioning matrix  $\mathbf{P}_i(\theta_i) \in \mathbb{R}^{m_i \times m_i}$  is to be designed so that (6) still holds under (7). In this regard, a straightforward calculation (which we prove in Appendix A) yields the following:
Lemma 1. Under the dynamics (7), we have
$$
\dot {E} (\theta) = - \sum_ {i \in \mathcal {N}} [ g _ {i} (\chi (\theta)) ] ^ {\intercal} \mathbf {J} _ {i} (\theta_ {i}) \mathbf {P} _ {i} (\theta_ {i}) [ \mathbf {J} _ {i} (\theta_ {i}) ] ^ {\intercal} \left(\chi_ {i} (\theta_ {i}) - x _ {i} ^ {*}\right) \tag {9}
$$
where, as per Section 2,  $\mathbf{J}_i(\theta_i) \in \mathbb{R}^{d_i \times m_i}$  denotes the Jacobian matrix of the map  $\chi_i: \Theta_i \to \mathcal{X}_i$ . More compactly, letting  $\mathbf{P} := \bigoplus_i \mathbf{P}_i$  denote the block-diagonal ensemble of the players' individual preconditioning matrices  $\mathbf{P}_i$  (and suppressing control variable arguments for concision), we have
$$
\dot {E} = - g (x) ^ {\mathsf {T}} \cdot \mathbf {J P J} ^ {\mathsf {T}} \cdot \left(x - x ^ {*}\right) \tag {10}
$$
In view of Lemma 1, a direct way to achieve the target Lyapunov property (6) would be to find a preconditioning matrix  $\mathbf{P}$  such that  $\mathbf{J} \mathbf{P} \mathbf{J}^{\intercal} = \mathbf{I}$ . However, since  $\mathbf{J}$  is surjective (by the faithfulness assumption for  $\chi$ ), the Moore-Penrose inverse  $\mathbf{J}^{+}$  of  $\mathbf{J}$  will be a right inverse to  $\mathbf{J}$ , i.e.,  $\mathbf{J} \mathbf{J}^{+} = \mathbf{I}$ . Hence, letting  $\mathbf{P} = (\mathbf{J}^{\intercal} \mathbf{J})^{+} = \mathbf{J}^{+}[\mathbf{J}^{+}]^{\intercal}$ , we obtain:
$$
\mathbf {J P J} ^ {\intercal} = \mathbf {J} \left(\mathbf {J} ^ {\intercal} \mathbf {J}\right) ^ {+} \mathbf {J} ^ {\intercal} = \mathbf {J} \mathbf {J} ^ {+} \left(\mathbf {J} ^ {\intercal}\right) ^ {+} \mathbf {J} ^ {\intercal} = \mathbf {I} \cdot \mathbf {I} = \mathbf {I} \tag {11}
$$
In this way, unraveling the above, we obtain the preconditioned hidden gradient flow
$$
\dot {\theta} _ {i} = - \mathbf {P} _ {i} (\theta_ {i}) \nabla_ {i} \ell_ {i} (\theta_ {i}) \quad \text {w i t h} \quad \mathbf {P} _ {i} (\theta_ {i}) = [ \mathbf {J} _ {i} (\theta_ {i}) ^ {\top} \mathbf {J} _ {i} (\theta_ {i}) ] ^ {+} \tag {PHGF}
$$
By virtue of design, the following property of (PHGF) is then immediate:
Proposition 1. Suppose that  $\Gamma$  admits a latent strictly monotone structure in the sense of (3b). Then the energy function (5) is a strict Lyapunov function for (PHGF), and every limit point  $\theta^{*}$  of  $\theta(t)$  is a Nash equilibrium of  $\Gamma$ .
2We do not attempt to provide here a primer on Riemannian geometry; for a masterful introduction, see [22].
Figure 3: Exploiting a hidden convex structure in the control layer. On the left, we present the dynamics (PHGF) in the example of minimizing the simple function  $f(\theta) = \mathrm{KL}(\mathrm{logit}(\theta_1,\theta_2)\| (1 / 2,1 / 3,1 / 6))$  over  $\Theta \equiv \mathbb{R}^2$ . In the subfigure to the right, we illustrate the hidden convex structure of the energy landscape, from the non-convex sublevel sets of  $\Theta$  to the latent space  $\mathcal{X} \equiv \{(x_{1},x_{2})\in \mathbb{R}_{\geq 0}^{2}: x_{1} + x_{2}\leq 1\}$ .
Proposition 1 validates our design choices for the preconditioning matrix  $\mathbf{P}$  as it illustrates that the dynamics (PHGF) converge to Nash equilibrium, without any separability requirements or dimensionality restrictions; in this regard, Proposition 1 already provides a marked improvement over the corresponding convergence result of [32] for (NHGD). To streamline our presentation, we defer the proof of Lemma 1 and Proposition 1 to Appendix A, and instead proceed directly to provide a bona fide, algorithmic implementation of (PHGF).
3.2. Preconditioned hidden gradient descent. In realistic machine learning problems, any algorithmic scheme based on (PHGF) will have to be run in an iterative, discrete-time environment; moreover, due to the challenges posed by applications with large datasets and optimization objectives driven by the minimization of empirical risk, we will need to assume that players only have access to a stochastic version of their full, deterministic gradients. As such, we will make the following blanket assumptions that are standard in the stochastic optimization literature [17, 21]:
Assumption 1. Each agent's loss function is an expectation of a random function  $L_{i}\colon \Theta \times \Omega \to \mathbb{R}$  over a complete probability space  $(\Omega ,\mathcal{F},\mathbb{P})$  , i.e.,  $\ell_i(\theta) = \mathbb{E}[L_i(\theta ;\omega)]$  . We further assume that:
1.  $L_{i}(\theta ;\omega)$  is measurable in  $\omega$  and  $\beta$  -Lipschitz smooth in  $\theta$  (for all  $\theta \in \Theta$  and  $\omega \in \Omega$  respectively).
2. The gradients of  $L_{i}$  have bounded second moments, i.e.,  $\sup_{\theta \in \Theta} \mathbb{E}[\| \nabla L_{i}(\theta ;\omega)\|^{2}] \leq M^{2}$ .
Taken together, the two components of Assumption 1 imply that each  $\ell_i$  is differentiable and Lipschitz continuous (indeed, by Jensen's inequality, we have  $\| \mathbb{E}[\nabla L_i(\theta ;\omega)]\| ^2\leq \mathbb{E}[\| \nabla L_i(\theta ;\omega)\| ^2 ]\leq M^2$ ). Moreover, by standard dominated convergence arguments,  $\nabla L_{i}(\theta ;\omega)$  can be seen as an unbiased estimator of  $\nabla \ell_{i}(\theta)$ , that is,  $\ell_i(\theta) = \mathbb{E}[\nabla L_i(\theta ;\omega)]$  for all  $\theta \in \Theta$ . In view of this, we will refer to the individual loss gradients  $\nabla_{i}L_{i}(\theta ;\omega)$  of player  $i\in \mathcal{N}$  as the player's individual stochastic gradients.
With all this in mind, the preconditioned hidden gradient descent algorithm is defined as the stochastic first-order recursion
$$
\theta_ {i, t + 1} = \theta_ {i, t} - \gamma_ {t} \mathbf {P} _ {i, t} V _ {i, t} \tag {PHGD}
$$
1.  $\theta_{i,t}$  denotes the control variable configuration of player  $i\in \mathcal{N}$  at each stage  $t = 1,2,\ldots$
2.  $\gamma_t > 0$  is a variable step-size sequence, typically of the form  $\gamma_t \propto 1 / t^p$  for some  $p \in [0,1]$ .
3.  $\mathbf{P}_{i,t} := \mathbf{P}_i(\theta_{i,t}) = [\mathbf{J}_i(\theta_{i,t})^\top \mathbf{J}_i(\theta_{i,t})]^+$  is the preconditioner of player  $i \in \mathcal{N}$  at the  $t$ -th epoch.
4.  $V_{i,t} \coloneqq \nabla_i L(\theta_{i,t};\omega_t)$  is an individual stochastic loss gradient generated at the control variable configuration  $\theta_t$  by an i.i.d. sample sequence  $\omega_t \in \Omega, t = 1,2,\ldots$
The basic recursion (PHGD) can be seen as a noisy Euler discretization of the continuous flow (PHGF), in the same way that ordinary stochastic gradient descent can be seen as a noisy discretization of gradient flows. In this way, (PHGD) is subject to the same difficulties underlying the analysis of stochastic gradient descent methods; we address these challenges in the next section.
4 Convergence analysis and results
In this section, we present our main results regarding the convergence of (PHGD) in hidden monotone games. Because we are interested not only on the asymptotic convergence of the method but also on its rate of convergence, we begin this section by defining a suitable merit function for each class of hidden monotone games under consideration; subsequently, we present our main results in
Section 4.2, where we also discuss the main technical tools enabling our analysis. To streamline our presentation, all proofs are deferred to Appendix B.
4.1. Merit functions and convergence metrics. In the variational context of (SVI)/(MVI), the quality of a candidate solution  $\hat{x}$  is typically evaluated by means of the restricted merit function
$$
\operatorname {G a p} _ {\mathcal {C}} (\hat {x}) := \sup  _ {x \in \mathcal {C}} \langle g (x), \hat {x} - x \rangle \tag {12}
$$
where the "test domain"  $\mathcal{C}$  is a relatively open subset of  $\mathcal{X}$  [3]. The raison d'etre of this definition is that, if  $g$  is monotone and  $x^{*} \in \mathcal{C}$  is a solution of (SVI)/(MVI), we have
$$
\langle g (x), x ^ {*} - x \rangle \leq \langle g \left(x ^ {*}\right), x ^ {*} - x \rangle \leq 0 \quad \text {f o r a l l} x \in \mathcal {C}, \tag {13}
$$
so the supremum in (B.6) cannot be too positive if  $\hat{x}$  is an approximate solution of (SVI)/(MVI). This is encoded in the following lemma, which, among others, justifies the terminology "merit function":
Lemma 2. Suppose that  $g$  is monotone. If  $\hat{x}$  is a solution of (SVI)/(MVI), we have  $\mathrm{Gap}_{\mathcal{C}}(\hat{x}) = 0$  whenever  $\hat{x} \in \mathcal{C}$ . Conversely, if  $\mathrm{Gap}_{\mathcal{C}}(\hat{x}) = 0$  and  $\mathcal{C}$  is a neighborhood of  $\hat{x}$  in  $\mathcal{X}$ , then  $\hat{x}$  is a solution of (SVI)/(MVI).
Lemma 2 extends similar statements by Auslender & Teboulle [3] and Nesterov [33]; the precise variant that we state above can be found in [1], but, for completeness, we provide a proof in Appendix B.
Now, since a latent variable profile  $x^{*} = \chi(\theta^{*})$  solves (SVI) if and only if the control variable configuration  $\theta^{*}$  is Nash equilibrium of the base game, the quality of a candidate solution  $\hat{\theta} \in \Theta$  with  $\hat{x} = \chi(\hat{\theta})$  can be assessed by the induced gap function
$$
\operatorname {G a p} (\hat {\theta}) := \operatorname {G a p} _ {\mathcal {X}} (\chi (\hat {\theta})) = \sup  _ {x \in \mathcal {X}} \langle g (x), \hat {x} - x \rangle . \tag {14}
$$
Indeed, since  $\hat{x} = \chi(\hat{\theta}) \in \mathcal{X}$ , Lemma 2 shows that  $\mathrm{Gap}(\hat{\theta}) \geq 0$  with equality if and only if  $\hat{x}$  is a Nash equilibrium of the base game  $\Gamma$ . In this regard, Eq. (14) provides a valid equilibrium convergence metric for  $\Gamma$ , so we will use it freely in the sequel as such; for a discussion of alternative convergence metrics, we refer the reader to Appendix B.
4.2. Convergence results. We are now in a position to state our main results regarding the equilibrium convergence properties of (PHGD). To streamline our presentation, we present our results from coarser to finer, starting with games that admit a hidden merely monotone structure and stochastic gradient feedback, and refining our analysis progressively to games with a hidden strongly monotone structure and full gradient feedback.<sup>3</sup> In addition, to quantify this distortion between the game's latent and control layers, we will require a technical regularity assumption for the game's representation map  $\chi: \Theta \to \mathcal{X}$ .
Assumption 2. The singular values of the Jacobian  $\mathbf{J}(\theta)$  of the representation map  $\chi$  are bounded as
$$
\sigma_ {\min } ^ {2} \leq \operatorname {e i g} (\mathbf {J} (\theta) \mathbf {J} (\theta) ^ {\top}) \leq \sigma_ {\max } ^ {2} \tag {15}
$$
for some  $\sigma_{\mathrm{min}},\sigma_{\mathrm{max}}\in (0,\infty)$  and for all  $\theta \in \Theta$
With all this in hand, we begin by studying the behavior of (PHGD) in games with a hidden monotone structure.
Theorem 1 (PHGD in hidden monotone games). Suppose that players run (PHGD) in a hidden monotone game with learning rate  $\gamma_t \propto 1 / t^{1/2}$ . Then, under Assumptions 1 and 2, the averaged process  $\bar{\theta}_t \in \chi^{-1}(t^{-1} \sum_{s=1}^t x_s)$  enjoys the equilibrium convergence rate
$$
\mathbb {E} [ \operatorname {G a p} (\bar {\theta} _ {t}) ] = \mathcal {O} (\log t / \sqrt {t}). \tag {16}
$$
As far as we are aware, Theorem 1 is the first result of its kind in the hidden games literature – that is, describing the long-run behavior of a discrete-time algorithm with stochastic gradient input. At the same time, it is subject to two important limitations: the first is that the averaged state  $\bar{\theta}_t$  cannot
<sup>3</sup>As expected, convergence rates improve along the way.
be efficiently computed for general representation maps; second, even if it could, the  $\mathcal{O}(\log t / \sqrt{t})$  convergence rate is relatively slow. The two results that follow show that both limitations can be overcome in games with a hidden strongly monotone structure.
In this case (SVI)/(MVI) admits a (necessarily) unique solution  $x^{*}$  in the game's control space, so we will measure convergence in terms of the latent equilibrium distance
$$
\operatorname {E r r} (\hat {\theta}) := \frac {1}{2} \| \chi (\hat {\theta}) - x ^ {*} \| ^ {2}. \tag {17}
$$
With this in mind, we have the following convergence results:
Theorem 2 (PHGD in hidden strongly monotone games). Suppose that players run (PHGD) in a hidden  $\mu$ -strongly monotone game with  $\gamma_t = \gamma / t$  for some  $\gamma > \mu$ . Then, under Assumptions 1 and 2, the induced sequence of play  $\theta_t \in \Theta$ ,  $t = 1, 2, \ldots$ , enjoys the equilibrium convergence rate
$$
\mathbb {E} \left[ \operatorname {E r r} _ {\Theta} \left(\theta_ {t}\right) \right] = \mathcal {O} (1 / t). \tag {18}
$$
This rate is tight, even for standard strongly monotone games. To improve it further, we will need to assume that (PHGD) is run with full, deterministic gradients, i.e.,  $V_{t} = g(\theta_{t})$ . In this case, we obtain the following refinement of Theorem 2.
Theorem 3 (PHGD with full gradient feedback in hidden strongly monotone games). Suppose that players run (PHGD) in a hidden  $\mu$ -strongly monotone game with full gradient feedback, and a sufficiently small learning rate  $\gamma > 0$ . Then, under Assumptions 1 and 2, the induced sequence of play  $\theta_t \in \Theta$ ,  $t = 1, 2, \ldots$ , converges to equilibrium at a geometric rate, i.e.,
$$
\operatorname {E r r} _ {\Theta} \left(\theta_ {t}\right) = \mathcal {O} \left(\rho^ {t}\right) \tag {19}
$$
for some constant  $\rho \in (0,1)$  that depends only on the primitives of  $\Gamma$  and the representation map  $\chi$ .
Importantly, up to logarithmic factors, the convergence rates of Theorems 1-3 mirror the corresponding rates for learning in monotone games. This take-away is particularly important as it shows that, when it exists, a hidden convex structure can be exploited to the greatest possible degree, without any loss of speed in convergence relative to standard, non-hidden convex problems.
The proofs of Theorems 1-3 are quite involved, so we defer the details to Appendix B. That said, to give an idea of the technical steps involved, we provide below (without proof) two lemmas that play a pivotal role in our analysis. The first one hinges on a transformation of the problem's defining vector field  $v(\theta) = (\nabla_i \ell_i(\theta))_{i \in \mathcal{N}}$  which, coupled with the specific choice of preconditioner  $\mathbf{P}_i(\theta_i) = [\mathbf{J}_i(\theta_i)^\top \mathbf{J}_i(\theta_i)]^+$  in (PHGD) allows us to effectively couple the latent and control layers of the problem in a "covariant" manner:
Lemma 3. Fix some  $\hat{x} \in \mathcal{X}$ , and consider the energy function  $E(\theta; \hat{x}) = (1/2) \| \chi(\theta) - \hat{x} \|^2$ . Then, for all  $\theta \in \Theta$ , we have
$$
\mathbf {J} (\theta) \mathbf {P} (\theta) \nabla_ {\theta} E (\theta ; \hat {x}) = \chi (\theta) - \hat {x}. \tag {20}
$$
Our second intermediate result builds on Lemma 3 and provides a "template inequality" for the energy function  $E(\theta; \hat{x})$  in the spirit of [7, 11, 18].
Lemma 4 (Template inequality). Suppose that Assumptions 1 and 2 hold. Then, with notation as in Lemma 3, the sequence  $E_{t} \coloneqq (1 / 2)\| \chi (\theta_{t}) - \hat{x}\|^{2}, t = 1,2,\ldots,$  satisfies
$$
E _ {t + 1} \leq E _ {t} - \gamma_ {t} g \left(x _ {t}\right) ^ {\intercal} \left(x _ {t} - \hat {x}\right) + \gamma_ {t} \phi_ {t} + \gamma_ {t} ^ {2} \psi_ {t}, \tag {21}
$$
where  $x_{t}\coloneqq \chi (\theta_{t})$ ,  $\phi_t\coloneqq (\mathbf{J}(\theta_t)\mathbf{P}(\theta_t)V_t - g(x_t))^{\intercal}(x_t - \hat{x})$  and  $\psi_t$  is a random error sequence with  $\sup_t\mathbb{E}[\psi_t] < \infty$ .
This inequality plays a pivotal role in our analysis because it allows us to couple the restricted merit function (B.8) in the game's control space with the evolution of the algorithm's quasi-Lyapunov function in the game's latent space. We provide the relevant details and calculations in Appendix B.
This section demonstrates our method's applicability in a couple of different and insightful setups. Technical details of those setups, as well as additional experimental results are deferred to the supplementary material. We start with a regularized version of Matching Pennies zero-sum game where the players' strategies are controlled by two individual preconfigured differentiable MLPs. Each MLP acts as the player's representation map  $\chi_{i}$ , which for each input  $\theta_{i}$  outputs a uni-dimensional latent variable  $x_{i} = \chi_{i}(\theta_{i})$  guaranteed to lie in  $\mathcal{X}_i\equiv [0,1]$ ; the player's latent space.
Figure 4 illustrates the trajectory of (PHGD), represented by the black curve. The algorithm employs a constant step-size of 0.01 and is initialized at the arbitrary state (1.25, 2.25) in the control variables' space. The color map in the figure serves as a visual representation of the level sets associated with the proposed energy function (5). Notably, the trajectory of the algorithm intersects each of the energy function's level sets at most once, indicating its non-cycling behavior, which is an issue that shows up often in the equilibration task. Due to the design of our hidden maps, the stabilization at the point  $(0,0)$  corresponds to the  $(\chi_{1}(0),\chi_{2}(0)) = (\frac{1}{2},\frac{1}{2})$  the unique equilibrium of the game.
In the second, more complex, example, we consider a strongly-monotone regularized modification of an (atomic) El Farol Bar congestion game among  $N = 30$  players [2,
43]. In this setup, we let the control space of each player  $i$  be multi-dimensional, namely,  $\Theta_i \equiv \mathbb{R}^5$ ,  $i \in \mathcal{N}$ , and, as in the previous example, the representation map of each player is instantiated by some preconfigured differentiable MLP whose output  $x_i \coloneqq \chi_i(\theta_i)$  is guaranteed to lie in  $[0,1]$ . The MLP's output is going to be the probability with which player  $i$  visits the El Farol bar. For the interested reader, further details, including technical specification of the MLPs, and loss functions of the games, can be found in the supplementary material. Figure 5 provides a comparative analysis of
Figure 4: The trajectory of (PHGD) in a hidden Matching Pennies game over the sublevel sets of the energy function in (5)
Figure 5: The evolution of the  $L^2$  error function  $\mathrm{Err}_{\mathcal{X}}(\chi(\theta))$  of (PHGD) and gradient descent (GD) with constant step-size 0.01 in the regularized games of Matching Pennies and El Farol Bar as depicted in a semilogarithmic scale. In the Matching Pennies game (left) we depict a single trajectory initialized at (1.25, 2.25), while in the El Farol Bar game (right) game we depict the mean and confidence bounds of 100 random trajectories.
the performance between (PHGD), and the standard GD method in the aforementioned two game scenarios. In Figure 5 (left) we explore the Matching Pennies game, where GD exhibits slightly erratic behavior. Despite eventually converging to the game's equilibrium point, GD's convergence rate, in this case, can be described as linear at best. In Figure 5 (right), we examine the El Farol Bar game. Interestingly, in this highly complex setup, GD fails to reach the equilibrium point entirely. In contrast, (PHGD) not only manages to converge in both of these setups, but it also consistently maintains an exponential rate of convergence. This stark difference underscores the efficacy and robustness of our algorithm.
This paper proposed a new algorithmic framework with strong formal convergence guarantees in a general class non-convex games with a latent monotone structure. Our algorithmic method – which we call preconditioned hidden gradient descent – relies on an appropriately chosen gradient preconditioning scheme akin to natural gradient ideas. Our class of games combines the useful structure of monotone operators as well as the notion of latent/hidden variables that arise in neural networks and can thus model numerous AI applications. Our results indicate the possibility of deep novel algorithmic ideas emerging at the intersection of game theory, non-convex optimization and ML and offers exciting directions for future work.
This research is supported in part by the National Research Foundation, Singapore and DSO National Laboratories under its AI Singapore Program (AISG Award No: AISG2-RP-2020-016), grant PIESGP-AI-2020-01, AME Programmatic Fund (Grant No.A20H6b0151) from A*STAR and Provost's Chair Professorship grant RGEPPV2101, and project MIS 5154714 of the National Recovery and Resilience Plan Greece 2.0 funded by the European Union under the NextGenerationEU Program. PM is a member of the Archimedes Research Unit - Athena RC - Department of Mathematics, National & Kapodistrian University of Athens, and gratefully acknowledges financial support by the French National Research Agency (ANR) in the framework of the "Investissements d'avenir" program (ANR-15-IDEX-02), the LabEx PERSYVAL (ANR-11-LABX-0025-01), and MIAI@Grenoble Alpes (ANR-19-P3IA-0003).
[42] Vlatakis-Gkaragkounis, E.-V., Flokas, L., and Piliouras, G. Solving min-max optimization with hidden structure via gradient descent ascent. Advances in Neural Information Processing Systems, 34:2373-2386, 2021.
[43] Whitehead, D. et al. The el farol bar problem revisited: Reinforcement learning in a potential game. ESE discussion papers, 186, 2008.
Organization of the appendix
A Omitted proofs from Section 3 14
Proof of Lemma 1 14
Proof of Proposition 1 15
B Auxiliary results from Section 4 15
B.1 Convergence metrics and merit functions 15
Proof of Lemma 2 15
B.2 Convergence analysis and template inequalities 17
Proof of Lemma 3 17
Proof of Lemma 4 18
C Proofs of Theorems 1-3 19
Proof of Theorem 1 19
Proof of Theorem 2 21
Proof of Theorem 3 22
D Experiments 23
A Omitted proofs from Section 3
In this first appendix, we provide the technical proofs of our results for the continuous-time dynamics (PHGF), namely Lemma 1 and Proposition 1. For convenience, we restate the relevant results as needed.
where, as per Section 2,  $\mathbf{J}_i(\theta_i) \in \mathbb{R}^{d_i \times m_i}$  denotes the Jacobian matrix of the map  $\chi_i \colon \Theta_i \to \mathcal{X}_i$ . More compactly, letting  $\mathbf{P} \coloneqq \bigoplus_i \mathbf{P}_i$  denote the block-diagonal ensemble of the players' individual preconditioning matrices  $\mathbf{P}_i$  (and suppressing control variable arguments for concision), we have
$$
\dot {E} = - g (x) ^ {\intercal} \cdot \mathbf {J P J} ^ {\intercal} \cdot \left(x - x ^ {*}\right) \tag {10}
$$
Proof. Observe that if we expand the dynamical system of equations in (PHGF) with respect to each individual coordinate, we have that for each player  $i \in \mathcal{N}$ , for all profiles  $\theta \in \Theta$ , and for all coordinates  $\alpha \in \{1, \ldots, m_i\}$ :
$$
\dot {\theta} _ {i \alpha} = \sum_ {\beta = 1} ^ {m _ {i}} \mathbf {P} _ {i \alpha \beta} \left(\theta_ {i}\right) \frac {\partial \ell_ {i} (\theta)}{\partial \theta_ {i \beta}}. \tag {A.1}
$$
Now, since  $\Gamma$  has a hidden monotone structure, and due to the decomposition  $\ell_i(\theta) = f_i(x)$ , we can expand the above as