SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\dot {\theta} _ {i \alpha} = \sum_ {\beta = 1} ^ {m _ {i}} \mathbf {P} _ {i \alpha \beta} \left(\theta_ {i}\right) \sum_ {r = 1} ^ {d _ {i}} \frac {\partial f _ {i} (x)}{\partial x _ {i r}} \frac {\partial x _ {i r}}{\partial \theta_ {i \beta}}. \tag {A.2}
$$
Furthermore, if we expand the left-hand side (LHS) of (9) we also get
$$
\dot {E} (\theta) = \sum_ {i = 1} ^ {N} \sum_ {\alpha = 1} ^ {m _ {i}} \frac {\partial E (\theta)}{\partial \theta_ {i \alpha}} \dot {\theta} _ {i \alpha}, \tag {A.3}
$$
where each of the summands of the above expression can also be expanded further to
$$
\frac {\partial E (\theta)}{\partial \theta_ {i \alpha}} \dot {\theta} _ {i \alpha} = - \sum_ {l = 1} ^ {d _ {i}} \left(x _ {i l} - x _ {i l} ^ {*}\right) \frac {\partial x _ {i l}}{\partial \theta_ {i \alpha}} \dot {\theta} _ {i \alpha}. \tag {A.4}
$$
Putting everything together (9) follows by some trivial substitutions.
Proof. First, observe that for any player  $i \in \mathcal{N}$ , and  $\theta_i \in \Theta_i$ , since the player's representation map  $\chi_i$  is faithful, i.e.,  $\mathbf{J}_i(\theta_i)$  is maximal rank, it holds:
$$
\begin{array}{l} \mathbf {J} _ {i} \left(\theta_ {i}\right) \mathbf {P} _ {i} \left(\theta_ {i}\right) \mathbf {J} _ {i} \left(\theta_ {i}\right) ^ {\intercal} = \mathbf {J} _ {i} \left(\theta_ {i}\right) \mathbf {J} _ {i} \left(\theta_ {i}\right) ^ {+} \left[ \mathbf {J} _ {i} \left(\theta_ {i}\right) ^ {+} \right] ^ {\intercal} \mathbf {J} _ {i} \left(\theta_ {i}\right) ^ {\intercal} \\ = \left[ \mathbf {J} _ {i} \left(\theta_ {i}\right) \mathbf {J} _ {i} \left(\theta_ {i}\right) ^ {+} \right] ^ {\intercal} \tag {A.5} \\ = \mathbf {I}. \\ \end{array}
$$
That is,  $\sum_{\alpha = 1}^{m_i}\sum_{\beta = 1}^{m_i}\frac{\partial x_{il}}{\partial\theta_{i\alpha}}\frac{\partial x_{ir}}{\partial\theta i\beta}\mathbf{P}_{i\alpha \beta}(\theta_i) = \delta_{lr}$  for all  $l,r\in \{1,\dots ,d_i\}$ , where  $\delta_{lr}$  is the Kronecker delta.
Now, by Lemma 1 we have that:
$$
\begin{array}{l} \dot {E} (\theta) = - \sum_ {i = 1} ^ {N} \sum_ {l = 1} ^ {d _ {i}} \sum_ {r = 1} ^ {d _ {i}} \delta_ {l r} \left(x _ {i l} - x _ {i l} ^ {*}\right) \frac {\partial f _ {i} (x)}{\partial x _ {i r}} \\ = - \sum_ {i = 1} ^ {N} \sum_ {l = 1} ^ {d _ {i}} \left(x _ {i l} - x _ {i l} ^ {*}\right) \frac {\partial f _ {i} (x)}{\partial x _ {i l}} \tag {A.6} \\ = - \left\langle g (x), x - x ^ {*} \right\rangle . \\ \end{array}
$$
which is negative, since  $x^{*}$  is an optimizer of  $f$ , i.e., it satisfies the (MVI) due to monotonicity of  $g$ . Additionally, since  $g$  is strictly monotone,  $x^{*}$  is the unique optimizer of  $f$ , and, therefore, the above condition holds with equality if and only if  $x = x^{*}$ .
B Auxiliary results from Section 4
B.1. Convergence metrics and merit functions. In this appendix, we provide the technical scaffolding required for the analysis of (PHGD), namely Lemmas 2-4 in Section 3.2. As before, we restate the relevant results as needed.
Proof of Lemma 2. Let  $\hat{x} \in \mathcal{X}$  be a solution of (SVI)/(MVI) so  $\langle g(\hat{x}), x - \hat{x} \rangle \geq 0$  for all  $x \in \mathcal{X}$ . Then, by the monotonicity of  $g$ , we get:
$$
\begin{array}{l} \langle g (x), \hat {x} - x \rangle \leq \langle g (x) - g (\hat {x}), \hat {x} - x \rangle + \langle g (\hat {x}), \hat {x} - x \rangle \\ = - \left\langle g (\hat {x}) - g (x), \hat {x} - x \right\rangle - \left\langle g (\hat {x}), x - \hat {x} \right\rangle \leq 0, \tag {B.1} \\ \end{array}
$$
so  $\operatorname{Gap}_{\mathcal{C}}(\hat{x}) \leq 0$ . On the other hand, if  $\hat{x} \in \mathcal{C}$ , we also get  $\operatorname{Gap}(\hat{x}) \geq \langle g(\hat{x}), \hat{x} - \hat{x} \rangle = 0$ , so we conclude that  $\operatorname{Gap}_{\mathcal{C}}(\hat{x}) = 0$ .
For the converse statement, assume that  $\mathrm{Gap}_{\mathcal{C}}(\hat{x}) = 0$  for some  $\hat{x} \in \mathcal{C}$  and suppose that  $\mathcal{C}$  contains a neighborhood of  $\hat{x}$  in  $\mathcal{X}$ . We then claim that  $\hat{x}$  is a solution of (MVI) over  $\mathcal{C}$ , i.e.,
$$
\langle g (x), x - \hat {x} \rangle \geq 0 \quad \text {f o r a l l} x \in \mathcal {C}. \tag {B.2}
$$
To see this, assume to the contrary that there exists some  $x_{1}\in \mathcal{C}$  such that
$$
\langle g (x _ {1}), x _ {1} - \hat {x} \rangle <   0 \tag {B.3}
$$
so, in turn, we get
$$
0 = \operatorname {G a p} _ {\mathcal {C}} (\hat {x}) \geq \left\langle g \left(x _ {1}\right), \hat {x} - x _ {1} \right\rangle > 0, \tag {B.4}
$$
a contradiction.
With this intermediate "local" result in hand, we are now in a position to prove that  $\hat{x}$  solves (SVI). Indeed, if we suppose to the contrary that there exists some  $z_{1} \in \mathcal{X}$  such that  $\langle g(\hat{x}), z_{1} - \hat{x} \rangle < 0$ , then, by the continuity of  $g$ , there exists a neighborhood  $\mathcal{U}'$  of  $\hat{x}$  in  $\mathcal{X}$  such that
$$
\langle g (x), z _ {1} - x \rangle <   0 \quad \text {f o r a l l} x \in \mathcal {U} ^ {\prime}. \tag {B.5}
$$
Hence, assuming without loss of generality that  $\mathcal{U}' \subset \mathcal{U} \subset \mathcal{C}$  (the latter assumption due to the assumption that  $\mathcal{C}$  contains a neighborhood of  $\hat{x}$ ), and taking  $\lambda > 0$  sufficiently small so that  $x = \hat{x} + \lambda (z_1 - \hat{x}) \in \mathcal{U}'$ , we get that  $\langle g(x), x - \hat{x} \rangle = \lambda \langle g(x), z_1 - \hat{x} \rangle < 0$ , in contradiction to (B.2). We thus conclude that  $\hat{x}$  is a solution of (SVI) - and hence, by monotonicity, also of (MVI).
For intuition, we discuss below some other merit functions that could be considered as valid convergence metrics. In this regard, the first thing to note is that the definition of  $\mathrm{Gap}(\hat{\theta})$  effectively goes through the game's latent space, so it is natural to ask whether a similar merit function can be defined directly on the game's control space. To do so, it will be more convenient to start with a linearized variant of  $\mathrm{Gap}_{\mathcal{X}}$  defined over the cone of tangent directions at  $\hat{x}$ , namely the so-called "tangent residual gap"
$$
\operatorname {T G a p} _ {\mathcal {X}} (\hat {x}) := - \min  _ {z \in \operatorname {T C} _ {\mathcal {X}} (\hat {x}), \| z \| \leq 1} \langle g (\hat {x}), z \rangle \tag {B.6}
$$
i.e., the maximum "ascent" step  $\langle -g(x),z\rangle$  over all admissible displacement directions  $z$  from  $\hat{x}$ . Just like  $\mathrm{Gap}_{\mathcal{C}}$ , this linearized merit function correctly identifies solutions of (SVI)/(MVI):
Lemma B.1. For all  $\hat{x} \in \mathcal{X}$ , we have  $\mathrm{TGap}_{\mathcal{X}}(\hat{x}) \geq 0$  with equality if and only if  $\hat{x}$  solves (SVI)/(MVI).
Proof. Let  $\hat{x} \in \mathcal{X}$  be some arbitrary profile of latent variables. By definition, we have that  $\mathrm{TGap}_{\mathcal{X}}(\hat{x}) = -\min_{z \in \mathrm{TC}_{\mathcal{X}}(\hat{x}), \|z\| \leq 1} \langle g(\hat{x}), z \rangle$ , where  $\mathrm{TC}_{\mathcal{X}}(\hat{x})$  is the tangent cone to  $\mathcal{X}$  at  $\hat{x}$ . Observe that since  $\mathrm{TC}_{\mathcal{X}}(\hat{x})$  is a cone,  $0 \in \mathrm{TC}_{\mathcal{X}}(\hat{x})$ , so  $\mathrm{TGap}_{\mathcal{X}}(\hat{x}) \geq -\langle g(\hat{x}), 0 \rangle = 0$ . Now assume that  $\mathrm{TGap}_{\mathcal{X}}(\hat{x}) = 0$ . Then, the following are equivalent:
$$
\begin{array}{l} \operatorname {T G a p} _ {\mathcal {X}} (\hat {x}) = 0 \iff - \min  _ {z \in \operatorname {T C} _ {\mathcal {X}} (\hat {x}), \| z \| \leq 1} \langle g (\hat {x}), z \rangle = 0 \\ \Longleftrightarrow \langle g (\hat {x}), z \rangle \geq 0 \quad \text {f o r a l l} z \in \mathrm {T C} _ {\mathcal {X}} (\hat {x}), \| z \| \leq 1 \tag {B.7} \\ \Longleftrightarrow \langle g (\hat {x}), z \rangle \geq 0 \quad \text {f o r a l l} z \in \operatorname {T C} _ {\mathcal {X}} (\hat {x}), \\ \end{array}
$$
where the last equivalence follows because  $\mathrm{TC}_{\mathcal{X}}(\hat{x})$  is a cone. Rearranging the terms, we may equivalently write  $\langle -g(\hat{x}),z\rangle \leq 0$  for all  $z\in \mathrm{TC}_{\mathcal{X}}(\hat{x})$ . Notice that, by the definition of the tangent cone, the latter is equivalent to  $\langle -g(\hat{x}),\hat{x} -z\rangle \leq 0$  for all  $z\in \mathcal{X}$ . Therefore,  $\mathrm{TGap}_{\mathcal{X}}(\hat{x}) = 0$  if and only if  $\hat{x}$  satisfies (SVI).
Now, if  $\theta^{*}$  is a Nash equilibrium of the base game, the latent variable configuration  $x^{*} = \chi (\theta^{*})$  solves (SVI)/(MVI), so  $\mathrm{TGap}_{\mathcal{X}}$  is a valid equilibrium convergence metric for  $\Gamma$ . However, since  $\mathrm{TGap}_{\mathcal{X}}$  is still defined on the game's latent space, it does not give a straightforward way of defining the quality of a candidate solution directly on the game's control space. To that end, one could consider the gap function
$$
\operatorname {T G a p} _ {\Theta} (\hat {\theta}) := - \min  _ {\eta \in \mathbb {R} ^ {m}, \| \eta \| \leq 1} \left\langle v (\hat {\theta}), \eta \right\rangle = \| v (\hat {\theta}) \| _ {*} \tag {B.8}
$$
where  $v(\theta) \coloneqq (\nabla_i\ell_i(\theta))_{i\in \mathcal{N}}$  collects the players' loss gradients relative to their control variables, and  $\| \cdot \|_*$  denotes its dual norm (so  $\mathrm{TGap}_{\Theta}(\hat{\theta})$  is evaluated directly on the game's control space). Of course, as control variables are mapped to latent variables,  $\chi$  introduces a certain distortion (due to nonlinearities). This distortion can be quantified by the following lemma (see also Fig. 6 above):
Lemma B.2. Let  $\hat{x} = \chi (\hat{\theta})$  for some  $\hat{\theta}\in \Theta$ . We then have
$$
\sigma_ {\min } \operatorname {T G a p} _ {\mathcal {X}} (\hat {x}) \leq \operatorname {T G a p} _ {\Theta} (\hat {\theta}) \leq \sigma_ {\max } \operatorname {T G a p} _ {\mathcal {X}} (\hat {x}) \tag {B.9}
$$
In particular,  $\mathrm{TGap}_{\Theta}(\hat{\theta}) \geq 0$  for all  $\hat{\theta} \in \Theta$ , with equality if and only if  $\hat{\theta}$  is a Nash equilibrium of  $\Gamma$ .
<sup>4</sup>In the above,  $\mathrm{TC}_{\mathcal{X}}(\hat{x})$  denotes the tangent cone to  $\mathcal{X}$  at  $\hat{x}$ , that is, the closure of the set of all rays emanating from  $\hat{x}$  and intersecting  $\mathcal{X}$  in at least one other point.
Figure 6: The distortion of a ball by  $\mathbf{J}(\theta)$ .
Proof. To begin with, let us define the balls  $\mathbb{B}_{\Theta} \equiv \{\eta \in \mathbb{R}^m : \| \eta \| \leq 1\}$ , and  $\mathbb{B}_{\mathcal{X}} \equiv \{z \in \mathbb{R}^d : \| z \| \leq 1\}$ . Now, let  $\hat{\theta} \in \Theta$  be some arbitrary profile of control variables. By Assumption 2, we have that the singular values of the Jacobian  $\mathbf{J}(\hat{\theta})$  of the representation map  $\chi$  are bounded from above and below by  $\sigma_{\max} < \infty$  and  $\sigma_{\min} > 0$  respectively. That is,  $\sigma_{\min} \| \eta \| \leq \| \mathbf{J}(\hat{\theta})\eta \| \leq \sigma_{\max} \| \eta \|$  for all  $\eta \in \mathbb{R}^m$ , which implies that  $\sigma_{\min} \mathbb{B}_{\mathcal{X}} \subseteq \mathbf{J}(\hat{\theta})\mathbb{B}_{\Theta} \subseteq \sigma_{\max} \mathbb{B}_{\mathcal{X}}$ . By definition, we also have that,  $v(\hat{\theta}) = \mathbf{J}(\hat{\theta})^\intercal g(\chi(\hat{\theta})) = \mathbf{J}(\hat{\theta})^\intercal g(\hat{x})$ , and, hence
$$
\operatorname {T G a p} _ {\Theta} (\hat {\theta}) = - \min  _ {\eta \in \mathbb {B} _ {\Theta}} \left\langle v (\hat {\theta}), \eta \right\rangle = - \min  _ {z \in \mathbf {J} (\hat {\theta}) \mathbb {B} _ {\Theta}} \left\langle g (\hat {x}), z \right\rangle \tag {B.10}
$$
Now, recall that the set of Nash equilibria  $\Theta^{*} \subset \Theta$  is non-empty. In particular, since the  $\Theta \equiv \mathbb{R}^{m}$  is an open set, the solution set lies in an open set, which, in conjunction with the fact that the representation maps are faithful representations of  $\Theta$  to  $\mathcal{X}$ , also implies that  $\chi(\Theta^{*})$  lies in the interior of  $\mathcal{X}$ . Finally, since  $\chi$  is smooth, and in conjunction with the previous observations, it follows that  $\mathrm{TGap}_{\Theta}(\hat{\theta}) = \|v(\hat{\theta})\|$  and  $\mathrm{TGap}_{\mathcal{X}}(\hat{x}) = \|g(\hat{x})\|$ , hence it is easy to see from the above discussion that
$$
\sigma_ {\min } \operatorname {T G a p} _ {\mathcal {X}} (\chi (\hat {\theta})) \leq \operatorname {T G a p} _ {\Theta} (\hat {\theta}) \leq \sigma_ {\max } \operatorname {T G a p} _ {\mathcal {X}} (\chi (\hat {\theta})) \tag {B.11}
$$
and our proof is complete.
In view of the above discussion, the reader may wonder why not use the tangent gap  $\mathrm{TGap}_{\Theta}$  directly as a performance metric. The reason for this is that, if the solutions  $x^{*}$  of (SVI)/(MVI) do not belong to the image of  $\chi$ , the dual norm of  $v(\hat{\theta})$  may be too stringent as a performance metric as it does not vanish near an equilibrium (e.g., think of the operator  $g(x) = x$  for  $x$  between 0 and 1). In this case, equilibria can be approximated to arbitrary accuracy but never attained, so the latent gap functions Gap and Err are more appropriate as performance measures.
B.2. Convergence analysis and template inequalities.
Proof. Let  $i \in \mathcal{N}$  be some arbitrary player, let  $\theta \in \Theta$  be some arbitrary profile of control variables, and let  $\eta \in \mathbb{R}^{d_i}$  be some arbitrary vector. Then for each coordinate  $\alpha \in \{1, \dots, m_i\}$ , and latent profile  $\hat{x} \in \mathcal{X}$ , we have that
$$
\frac {\partial E (\theta ; \hat {x})}{\partial \theta_ {i \alpha}} = \sum_ {j = 1} ^ {d _ {i}} \left(x _ {i j} - \hat {x} _ {i j}\right) \frac {\partial x _ {i j}}{\partial \theta_ {i \alpha}}. \tag {B.12}
$$
That is  $\nabla_{\theta_i}E(y;\hat{x}) = \mathbf{J}_i(\theta_i)^\top (x_i - \hat{x}_i)$ . Now, we can multiply both sides of the above expression with  $\mathbf{P}_i(\theta_i)$  to get
$$
\begin{array}{l} \mathbf {P} _ {i} \left(\theta_ {i}\right) \nabla_ {\theta_ {i}} E (\theta ; \hat {x}) = \mathbf {P} _ {i} \left(\theta_ {i}\right) \mathbf {J} _ {i} \left(\theta_ {i}\right) ^ {\intercal} \left(x _ {i} - \hat {x} _ {i}\right) \\ = \mathbf {J} _ {i} \left(\theta_ {i}\right) ^ {+} \left(\mathbf {J} _ {i} \left(\theta_ {i}\right) ^ {+}\right) ^ {\intercal} \mathbf {J} _ {i} \left(\theta_ {i}\right) ^ {\intercal} \left(x _ {i} - \hat {x} _ {i}\right) \tag {B.13} \\ = \mathbf {J} _ {i} \left(\theta_ {i}\right) ^ {+} \left(x _ {i} - \hat {x} _ {i}\right), \\ \end{array}
$$
where in the last equality we used the fact that the  $i$ -th player's representation map  $\chi_{i}$  is a faithful representation, i.e.,  $\mathbf{J}_i(\theta_i)$  is maximal rank. Finally, by expanding the LHS of (20), and applying the above simplification the result follows.
where  $x_{t} \coloneqq \chi(\theta_{t})$ ,  $\phi_{t} \coloneqq (\mathbf{J}(\theta_{t})\mathbf{P}(\theta_{t})V_{t} - g(x_{t}))^{\intercal}(x_{t} - \hat{x})$  and  $\psi_{t}$  is a random error sequence with  $\sup_{t}\mathbb{E}[\psi_{t}] < \infty$ .
Proof. For each  $t = 1,2,\ldots$  we expand  $x_{t + 1}$  using its second-order Taylor approximation at  $\theta_t$ ; i.e., for each player  $i = 1,\dots ,N$  and each coordinate  $l = 1,\dots ,d_{i}$ :
$$
\begin{array}{l} x _ {i l, t + 1} = \chi_ {i l} \left(\theta_ {t + 1}\right) \\ = \chi_ {i l} \left(\theta_ {i, t}\right) + \gamma_ {t} \left\langle \nabla \chi_ {i l} \left(\theta_ {i, t}\right), \theta_ {i, t + 1} - \theta_ {i, t} \right\rangle \tag {B.14} \\ + \gamma_ {t} ^ {2} \left(\theta_ {i, t + 1} - \theta_ {i, t}\right) ^ {\intercal} H _ {i l} \left(\tilde {\theta} _ {i, t}\right) \left(\theta_ {i, t + 1} - \theta_ {i, t}\right), \\ \end{array}
$$
where  $H_{il}(\tilde{\theta}_{i,t})$  is the Hessian of the latent map  $\chi_{il}$  at some convex combination  $\tilde{\theta}_{i,t}$  of  $\theta_{i,t}$  and  $\theta_{i,t+1}$ . Then, further expanding  $\theta_{t+1}$ , we also get
$$
x _ {i l, t + 1} = x _ {i l, t} - \gamma_ {t} \left\langle \nabla \chi_ {i l} \left(\theta_ {i, t}\right), \mathbf {P} _ {i, t} V _ {i, t} \right\rangle + \gamma_ {t} ^ {2} \left(\mathbf {P} _ {i, t} V _ {i, t}\right) ^ {\top} H _ {i l} \left(\tilde {\theta} _ {i, t}\right) \left(\mathbf {P} _ {i, t} V _ {i, t}\right). \tag {B.15}
$$
Plugging the above expansion to  $E(\theta_{t + 1};\hat{x})$  for arbitrary state  $\hat{x}$  we get
$$
\begin{array}{l} E _ {t + 1} = E \left(\theta_ {t + 1}; \hat {x}\right) \\ = \frac {1}{2} \sum_ {i = 1} ^ {N} \sum_ {l = 1} ^ {d _ {i}} \left(x _ {t + 1} - \hat {x}\right) ^ {2} \\ = \frac {1}{2} \sum_ {i = 1} ^ {N} \sum_ {l = 1} ^ {d _ {i}} \left[ x _ {i l, t} - \hat {x} _ {i l} - \gamma_ {t} \left\langle \nabla \chi_ {i l} \left(\theta_ {i, t}\right), \mathbf {P} _ {i, t} V _ {i, t} \right\rangle \right. \tag {B.16} \\ + \gamma_ {t} ^ {2} (\mathbf {P} _ {i, t} V _ {i, t}) ^ {\intercal} H _ {i l} (\tilde {\theta} _ {i, t}) (\mathbf {P} _ {i, t} V _ {i, t}) ] ^ {2} \\ = E _ {t} - \gamma_ {t} \sum_ {i = 1} ^ {N} \sum_ {l = 1} ^ {d _ {i}} \langle \nabla \chi_ {i l} (\theta_ {i, t}), \mathbf {P} _ {i, t} V _ {i, t} \rangle (x _ {i l, t} - \hat {x} _ {i l}) + \gamma_ {t} ^ {2} \eta_ {t} \\ = E _ {t} - \gamma_ {t} \left(\mathbf {J} _ {t} \mathbf {P} _ {t} V _ {t}\right) ^ {\intercal} \left(x _ {t} - \hat {x}\right) + \gamma_ {t} ^ {2} \eta_ {t}, \\ \end{array}
$$
where the second-order term  $\eta_t$  is given by the formula
$$
\begin{array}{l} \eta_ {t} = \sum_ {i = 1} ^ {N} \sum_ {l = 1} ^ {d _ {i}} \langle \nabla \chi_ {i l} (\theta_ {i, t}), \mathbf {P} _ {i, t} V _ {i, t} \rangle^ {2} \\ + \sum_ {i = 1} ^ {N} \sum_ {l = 1} ^ {d _ {i}} \left(\mathbf {P} _ {i, t} V _ {i, t}\right) ^ {\intercal} H _ {i l} \left(\tilde {\theta} _ {i, t}\right) \left(\mathbf {P} _ {i, t} V _ {i, t}\right) \left(x _ {i l, t} - \hat {x} _ {i l}\right) \tag {B.17} \\ - \gamma_ {t} \sum_ {i = 1} ^ {N} \sum_ {l = 1} ^ {d _ {i}} \langle \nabla \chi_ {i l} (\theta_ {i, t}), \mathbf {P} _ {i, t} V _ {i, t} \rangle (\mathbf {P} _ {i, t} V _ {i, t}) ^ {\intercal} H _ {i l} (\tilde {\theta} _ {i, t}) (\mathbf {P} _ {i, t} V _ {i, t}) \\ + \gamma_ {t} ^ {2} \sum_ {i = 1} ^ {N} \sum_ {l = 1} ^ {d _ {i}} [ (\mathbf {P} _ {i, t} V _ {i, t}) ^ {\intercal} H _ {i l} (\tilde {\boldsymbol {\theta}} _ {i, t}) (\mathbf {P} _ {i, t} V _ {i, t}) ] ^ {2}. \\ \end{array}
$$
For the second-order term, observe that by Assumption 2, we have that  $[\mathbf{J}_t^\intercal \mathbf{J}_t]^+\preceq \frac{1}{\sigma_{\min}^2}\mathbf{I}$ . Using that fact, it follows that
$$
\begin{array}{l} \sum_ {i = 1} ^ {N} \sum_ {l = 1} ^ {d _ {i}} \left\langle \nabla \chi_ {i l} \left(\theta_ {i, t}\right), \mathbf {P} _ {i, t} V _ {i, t} \right\rangle^ {2} = \left(\mathbf {J} _ {t} \mathbf {P} _ {t} V _ {t}\right) ^ {\intercal} \mathbf {J} _ {t} \mathbf {P} _ {t} V _ {t} \\ = V _ {t} ^ {\mathsf {T}} \left[ \mathbf {J} _ {t} ^ {\mathsf {T}} \mathbf {J} _ {t} \right] ^ {+} \mathbf {J} _ {t} ^ {\mathsf {T}} \mathbf {J} _ {t} \left[ \mathbf {J} _ {t} ^ {\mathsf {T}} \mathbf {J} _ {t} \right] ^ {+} V _ {t} \tag {B.18} \\ = V _ {t} ^ {\intercal} \left[ \mathbf {J} _ {t} ^ {\intercal} \mathbf {J} _ {t} \right] ^ {+} V _ {t} \\ \leq \frac {1}{\sigma_ {\operatorname* {m i n}} ^ {2}} \| V _ {t} \| ^ {2}, \\ \end{array}
$$
Furthermore, since the representation maps  $\chi_{i}$  are  $\beta$ -Lipschitz smooth for some Lipschitz modulus  $\beta$ , we have that  $H_{i,l}(\overline{\theta}_{i,t}) \preceq \beta \mathbf{I}$  for each player  $i$  and coordinate  $l$ . Consequently, we have that
$$
\left(\mathbf {P} _ {i, t} V _ {i, t}\right) ^ {\intercal} H _ {i l} \left(\tilde {\theta} _ {i, t}\right) \left(\mathbf {P} _ {i, t} V _ {i, t}\right) \leq \beta V _ {i, t} ^ {\intercal} \mathbf {P} _ {i, t} ^ {2} V _ {i, t} \leq \frac {\beta}{\sigma_ {\min } ^ {4}} \| V _ {i, t} \| ^ {2} \leq \frac {\beta}{\sigma_ {\min } ^ {4}} \| V _ {t} \| ^ {2}. \tag {B.19}
$$
Let  $D = \mathrm{diam}(\chi)$ , then, by applying the Cauchy-Schwarz inequality, we also get that
$$
\eta_ {t} \leq \frac {1}{\sigma_ {\min } ^ {2}} \| V _ {t} \| ^ {2} + \frac {d D \sqrt {\beta}}{\sigma_ {\min } ^ {2}} \| V _ {t} \| + \gamma \frac {\sqrt {\beta}}{\sigma_ {\min } ^ {3}} \| V _ {t} \| ^ {2} + \gamma^ {2} \frac {d \beta^ {2}}{\sigma_ {\min } ^ {8}} \| V _ {t} \| ^ {2}, \tag {B.20}
$$
where  $\gamma = \sup_{t=1,2,\ldots} \gamma_t$ , and  $d = \sum_{i=1}^{N} d_i$ .
Finally, let us consider the first-order term of (B.16). Let  $v_{t} = (\nabla_{i}\ell_{i}(\theta_{t}))_{i\in \mathcal{N}}$ , and  $g_{t} = g(x_{t})$ . Then, by definition, we also have that  $v_{t} = \mathbf{J}_{t}^{\intercal}g_{t}$ . Moreover, recall that, by construction,  $\mathbf{J}_t\mathbf{P}_t\mathbf{J}_t^\intercal = \mathbf{I}$ . Consequently, we can write
$$
\begin{array}{l} \left(\mathbf {J} _ {t} \mathbf {P} _ {t} V _ {t}\right) ^ {\intercal} (x _ {t} - \hat {x}) = \left(\mathbf {J} _ {t} \mathbf {P} _ {t} v _ {t}\right) ^ {\intercal} (x _ {t} - \hat {x}) + \left[ \mathbf {J} _ {t} \mathbf {P} _ {t} (V _ {t} - v _ {t}) \right] ^ {\intercal} (x _ {t} - \hat {x}) \\ = \left(\mathbf {J} _ {t} \mathbf {P} _ {t} \mathbf {J} _ {t} ^ {\intercal} g _ {t}\right) ^ {\intercal} \left(x _ {t} - \hat {x}\right) + \left[ \mathbf {J} _ {t} \mathbf {P} _ {t} \left(V _ {t} - \mathbf {J} _ {t} ^ {\intercal} g _ {t}\right) \right] ^ {\intercal} \left(x _ {t} - \hat {x}\right) \tag {B.21} \\ = g _ {t} ^ {\intercal} (x _ {t} - \hat {x}) + (\mathbf {J} _ {t} \mathbf {P} _ {t} V _ {t} - g _ {t}) ^ {\intercal} (x _ {t} - \hat {x}), \\ \end{array}
$$
C Proofs of Theorems 1-3
We are now in a position to prove Theorems 1-3 regarding the convergence properties of (PHGD). We proceed sequentially, restating the relevant results as needed.
$$
\mathbb {E} \left[ \operatorname {G a p} \left(\bar {\theta} _ {t}\right) \right] = \mathcal {O} \left(\log t / \sqrt {t}\right). \tag {16}
$$
Proof. Let  $\theta_1 \in \Theta$  be some arbitrary initialization of the algorithm, and let  $\tilde{x} \in \mathcal{X}$  be some arbitrary profile of latent variables. Next, by Lemma 4, we have that for all  $t \geq 1$ :
$$
E _ {t + 1} \leq E _ {t} - \gamma_ {t} \langle g (x _ {t}), x _ {t} - \hat {x} \rangle + \gamma_ {t} \phi_ {t} + \gamma_ {t} ^ {2} \psi_ {t}
$$
$$
\phi_ {t} = \left(\mathbf {J} \left(\theta_ {t}\right) \mathbf {P} \left(\theta_ {t}\right) V _ {t} - g \left(x _ {t}\right)\right) ^ {\intercal} \left(x _ {t} - \hat {x}\right) \tag {C.1}
$$
$$
\psi_ {t} = \kappa \| V _ {t} \| ^ {2}, \quad \text {f o r s o m e} \kappa > 0.
$$
Rearranging the terms, the above is equivalent to
$$
\gamma_ {t} \left\langle g \left(x _ {t}\right), x _ {t} - \tilde {x} \right\rangle \leq E _ {t} - E _ {t + 1} + \gamma_ {t} \phi_ {t} + \gamma_ {t} ^ {2} \psi_ {t}. \tag {C.2}
$$
Furthermore, by the monotonicity of  $g$ , we also have that  $\left\langle g(x_{t}) - g(\tilde{x}), x_{t} - \tilde{x} \right\rangle \geq 0$ , and by combining the two, we get that for all  $t \geq 1$ :
$$
\left. \gamma_ {t} \left\langle g (\tilde {x}), x _ {t} - \tilde {x} \right\rangle \leq \gamma_ {t} \left\langle g \left(x _ {t}\right), x _ {t} - \tilde {x} \right\rangle \leq E _ {t} - E _ {t + 1} + \gamma_ {t} \phi_ {t} + \gamma_ {t} ^ {2} \psi_ {t}. \right. \tag {C.3}
$$
Summing up the above terms in both sides of the inequality, we also get that
$$
\begin{array}{l} \sum_ {s = 1} ^ {t} \gamma_ {s} \left\langle g (\tilde {x}), x _ {s} - \tilde {x} \right\rangle \leq \sum_ {s = 1} ^ {t} \left[ E _ {s} - E _ {s + 1} + \gamma_ {s} \phi_ {s} + \gamma_ {s} ^ {2} \psi_ {s} \right] \\ = E _ {1} - E _ {t + 1} + \sum_ {s = 1} ^ {t} \gamma_ {s} \phi_ {s} + \sum_ {s = 1} ^ {t} \gamma_ {s} ^ {2} \psi_ {s} \tag {C.4} \\ \leq E _ {1} + \sum_ {s = 1} ^ {t} \gamma_ {s} \phi_ {s} + \sum_ {s = 1} ^ {t} \gamma_ {s} ^ {2} \psi_ {s}. \\ \end{array}
$$
Dividing all terms by  $\tilde{\gamma}_t\coloneqq \sum_{s = 1}^t\gamma_s$  , also yields