SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\mathbb {E} \left[ \operatorname {t r} \left[ \nabla_ {\mathbf {X} _ {j} ^ {\top}} u _ {i} \left(\mathbf {X} _ {t}\right) ^ {\dagger} \boldsymbol {\Xi} _ {j, t} \right] \mathbf {Z} _ {i, t} \mid \mathcal {F} _ {t} \right] = 0 \tag {C.36}
$$
Therefore, we obtain:
$$
\begin{array}{l} \mathbb {E} \left[ \sum_ {j \in \mathcal {N}} \operatorname {t r} \left[ \nabla_ {\mathbf {X} _ {j} ^ {\top}} u _ {i} \left(\mathbf {X} _ {t}\right) ^ {\dagger} \boldsymbol {\Xi} _ {j, t} \right] \mathbf {Z} _ {i, t} \mid \mathcal {F} _ {t} \right] = \mathbb {E} \left[ \operatorname {t r} \left[ \mathbf {V} _ {i} \left(\mathbf {X} _ {t}\right) ^ {\dagger} \boldsymbol {\Xi} _ {i, t} \right] \mathbf {Z} _ {i, t} \mid \mathcal {F} _ {t} \right] \\ = \delta_ {t} \mathbb {E} \left[ \operatorname {t r} \left[ \mathbf {V} _ {i} \left(\mathbf {X} _ {t}\right) ^ {\dagger} \mathbf {Z} _ {i, t} \right] \mathbf {Z} _ {i, t} \mid \mathcal {F} _ {t} \right] \\ = \frac {\delta_ {t}}{D _ {i}} \sum_ {W \in \mathcal {E} _ {i}} \operatorname {t r} \left[ \mathbf {V} _ {i} \left(\mathbf {X} _ {t}\right) ^ {\dagger} W \right] W \\ = \frac {\delta_ {t}}{D _ {i}} \operatorname {p r o j} _ {\mathcal {E} _ {i}} \left(\mathbf {V} _ {i} \left(\mathbf {X} _ {t}\right)\right) = \frac {\delta_ {t}}{D _ {i}} \mathbf {V} _ {i} \left(\mathbf {X} _ {t}\right) \tag {C.37} \\ \end{array}
$$
where in the last step, we identify  $\mathrm{proj}_{\mathcal{E}_i}(\mathbf{V}_i(\mathbf{X}_t))$  with  $\mathbf{V}_i(\mathbf{X}_t)$ , as explained in the proof of Proposition 3. Moreover, we have that
$$
\left| R _ {2} \left(\Xi_ {t}\right) \right| \leq \frac {L}{2} \| \Xi_ {t} \| _ {F} ^ {2} \leq L \delta_ {t} ^ {2} \tag {C.38}
$$
In view of the above, we have:
$$
\left\| \mathbb {E} \left[ \hat {\mathbf {V}} _ {i, t} \mid \mathcal {F} _ {t} \right] - \mathbf {V} _ {i} \left(\mathbf {X} _ {t}\right) \right\| _ {F} = D _ {i} L \delta_ {t} \tag {C.39}
$$
$$
\left\| \mathbb {E} \left[ \hat {\mathbf {V}} _ {t} \mid \mathcal {F} _ {t} \right] - \mathbf {V} \left(\mathbf {X} _ {t}\right) \right\| _ {F} = \sqrt {2} D L \delta_ {t} \tag {C.40}
$$
$$
\left\| \hat {\mathbf {V}} _ {i, t} \right\| _ {F} = \frac {D _ {i}}{\delta_ {t}} \left| u _ {i} \left(\mathbf {X} _ {t} ^ {(\delta)} + \delta_ {t} \mathbf {Z} _ {t}\right) \right| \| \mathbf {Z} _ {i, t} \| _ {F} \leq \frac {D _ {i} B}{\delta_ {t}} \tag {C.41}
$$
$$
\mathbb {E} \left[ \| \hat {\mathbf {V}} _ {i, t} \| _ {F} ^ {2} \mid \mathcal {F} _ {t} \right] \leq \frac {D _ {i} ^ {2} B ^ {2}}{\delta_ {t} ^ {2}} \tag {C.42}
$$
We thus obtain
$$
\mathbb {E} \left[ \| \hat {\mathbf {V}} _ {t} \| _ {F} ^ {2} \mid \mathcal {F} _ {t} \right] \leq \frac {2 D ^ {2} B ^ {2}}{\delta_ {t} ^ {2}} \tag {C.43}
$$
The only step missing is the proof of the actual guarantee of (3MW) with bandit feedback. We restate and prove the relevant result below:
Proof. Following the same procedure as in the proof of Theorem 2, we readily obtain:
$$
\begin{array}{l} \operatorname {G a p} _ {\mathcal {L}} \left(\bar {\mathbf {X}} _ {T}\right) \leq \frac {1}{T} \sum_ {t = 1} ^ {T} \operatorname {t r} \left[ \mathbf {V} \left(\mathbf {X} _ {t}\right) ^ {\dagger} \left(\mathbf {X} ^ {*} - \mathbf {X} _ {t}\right) \right] \\ \leq \frac {1}{T} \sum_ {t = 1} ^ {T} F _ {t} \left(\frac {1}{\gamma_ {t}} - \frac {1}{\gamma_ {t - 1}}\right) + \frac {1}{2 T} \sum_ {t = 1} ^ {T} \gamma_ {t} \| \hat {\mathbf {V}} _ {t} \| _ {F} ^ {2} \\ + \frac {4}{T} \sum_ {t = 1} ^ {T} \| b _ {t} \| _ {F} + \frac {1}{T} \sum_ {t = 1} ^ {T} \operatorname {t r} \left[ U _ {t} ^ {\dagger} \left(\mathbf {X} _ {t} - \mathbf {X} ^ {*}\right) \right] \tag {C.44} \\ \end{array}
$$
Now, we have:
$$
\mathbb {E} \left[ \operatorname {t r} \left[ U _ {t} ^ {\dagger} \left(\mathbf {X} _ {t} - \mathbf {X} ^ {*}\right) \right] \right] = \mathbb {E} \left[ \mathbb {E} \left[ \operatorname {t r} \left[ U _ {t} ^ {\dagger} \left(\mathbf {X} _ {t} - \mathbf {X} ^ {*}\right) \right] \mid \mathcal {F} _ {t} \right] \right] = 0 \tag {C.45}
$$
Moreover, by Proposition 4, we have:
$$
\left\| b _ {i, t} \right\| _ {F} = \left\| \mathbb {E} \left[ \hat {\mathbf {V}} _ {i, t} \mid \mathcal {F} _ {t} \right] - \mathbf {V} _ {i} \left(\mathbf {X} _ {t}\right) \right\| _ {F} \leq 2 D _ {i} L \delta_ {t} \tag {C.46}
$$
$$
\mathbb {E} \left[ \left\| \hat {\mathbf {V}} _ {i, t} \right\| _ {F} ^ {2} \right] = \mathbb {E} \left[ \mathbb {E} \left[ \left\| \hat {\mathbf {V}} _ {i, t} \right\| _ {F} ^ {2} \mid \mathcal {F} _ {t} \right] \right] \leq \frac {D _ {t} ^ {2} B ^ {2}}{\delta_ {t} ^ {2}} \tag {C.47}
$$
Hence, taking expectation in (D.34), we obtain:
$$
\begin{array}{l} \mathbb {E} \left[ \operatorname {G a p} _ {\mathcal {L}} \left(\bar {\mathbf {X}} _ {T}\right) \right] \leq \frac {1}{T} \sum_ {t = 1} ^ {T} \mathbb {E} \left[ F _ {t} \right] \left(\frac {1}{\gamma_ {t}} - \frac {1}{\gamma_ {t - 1}}\right) + \frac {1}{2 T} \sum_ {t = 1} ^ {T} \gamma_ {t} \mathbb {E} \left[ \| \hat {\mathbf {V}} _ {t} \| _ {F} ^ {2} \right] + \frac {4}{T} \sum_ {t = 1} ^ {T} \mathbb {E} \left[ \| b _ {t} \| _ {F} \right] \\ \leq \frac {1}{T} \sum_ {t = 1} ^ {T} \mathbb {E} \left[ F _ {t} \right] \left(\frac {1}{\gamma_ {t}} - \frac {1}{\gamma_ {t - 1}}\right) + \frac {2 D ^ {2} B ^ {2}}{T} \sum_ {t = 1} ^ {T} \frac {\gamma_ {t}}{\delta_ {t} ^ {2}} + \frac {1 6 D L}{T} \sum_ {t = 1} ^ {T} \delta_ {t} \tag {C.48} \\ \end{array}
$$
$$
\mathbb {E} \left[ \operatorname {G a p} _ {\mathcal {L}} \left(\bar {\mathbf {X}} _ {T}\right) \right] \leq \frac {\mathbb {E} \left[ F _ {1} \right]}{\gamma T} + \frac {2 D ^ {2} B ^ {2} \gamma}{\delta^ {2}} + 1 6 D L \delta \tag {C.49}
$$
$$
\mathbb {E} \left[ F _ {1} \right] = D \left(\mathbf {X} ^ {*}, \mathbf {X} _ {1}\right) \leq \log \left(d _ {1} d _ {2}\right) \tag {C.50}
$$
$$
\mathbb {E} \left[ \operatorname {G a p} _ {\mathcal {L}} \left(\bar {\mathbf {X}} _ {T}\right) \right] \leq \frac {H}{\gamma T} + \frac {2 D ^ {2} B ^ {2} \gamma}{\delta^ {2}} + 1 6 D L \delta \tag {C.51}
$$
where  $H = \log(d_1 d_2)$ . Hence, after tuning  $\gamma$  and  $\delta$  to optimize this last expression, our result follows by setting  $\gamma = \left(\frac{H}{2T}\right)^{3/4} \frac{1}{2D\sqrt{BL}}$  and  $\delta = \left(\frac{H}{2T}\right)^{1/4} \sqrt{\frac{B}{4L}}$ .
D Omitted proofs from Section 6
We provide first the bounds of the estimator (1PE) in a  $N$ -player quantum game. Formally, we have: Lemma D.1. The estimator (1PE) in a  $N$ -player quantum game  $\mathcal{Q}$  enjoys the conditional bounds
$$
(i) \quad \| \mathbb {E} [ \hat {\mathbf {V}} _ {t} \mid \mathcal {F} _ {t} ] - \mathbf {V} (\mathbf {X} _ {t}) \| _ {F} \leq \frac {1}{2} D L N ^ {3 / 2} \delta_ {t} \quad a n d \quad (i i) \quad \mathbb {E} [ \| \hat {\mathbf {V}} _ {t} \| _ {F} ^ {2} \mid \mathcal {F} _ {t} ] \leq \frac {D ^ {2} B ^ {2} N}{\delta_ {t} ^ {2}}. \tag {D.1}
$$
$$
\Xi_ {i, t} = \mathbf {X} _ {i, t} ^ {(\delta)} - \mathbf {X} _ {i, t} = \delta_ {t} \mathbf {Z} _ {i, t} + \frac {\delta_ {t}}{r _ {i}} (\mathbf {R} _ {i} - \mathbf {X} _ {i, t}) = \delta_ {t} \left[ \mathbf {Z} _ {i, t} + \frac {1}{r _ {i}} (\mathbf {R} _ {i} - \mathbf {X} _ {i, t}) \right] \qquad (D. 2)
$$
Taking a 1st-order Taylor expansion of  $u_{i}$ , we obtain:
$$
u _ {i} \left(\mathbf {X} _ {t} ^ {(\delta)} + \delta_ {t} \mathbf {Z} _ {t}\right) = u _ {i} \left(\mathbf {X} _ {t}\right) + \sum_ {j \in \mathcal {N}} \operatorname {t r} \left[ \nabla_ {\mathbf {X} _ {j} ^ {\top}} u _ {i} \left(\mathbf {X} _ {t}\right) ^ {\dagger} \boldsymbol {\Xi} _ {j, t} \right] + R _ {2} (\boldsymbol {\Xi} _ {t}) \tag {D.3}
$$
$$
\begin{array}{l} \mathbb {E} \left[ \hat {\mathbf {V}} _ {i, t} \mid \mathcal {F} _ {t}, \mathbf {Z} _ {t} \right] = \frac {D _ {i}}{\delta_ {t}} u _ {i} \left(\mathbf {X} _ {t} ^ {(\delta)} + \delta_ {t} \mathbf {Z} _ {t}\right) \mathbf {Z} _ {i, t} \\ = \frac {D _ {i}}{\delta_ {t}} u _ {i} \left(\mathbf {X} _ {t}\right) \mathbf {Z} _ {i, t} + \frac {D _ {i}}{\delta_ {t}} \sum_ {j \in \mathcal {N}} \operatorname {t r} \left[ \nabla_ {\mathbf {X} _ {j} ^ {\top}} u _ {i} \left(\mathbf {X} _ {t}\right) ^ {\dagger} \boldsymbol {\Xi} _ {j, t} \right] \mathbf {Z} _ {i, t} + \frac {D _ {i}}{\delta_ {t}} R _ {2} \left(\boldsymbol {\Xi} _ {t}\right) \mathbf {Z} _ {i, t} \tag {D.4} \\ \end{array}
$$
$$
\mathbb {E} \left[ u _ {i} \left(\mathbf {X} _ {t}\right) \mathbf {Z} _ {i, t} \mid \mathcal {F} _ {t} \right] = u _ {i} \left(\mathbf {X} _ {t}\right) \mathbb {E} \left[ \mathbf {Z} _ {i, t} \mid \mathcal {F} _ {t} \right] = 0 \tag {D.5}
$$
$$
\mathbb {E} \left[ \operatorname {t r} \left[ \nabla_ {\mathbf {X} _ {j} ^ {\top}} u _ {i} \left(\mathbf {X} _ {t}\right) ^ {\dagger} \boldsymbol {\Xi} _ {j, t} \right] \mathbf {Z} _ {i, t} \mid \mathcal {F} _ {t} \right] = 0 \tag {D.6}
$$
$$
\begin{array}{l} \mathbb {E} \left[ \sum_ {j \in \mathcal {N}} \operatorname {t r} \left[ \nabla_ {\mathbf {X} _ {j} ^ {\top}} u _ {i} \left(\mathbf {X} _ {t}\right) ^ {\dagger} \boldsymbol {\Xi} _ {j, t} \right] \mathbf {Z} _ {i, t} \mid \mathcal {F} _ {t} \right] = \mathbb {E} \left[ \operatorname {t r} \left[ \mathbf {V} _ {i} \left(\mathbf {X} _ {t}\right) ^ {\dagger} \boldsymbol {\Xi} _ {i, t} \right] \mathbf {Z} _ {i, t} \mid \mathcal {F} _ {t} \right] \\ = \delta_ {t} \mathbb {E} \left[ \operatorname {t r} \left[ \mathbf {V} _ {i} \left(\mathbf {X} _ {t}\right) ^ {\dagger} \mathbf {Z} _ {i, t} \right] \mathbf {Z} _ {i, t} \mid \mathcal {F} _ {t} \right] \\ = \frac {\delta_ {t}}{D _ {i}} \sum_ {W \in \mathcal {E} _ {i}} \operatorname {t r} \left[ \mathbf {V} _ {i} \left(\mathbf {X} _ {t}\right) ^ {\dagger} W \right] W \\ = \frac {\delta_ {t}}{D _ {i}} \operatorname {p r o j} _ {\mathcal {E} _ {i}} \left(\mathbf {V} _ {i} \left(\mathbf {X} _ {t}\right)\right) = \frac {\delta_ {t}}{D _ {i}} \mathbf {V} _ {i} \left(\mathbf {X} _ {t}\right) \tag {D.7} \\ \end{array}
$$
$$
\left| R _ {2} \left(\Xi_ {t}\right) \right| \leq \frac {L}{2} \| \Xi_ {t} \| _ {F} ^ {2} \leq \frac {1}{2} L N \delta_ {t} ^ {2} \tag {D.8}
$$
$$
\left\| \mathbb {E} \left[ \hat {\mathbf {V}} _ {i, t} \mid \mathcal {F} _ {t} \right] - \mathbf {V} _ {i} \left(\mathbf {X} _ {t}\right) \right\| _ {F} \leq \frac {1}{2} D _ {i} L N \delta_ {t} \tag {D.9}
$$
$$
\left\| \mathbb {E} \left[ \hat {\mathbf {V}} _ {t} \mid \mathcal {F} _ {t} \right] - \mathbf {V} \left(\mathbf {X} _ {t}\right) \right\| _ {F} \leq \frac {1}{2} D L N ^ {3 / 2} \delta_ {t} \tag {D.10}
$$
$$
\left\| \hat {\mathbf {V}} _ {i, t} \right\| _ {F} = \frac {D _ {i}}{\delta_ {t}} \left| u _ {i} \left(\mathbf {X} _ {t} ^ {(\delta)} + \delta_ {t} \mathbf {Z} _ {t}\right) \right| \| \mathbf {Z} _ {i, t} \| _ {F} \leq \frac {D _ {i} B}{\delta_ {t}} \tag {D.11}
$$
$$
\mathbb {E} \left[ \| \hat {\mathbf {V}} _ {i, t} \| _ {F} ^ {2} \mid \mathcal {F} _ {t} \right] \leq \frac {D _ {i} ^ {2} B ^ {2}}{\delta_ {t} ^ {2}} \tag {D.12}
$$
Hence, ultimately, we get the bound
$$
\mathbb {E} \left[ \| \hat {\mathbf {V}} _ {t} \| _ {F} ^ {2} \mid \mathcal {F} _ {t} \right] \leq \frac {D ^ {2} B ^ {2} N}{\delta_ {t} ^ {2}} \tag {D.13}
$$
With all this in hand, we are finally in a position to proceed with the proof of Theorem 4, which we restate below for convenience:
Proof. Since  $\mathbf{X}^*$  is variationally stable, there exists a neighborhood  $\mathcal{U}_{\mathrm{vs}}$  of it such that
$$
\operatorname {t r} \left[ \mathbf {V} (\mathbf {X}) \left(\mathbf {X} - \mathbf {X} ^ {*}\right) \right] <   0 \quad \text {f o r a l l} \mathbf {X} \in \mathcal {U} _ {\mathrm {v s}} \backslash \left\{\mathbf {X} ^ {*} \right\}. \tag {D.14}
$$
For any  $\varepsilon' > 0$ , defining
$$
\mathcal {U} _ {\varepsilon} ^ {\prime} := \left\{\mathbf {X} \in \mathcal {X}: D \left(\mathbf {X} ^ {*}, \mathbf {X}\right) <   \varepsilon^ {\prime} \right\} \tag {D.15}
$$
we readily obtain by the continuity of  $\mathbf{X} \mapsto D(\mathbf{X}^*, \mathbf{X})$  at  $\mathbf{X}^*$  that there exists a neighborhood  $\mathcal{U}_{\varepsilon}$  of  $\mathbf{X}^*$  such that  $\mathcal{U}_{\varepsilon} \subseteq \mathcal{U}_{\mathrm{vs}}$ . Note that if  $\varepsilon_1 < \varepsilon_2$ , we automatically get that  $\mathcal{U}_{\varepsilon_1} \subseteq \mathcal{U}_{\varepsilon_2}$ .
In view of this, we let  $\mathbf{X}_1\in \mathcal{U}_{\varepsilon /4}\subseteq \mathcal{U}_\varepsilon \subseteq \mathcal{U}_{\mathrm{vs}}$ . We divide the rest of the proof in steps.
Step 1. Deriving the general energy inequality
By Lemma 1 we have that:
$$
D \left(\mathbf {X} ^ {*}, \mathbf {X} _ {t + 1}\right) \leq D \left(\mathbf {X} ^ {*}, \mathbf {X} _ {t}\right) + \gamma_ {t} \operatorname {t r} \left[ \hat {\mathbf {V}} _ {t} \left(\mathbf {X} _ {t} - \mathbf {X} ^ {*}\right) \right] + \frac {\gamma_ {t} ^ {2}}{2} \| \hat {\mathbf {V}} _ {t} \| _ {F} ^ {2}. \tag {D.16}
$$
Decomposing  $\hat{\mathbf{V}}_t$  into
$$
\hat {\mathbf {V}} _ {t} = \mathbf {V} \left(\mathbf {X} _ {t}\right) + b _ {t} + U _ {t} \tag {D.17}
$$
as per (C.20) and applying (D.16) inequality iteratively, we get that
$$
\begin{array}{l} D \left(\mathbf {X} ^ {*}, \mathbf {X} _ {t + 1}\right) \leq D \left(\mathbf {X} ^ {*}, \mathbf {X} _ {1}\right) + \sum_ {s = 1} ^ {t} \gamma_ {s} \operatorname {t r} \left[ \hat {\mathbf {V}} _ {s} \left(\mathbf {X} _ {s} - \mathbf {X} ^ {*}\right) \right] + \frac {1}{2} \sum_ {s = 1} ^ {t} \gamma_ {s} ^ {2} \| \hat {\mathbf {V}} _ {s} \| _ {F} ^ {2} \\ \leq D \left(\mathbf {X} ^ {*}, \mathbf {X} _ {1}\right) + \sum_ {s = 1} ^ {t} \gamma_ {s} \operatorname {t r} \left[ \mathbf {V} \left(\mathbf {X} _ {s}\right) \left(\mathbf {X} _ {s} - \mathbf {X} ^ {*}\right) \right] + \sum_ {s = 1} ^ {t} \gamma_ {s} \operatorname {t r} \left[ b _ {s} \left(\mathbf {X} _ {s} - \mathbf {X} ^ {*}\right) \right] \\ + \sum_ {s = 1} ^ {t} \gamma_ {s} \operatorname {t r} \left[ U _ {s} \left(\mathbf {X} _ {s} - \mathbf {X} ^ {*}\right) \right] + \frac {1}{2} \sum_ {s = 1} ^ {t} \gamma_ {s} ^ {2} \| \hat {\mathbf {V}} _ {s} \| _ {F} ^ {2} \\ \leq D \left(\mathbf {X} ^ {*}, \mathbf {X} _ {1}\right) + \sum_ {s = 1} ^ {t} \gamma_ {s} \operatorname {t r} \left[ \mathbf {V} \left(\mathbf {X} _ {s}\right) \left(\mathbf {X} _ {s} - \mathbf {X} ^ {*}\right) \right] + \sum_ {s = 1} ^ {t} \gamma_ {s} \| b _ {s} \| _ {F} \| \mathbf {X} _ {s} - \mathbf {X} ^ {*} \| _ {F} \\ + \sum_ {s = 1} ^ {t} \gamma_ {s} \operatorname {t r} \left[ U _ {s} \left(\mathbf {X} _ {s} - \mathbf {X} ^ {*}\right) \right] + \frac {1}{2} \sum_ {s = 1} ^ {t} \gamma_ {s} ^ {2} \| \hat {\mathbf {V}} _ {s} \| _ {F} ^ {2} \\ \leq D \left(\mathbf {X} ^ {*}, \mathbf {X} _ {1}\right) + \sum_ {s = 1} ^ {t} \gamma_ {s} \operatorname {t r} \left[ \mathbf {V} \left(\mathbf {X} _ {s}\right) \left(\mathbf {X} _ {s} - \mathbf {X} ^ {*}\right) \right] + \operatorname {d i a m} (\mathcal {X}) \sum_ {s = 1} ^ {t} \gamma_ {s} \| b _ {s} \| _ {F} \\ + \sum_ {s = 1} ^ {t} \gamma_ {s} \operatorname {t r} \left[ U _ {s} \left(\mathbf {X} _ {s} - \mathbf {X} ^ {*}\right) \right] + \frac {1}{2} \sum_ {s = 1} ^ {t} \gamma_ {s} ^ {2} \| \hat {\mathbf {V}} _ {s} \| _ {F} ^ {2} \tag {D.18} \\ \end{array}
$$
Defining the processes  $\Psi_t, M_t$  and  $Z_t$  for  $t = 1, 2, \ldots$  as
$$
\Psi_ {t} := \frac {1}{2} \sum_ {s = 1} ^ {t} \gamma_ {s} ^ {2} \| \hat {\mathbf {V}} _ {s} \| _ {F} ^ {2} \tag {D.19a}
$$
$$
M _ {t} := \sum_ {s = 1} ^ {t} \gamma_ {s} \operatorname {t r} \left[ U _ {s} \left(\mathbf {X} _ {s} - \mathbf {X} ^ {*}\right) \right] \tag {D.19b}
$$
$$
Z _ {t} := \operatorname {d i a m} (\mathcal {X}) \sum_ {s = 1} ^ {t} \gamma_ {s} \| b _ {s} \| _ {F} \tag {D.19c}
$$
equation (D.18) can be rewritten as
$$
D \left(\mathbf {X} ^ {*}, \mathbf {X} _ {t + 1}\right) \leq D \left(\mathbf {X} ^ {*}, \mathbf {X} _ {1}\right) + \sum_ {s = 1} ^ {t} \gamma_ {s} \operatorname {t r} \left[ \mathbf {V} \left(\mathbf {X} _ {s}\right) \left(\mathbf {X} _ {s} - \mathbf {X} ^ {*}\right) \right] + Z _ {t} + M _ {t} + \Psi_ {t} \tag {D.20}
$$
Step 2. Bounding the noise terms
Let  $\varepsilon > 0$  as defined in the beginning of the proof.
- Regarding the term  $Z_{t}$ , it is clear that the process  $\{Z_{t}: t \geq 1\}$  is a sub-martingale. Hence, by Doob's maximal inequality for sub-martingales [24], we get that:
$$
\begin{array}{l} \mathbb {P} \left(\sup  _ {s \leq t} Z _ {s} \geq \varepsilon / 4\right) \leq \frac {\mathrm {E} \left[ Z _ {t} \right]}{\varepsilon / 4} \\ \leq \frac {\operatorname {d i a m} (\boldsymbol {\mathcal {X}}) \sum_ {s = 1} ^ {t} \gamma_ {s} \mathbb {E} [ \| b _ {s} \| _ {F} ]}{\varepsilon / 4} \\ \leq \frac {\mathrm {d i a m} (\pmb {\mathscr {X}}) \sum_ {t = 1} ^ {\infty} \gamma_ {t} \mathbb {E} [ \| b _ {t} \| _ {F} ]}{\varepsilon / 4} \\ = \frac {\mathrm {d i a m} (\pmb {\mathscr {X}}) \sum_ {t = 1} ^ {\infty} \gamma_ {t} \mathbb {E} [ \mathbb {E} [ \| b _ {t} \| _ {F} \mid \mathcal {F} _ {t} ] ]}{\varepsilon / 4} \\ \leq \frac {2 \operatorname {d i a m} (\mathcal {X}) D L N ^ {3 / 2} \sum_ {t = 1} ^ {\infty} \gamma_ {t} \delta_ {t}}{\varepsilon} \tag {D.21} \\ \end{array}
$$
By ensuring that
$$
\sum_ {t = 1} ^ {\infty} \gamma_ {t} \delta_ {t} \leq \frac {\varepsilon \eta}{6 \operatorname {d i a m} (\mathcal {X}) D L N ^ {3 / 2}} \tag {D.22}
$$
and taking  $t$  go to  $\infty$ , (D.21) becomes:
$$
\mathbb {P} \left(\sup  _ {t \geq 1} Z _ {t} \geq \varepsilon / 4\right) \leq \eta / 3 \tag {D.23}
$$
- Similarly, it is clear that the process  $\{\Psi_t : t \geq 1\}$  is a sub-martingale. Following the same procedure, by Doob's maximal inequality for sub-martingales [24], we get that:
$$
\begin{array}{l} \mathbb {P} \left(\sup  _ {s \leq t} \Psi_ {s} \geq \varepsilon / 4\right) \leq \frac {\mathbb {E} \left[ \Psi_ {t} \right]}{\varepsilon / 4} \leq \frac {\frac {1}{2} \sum_ {s = 1} ^ {t} \gamma_ {s} ^ {2} \mathbb {E} \left[ \| \hat {\mathbf {V}} _ {s} \| _ {F} ^ {2} \right]}{\varepsilon / 4} \\ \leq \frac {\frac {1}{2} \sum_ {t = 1} ^ {\infty} \gamma_ {t} ^ {2} \mathbb {E} \left[ \left\| \hat {\mathbf {V}} _ {t} \right\| _ {F} ^ {2} \right]}{\varepsilon / 4} \\ \leq \frac {2 D ^ {2} B ^ {2} N \sum_ {t = 1} ^ {\infty} \gamma_ {t} ^ {2} / \delta_ {t} ^ {2}}{\varepsilon} \tag {D.24} \\ \end{array}
$$
$$
\sum_ {t = 1} ^ {\infty} \gamma_ {t} ^ {2} / \delta_ {t} ^ {2} \leq \frac {\varepsilon \eta}{6 D ^ {2} B ^ {2} N} \tag {D.25}
$$
and taking  $t\to \infty$  , (D.24) becomes:
$$
\mathbb {P} \left(\sup  _ {t \geq 1} \Psi_ {t} \geq \varepsilon / 4\right) \leq \eta / 3 \tag {D.26}
$$
- Finally, regarding the term  $M_t$ , the process  $\{M_t : t \geq 1\}$  is a martingale. Following the same procedure, by Doob's maximal inequality for martingales [24], we get that:
$$
\begin{array}{l} \mathbb {P} \left(\sup  _ {s \leq t} M _ {s} \geq \varepsilon / 4\right) \leq \mathbb {P} \left(\sup  _ {s \leq t} | M _ {s} | \geq \varepsilon / 4\right) \leq \frac {\mathbb {E} \left[ M _ {t} ^ {2} \right]}{(\varepsilon / 4) ^ {2}} = \frac {\sum_ {s = 1} ^ {t} \gamma_ {s} ^ {2} \mathbb {E} \left[ \operatorname {t r} \left[ U _ {s} \left(\mathbf {X} _ {s} - \mathbf {X} ^ {*}\right) \right] ^ {2} \right]}{(\varepsilon / 4) ^ {2}} \\ \leq \frac {\operatorname {d i a m} (\boldsymbol {\mathcal {X}}) ^ {2} \sum_ {s = 1} ^ {t} \gamma_ {s} ^ {2} \mathbb {E} \left[ \| U _ {s} \| _ {F} ^ {2} \right]}{(\varepsilon / 4) ^ {2}} \\ \leq \frac {\mathrm {d i a m} (\boldsymbol {\mathcal {X}}) ^ {2} \sum_ {t = 1} ^ {\infty} \gamma_ {t} ^ {2} \mathbb {E} \left[ \| U _ {t} \| _ {F} ^ {2} \right]}{(\varepsilon / 4) ^ {2}} \\ \leq \frac {4 \operatorname {d i a m} (\boldsymbol {\mathcal {X}}) ^ {2} \sum_ {t = 1} ^ {\infty} \gamma_ {t} ^ {2} \mathbb {E} \left[ \left\| \hat {\mathbf {V}} _ {t} \right\| _ {F} ^ {2} \right]}{(\varepsilon / 4) ^ {2}} \\ \leq \frac {4 \operatorname {d i a m} (\mathcal {X}) ^ {2} D ^ {2} B ^ {2} N \sum_ {t = 1} ^ {\infty} \gamma_ {t} ^ {2} / \delta_ {t} ^ {2}}{(\varepsilon / 4) ^ {2}} \tag {D.27} \\ \end{array}
$$
$$
\begin{array}{l} \mathbb {E} \left[ M _ {t} ^ {2} \right] = \mathbb {E} \left[ \sum_ {s = 1} ^ {t} \gamma_ {s} ^ {2} \operatorname {t r} \left[ U _ {s} \left(\mathbf {X} _ {s} - \mathbf {X} ^ {*}\right) \right] ^ {2} + \sum_ {k <   \ell} \gamma_ {k} \gamma_ {\ell} \operatorname {t r} \left[ U _ {k} \left(\mathbf {X} _ {k} - \mathbf {X} ^ {*}\right) \right] \operatorname {t r} \left[ U _ {\ell} \left(\mathbf {X} _ {\ell} - \mathbf {X} ^ {*}\right) \right] \right] \\ = \mathbb {E} \left[ \sum_ {s = 1} ^ {t} \gamma_ {s} ^ {2} \operatorname {t r} \left[ U _ {s} \left(\mathbf {X} _ {s} - \mathbf {X} ^ {*}\right) \right] ^ {2} \right] \tag {D.28} \\ \end{array}
$$
itself following from the total expectation
$$
\begin{array}{l} \mathbb {E} \left[ \operatorname {t r} \left[ U _ {k} \left(\mathbf {X} _ {k} - \mathbf {X} ^ {*}\right) \right] \operatorname {t r} \left[ U _ {\ell} \left(\mathbf {X} _ {\ell} - \mathbf {X} ^ {*}\right) \right] \right] = \mathbb {E} \left[ \mathbb {E} \left[ \operatorname {t r} \left[ U _ {k} \left(\mathbf {X} _ {k} - \mathbf {X} ^ {*}\right) \right] \operatorname {t r} \left[ U _ {\ell} \left(\mathbf {X} _ {\ell} - \mathbf {X} ^ {*}\right) \right] \mid \mathcal {F} _ {\ell} \right] \right] \\ = \mathbb {E} \left[ \operatorname {t r} \left[ U _ {k} \left(\mathbf {X} _ {k} - \mathbf {X} ^ {*}\right) \right] \mathbb {E} \left[ \operatorname {t r} \left[ U _ {\ell} \left(\mathbf {X} _ {\ell} - \mathbf {X} ^ {*}\right) \right] \mid \mathcal {F} _ {\ell} \right] \right] \\ = 0 \tag {D.29} \\ \end{array}
$$
Now, by ensuring that
$$
\sum_ {t = 1} ^ {\infty} \gamma_ {t} ^ {2} / \delta_ {t} ^ {2} \leq \frac {(\varepsilon / 4) ^ {2} \eta}{1 2 \operatorname {d i a m} (\boldsymbol {\chi}) ^ {2} D ^ {2} B ^ {2} N} \tag {D.30}
$$
and taking  $t$  go to  $\infty$ , (D.27) becomes:
$$
\mathbb {P} \left(\sup  _ {t \geq 1} M _ {t} \geq \varepsilon / 4\right) \leq \eta / 3 \tag {D.31}
$$
Therefore, combining (D.23), (D.26) and (D.31) and applying a union bound, we get:
$$
\mathbb {P} \left(\left\{\sup  _ {t \geq 1} Z _ {t} \geq \varepsilon / 4 \right\} \cup \left\{\sup  _ {t \geq 1} \Psi_ {t} \geq \varepsilon / 4 \right\} \cup \left\{\sup  _ {t \geq 1} M _ {t} \geq \varepsilon / 4 \right\}\right) \leq \eta \tag {D.32}
$$
Thus, defining the event  $E \coloneqq \left\{\sup_{t \geq 1} Z_t + \Psi_t + M_t < \frac{3}{4}\varepsilon \right\}$ , Eq. (D.32) readily implies that:
$$
\mathbb {P} (E) \geq 1 - \eta \tag {D.33}
$$
Step 3.  $\mathbf{X}_t\in \mathcal{U}_{\mathrm{vs}}$  with high probability
Since  $\mathbf{X}_1\in \mathcal{U}_{\varepsilon /4}\subseteq \mathcal{U}_{\mathrm{vs}}$  , by induction on  $t$  we have that under the event  $E$
$$
\begin{array}{l} D \left(\mathbf {X} ^ {*}, \mathbf {X} _ {t + 1}\right) \leq D \left(\mathbf {X} ^ {*}, \mathbf {X} _ {1}\right) + \sum_ {s = 1} ^ {t} \gamma_ {s} \operatorname {t r} \left[ \mathbf {V} \left(\mathbf {X} _ {s}\right) \left(\mathbf {X} _ {s} - \mathbf {X} ^ {*}\right) \right] + Z _ {t} + M _ {t} + \Psi_ {t} (D.34) \\ \leq \frac {\varepsilon}{4} + \frac {\varepsilon}{4} + \frac {\varepsilon}{4} + \frac {\varepsilon}{4} = \varepsilon (D.35) \\ \end{array}
$$
where in the last step we used the inductive hypothesis that  $\mathbf{X}_s\in \mathcal{U}_{\mathrm{vs}}$  for all  $s = 1,\ldots ,t$  which implies  $\operatorname {tr}[\mathbf{V}(\mathbf{X}_s)(\mathbf{X}_s - \mathbf{X}^*)] <   0$  . This implies that  $\mathbf{X}_{t + 1}\in \mathcal{U}_{\varepsilon}\subseteq \mathcal{U}_{\mathrm{vs}}$
Therefore, we obtain that  $\mathbf{X}_{t + 1}\in \mathcal{U}_{\varepsilon}\subseteq \mathcal{U}_{\mathrm{vs}}$  for all  $t\geq 1$ . For the rest of the proof we will work under the event  $E$ .
Step 4. Subsequential convergence
Now we will show that there exists a subsequence  $\{\mathbf{X}_{t_k}:k\geq 1\}$  such that  $\lim_{k\to \infty}\mathbf{X}_{t_k} = \mathbf{X}^*$ . Suppose it does not. Then, this would mean that the quantity  $\mathrm{tr}[\mathbf{V}(\mathbf{X}_t)(\mathbf{X}_t - \mathbf{X}^*)]$  is bounded away from zero. Combining it with the fact that  $\mathbf{X}_t\in \mathcal{U}_{\mathrm{vs}}$  for all  $t\geq 0$ , we readily get that there exists  $c > 0$  such that:
$$
\operatorname {t r} \left[ \mathbf {V} \left(\mathbf {X} _ {s}\right) \left(\mathbf {X} _ {s} - \mathbf {X} ^ {*}\right) \right] <   - c \tag {D.36}
$$
Then, (D.20) would give:
$$
D \left(\mathbf {X} ^ {*}, \mathbf {X} _ {t + 1}\right) \leq \varepsilon - c \sum_ {s = 1} ^ {t} \gamma_ {s} \tag {D.37}
$$