SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Convergence speed analysis. In Fig. 1, we evaluate the convergence properties of (3MW) using the estimators (2PE) and (1PE), and compare it with the full information variant (MMW). For each method, we perform 10 different runs, with  $T = 10^{5}$  steps each, and compute the mean value of the duality gap as a function of the iteration  $t = 1,2,\dots,T$ . The solid lines correspond to the mean values of the duality gap of each method, and the shaded regions enclose the area of  $\pm 1$  (sample) standard deviation among the 10 different runs. Note that the red line, which corresponds to the full information (MMW), does not have a shaded region, since there is no randomness in the algorithm. All the runs for the three different methods were initialized for  $\mathbf{Y} = 0$  and we used  $\gamma = 10^{-2}$  for all methods. In particular, for (3MW) with gradient estimates given by (2PE) estimator, we used a sampling radius  $\delta = 10^{-2}$ , and for (3MW) with (1PE) estimator, we used  $\delta = 10^{-1}$  (in tune with our theoretical results which suggest the use of a tighter sampling radius when mixed payoff information is available to the players).
Figure 1 has several important take-aways. First and foremost, as is to be expected, the payoff-based methods lag behind the full-information variant of (MMW); however, what is particularly surprising is that the drop in performance is singularly mild. As we see in the second plot in Fig. 1, the various algorithms achieved a rate of convergence closer to  $\mathcal{O}(1 / T)$ , which is significantly faster than  $\mathcal{O}(1 / \sqrt{T})$  and/or  $\mathcal{O}(1 / T^{1 / 4})$ . This suggests that, in practice, the bandit variants of (MMW) may yield excellent performance benefits, despite the high degree of uncertainty incurred by the complete lack of information on the game being played.
PM is also with the Archimedes Research Unit – Athena RC – Department of Mathematics, National & Kapodistrian University of Athens. This work has been partially supported by the Air Force Office of Scientific Research under award number FA9550-20-1-0397. Additional support is gratefully acknowledged from NSF 1915967, 2118199, 2229012, 2312204. Furthermore, KL is grateful for the support by the Onassis Foundation (F ZR 033-1/2021-2022). PM was supported by project MIS 5154714 of the National Recovery and Resilience Plan Greece 2.0 funded by the European Union under the NextGenerationEU Program, and by the French National Research Agency (ANR) in the framework of the “Investissements d’avenir” program (ANR-15-IDEX-02), the LabEx PERSYVAL (ANR-11-LABX-0025-01), MIAI@Grenoble Alpes (ANR-19-P3IA-0003). NB was supported by the Koret Foundation via the Digital Living 2030 project.
[46] Mertikopoulos, P., Hsieh, Y.-P., and Cevher, V. A unified stochastic approximation framework for learning in games. Mathematical Programming, forthcoming, 2023.
[47] Nemirovski, A. S. and Yudin, D. B. Problem Complexity and Method Efficiency in Optimization. Wiley, New York, NY, 1983.
[48] Nielsen, M. A. and Chuang, I. L. Quantum Computation and Quantum Information: 10th Anniversary Edition. Cambridge University Press, 2010. doi: 10.1017/CBO9780511976667.
[49] Perkins, S. and Leslie, D. S. Stochastic fictitious play with continuous action sets. Journal of Economic Theory, 152:179-213, July 2014.
[50] Perkins, S., Mertikopoulos, P., and Leslie, D. S. Mixed-strategy learning with continuous action sets. IEEE Trans. Autom. Control, 62(1):379-384, January 2017.
[51] Preskill, J. Quantum computing in the NISQ era and beyond. Quantum, 2:79, August 2018.
[52] Robbins, H. and Siegmund, D. A convergence theorem for nonnegative almost supermartingales and some applications. In Rustagi, J. S. (ed.), Optimizing Methods in Statistics. Academic Press, New York, NY, 1971.
[53] Shamir, O. An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. Journal of Machine Learning Research, 18(52):1-11, 2017.
[54] Sion, M. On general minimax theorems. Pacific Journal of Mathematics, 8(1):171-176, 1958.
[55] Spall, J. C. Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Autom. Control, 37(3):332-341, March 1992.
[56] Taylor, P. D. Evolutionarily stable strategies with two types of player. Journal of Applied Probability, 16 (1):76-83, March 1979.
[57] Telatar, I. E. Capacity of multi-antenna Gaussian channels. European Transactions on Telecommunications and Related Technologies, 10(6):585–596, 1999.
[58] Tse, D. and Viswanath, P. Fundamentals of Wireless Communication. Cambridge University Press, Cambridge, UK, 2005.
[59] Tsuda, K., Ratsch, G., and Warmuth, M. K. Matrix exponentiated gradient updates for on-line Bregman projection. Journal of Machine Learning Research, 6:995-1018, 2005.
[60] van Apeldoorn, J. and Gily'en, A. Quantum algorithms for zero-sum games. arXiv: Quantum Physics, 2019.
[61] Vovk, V. G. Aggregating strategies. In COLT '90: Proceedings of the 3rd Workshop on Computational Learning Theory, pp. 371-383, 1990.
[62] Yu, W., Rhee, W., Boyd, S. P., and Cioffi, J. M. Iterative water-filling for Gaussian vector multiple-access channels. IEEE Trans. Inf. Theory, 50(1):145-152, 2004.
[63] Yu, Y.-L. The strong convexity of von Neumann's entropy. 2015.
[64] Zhong, H.-S., Wang, H., Deng, Y.-H., Chen, M.-C., Peng, L.-C., Luo, Y.-H., Qin, J., Wu, D., Ding, X., Hu, Y., Hu, P., Yang, X.-Y., Zhang, W.-J., Li, H., Li, Y., Jiang, X., Gan, L., Yang, G., You, L., Wang, Z., Li, L., Liu, N.-L., Lu, C.-Y., and Pan, J.-W. Quantum computational advantage using photons. Science, 370 (6523):1460-1463, December 2020.
[65] Zimmert, J. and Seldin, Y. Tsallis-INF: An optimal algorithm for stochastic and adversarial bandits. Journal of Machine Learning Research, 22(28):1-49, 2021.
In the series of technical appendices that follow, we provide the missing proofs from the main part of our paper, and we provide some numerical illustrations of the performance of the proposed algorithms. As a roadmap, we begin in Appendix A with some auxiliary results that are required throughout our analysis. Subsequently, in Appendices B-D, we provide the proofs of the results presented in Sections 4-6 respectively. Finally, in Appendix E, we provide a suite of numerical experiments to assess the practical performance of (3MW) using the estimators (2PE) and (1PE), and we compare it with the full information setting underlying (MMW).
We now introduce some notation for quantum games in a  $N$ -player setting, and explain how the extension from the 2-player setting is straightforward.
$N$ -player quantum games. First of all, a quantum game  $\mathcal{Q}$  consists of a finite set of players  $i \in \mathcal{N} = \{1, \dots, N\}$ , where each player  $i \in \mathcal{N}$  has access to a complex Hilbert space  $\mathcal{H}_i \cong \mathbb{C}^{d_i}$ . The set of pure states is the unit sphere  $\Psi_i := \{\psi_i \in \mathcal{H}_i : \| \psi_i \|_F = 1\}$  of  $\mathcal{H}_i$ . We will write  $\Psi := \prod_i \Psi_i$  for the space of all ensembles  $\psi = (\psi_1, \dots, \psi_N)$  of pure states  $\psi_i \in \Psi_i$  that are independently prepared by each  $i \in \mathcal{N}$ .
In analogy with the 2-player case, each outcome  $\omega \in \Omega$  is associated to a positive semi-definite operator  $\mathbf{P}_{\omega} \colon \mathcal{H} \to \mathcal{H}$  defined on the tensor product  $\mathcal{H} \coloneqq \otimes_{i} \mathcal{H}_{i}$  of the players' individual state spaces; we further assume that  $\sum_{\omega \in \Omega} \mathbf{P}_{\omega} = \mathbf{I}$ , thus, the probability of observing  $\omega \in \Omega$  at state  $\psi \in \Psi$  is
$$
P _ {\omega} (\psi) = \left\langle \psi_ {1} \otimes \dots \otimes \psi_ {N} \mid \mathbf {P} _ {\omega} \mid \psi_ {1} \otimes \dots \otimes \psi_ {N} \right\rangle \tag {A.1}
$$
and, the player's expected payoff at state  $\psi \in \Psi$  will be
$$
u _ {i} (\psi) := \left\langle U _ {i} \right\rangle \equiv \sum_ {\omega} P _ {\omega} (\psi) U _ {i} (\omega) \tag {A.2}
$$
Similarly to the 2-player setting, if each player  $i \in \mathcal{N}$  prepares a density matrix  $\mathbf{X}_i$  as per (1), the expected payoff of player  $i \in \mathcal{N}$  under  $\mathbf{X} = (\mathbf{X}_1, \dots, \mathbf{X}_N)$  will be
$$
u _ {i} (\mathbf {X}) = \sum_ {\omega \in \Omega} U _ {i} (\omega) \operatorname {t r} \left[ \mathbf {P} _ {\omega} \mathbf {X} _ {1} \otimes \dots \otimes \mathbf {X} _ {N} \right] = \operatorname {t r} \left[ \mathbf {W} _ {i} \mathbf {X} _ {1} \otimes \dots \otimes \mathbf {X} _ {N} \right] \tag {A.3}
$$
where  $\mathbf{W}_i = \sum_{\omega \in \Omega} U_i(\omega) \mathbf{P}_{\omega} \in \mathcal{H}$  for  $i \in \mathcal{N}$ . Finally, we denote by  $\mathbf{V}_i(\mathbf{X})$  the individual payoff gradient of player  $i$  under  $\mathbf{X}$  as
$$
\mathbf {V} _ {i} (\mathbf {X}) := \nabla_ {\mathbf {X} _ {i} ^ {\top}} u _ {i} (\mathbf {X}) \tag {A.4}
$$
All other notions are extended, accordingly.
As noted in Section 2, we define the norm  $\| A\| _F = \sqrt{\mathrm{tr}[A^\dagger A]}$  for any  $A\in \mathbb{H}^{d_i}$ , i.e.,  $(\mathbb{H}^{d_i},\| \cdot \| _F)$  is an inner-product space. With a slight abuse of notation, we define for  $\mathbf{X} = (\mathbf{X}_1,\dots ,\mathbf{X}_N)\in \mathcal{X}$  its norm as:
$$
\| \mathbf {X} \| _ {F} = \sqrt {\sum_ {i = 1} ^ {N} \| \mathbf {X} _ {i} \| _ {F} ^ {2}} \tag {A.5}
$$
Lemma A.1. For any  $\mathbf{X}_i\in \mathcal{X}_i$  , it holds  $\| \mathbf{X}_i\| _F\leq 1$  , and  $\mathrm{diam}(\mathcal{X}) = 2\sqrt{N}$
Proof. For the first part, since  $\mathbf{X}_i\in \mathcal{X}_i$ , it admits an orthonormal decomposition  $Q\Lambda Q^{\dagger}$  such that  $QQ^{\dagger} = Q^{\dagger}Q = \mathbf{I}$  and  $\Lambda = \mathrm{diag}(\lambda_1,\dots ,\lambda_{d_i})$  with  $\sum_{j = 1}^{d_i}\lambda_j = 1$ , and  $\lambda_{j}\geq 0$  for all  $j$ . Hence
$$
\left\| \mathbf {X} _ {i} \right\| _ {F} ^ {2} = \operatorname {t r} \left[ \mathbf {X} _ {i} ^ {\dagger} \mathbf {X} _ {i} \right] = \operatorname {t r} \left[ Q \Lambda Q ^ {\dagger} Q \Lambda Q ^ {\dagger} \right] = \operatorname {t r} \left[ Q \Lambda^ {2} Q ^ {\dagger} \right] = \sum_ {j = 1} ^ {d _ {i}} \lambda_ {i} ^ {2} \leq \sum_ {j = 1} ^ {d _ {i}} \lambda_ {i} = 1 \tag {A.6}
$$
where the last inequality holds, since  $0 \leq \lambda_{j} \leq 1$ , and the result follows.
For the second part, letting  $\mathbf{X} = (\mathbf{X}_1,\dots ,\mathbf{X}_N)$  and  $\mathbf{X}' = (\mathbf{X}_1',\ldots ,\mathbf{X}_N')$  be two points in  $\mathcal{X}$ , we have
$$
\left\| \mathbf {X} - \mathbf {X} ^ {\prime} \right\| _ {F} = \sqrt {\sum_ {i = 1} ^ {N} \left\| \mathbf {X} _ {i} - \mathbf {X} _ {i} ^ {\prime} \right\| _ {F} ^ {2}} \leq \sqrt {\sum_ {i = 1} ^ {N} \left(2 \left\| \mathbf {X} _ {i} \right\| _ {F} ^ {2} + 2 \left\| \mathbf {X} _ {i} ^ {\prime} \right\| _ {F} ^ {2}\right)} \leq 2 \sqrt {N} \tag {A.7}
$$
and since the equality is attained, we get the result.
Our next result concerns the quantum relative entropy
$$
D (\mathbf {P}, \mathbf {X}) = \sum_ {i = 1} ^ {N} D _ {i} \left(\mathbf {P} _ {i}, \mathbf {X} _ {i}\right) \tag {A.8}
$$
where  $\mathbf{P} = (\mathbf{P}_1,\dots ,\mathbf{P}_N)\in \mathcal{X}$  and  $\mathbf{X} = (\mathbf{X}_1,\ldots ,\mathbf{X}_N)\in \mathrm{ri}(\mathcal{X})$  and
$$
D _ {i} \left(\mathbf {P} _ {i}, \mathbf {X} _ {i}\right) := \operatorname {t r} \left[ \mathbf {P} _ {i} \left(\log \mathbf {P} _ {i} - \log \mathbf {X} _ {i}\right) \right] \tag {A.9}
$$
The lemma we will require is a semidefinite version of Pinsker's inequality which reads as follows:
Lemma A.2. For all  $\mathbf{P} \in \mathcal{X}$  and  $\mathbf{X} \in \mathrm{ri}(\mathcal{X})$  we have
$$
D (\mathbf {P}, \mathbf {X}) \geq \frac {1}{2} \| \mathbf {P} - \mathbf {X} \| _ {F} ^ {2} \tag {A.10}
$$
Proof. Focusing on player  $i \in \mathcal{N}$ , we will show first that
$$
D _ {i} \left(\mathbf {P} _ {i}, \mathbf {X} _ {i}\right) \geq \frac {1}{2} \left\| \mathbf {P} _ {i} - \mathbf {X} _ {i} \right\| _ {F} ^ {2} \tag {A.11}
$$
for all  $\mathbf{P} = (\mathbf{P}_1,\dots ,\mathbf{P}_N)\in \mathcal{X}$  and  $\mathbf{X} = (\mathbf{X}_1,\ldots ,\mathbf{X}_N)\in \mathrm{ri}(\mathcal{X})$
To this end, we define the function  $h_i \colon \mathbb{H}_+^{d_i} \to \mathbb{R}$  as  $h_i(\mathbf{X}_i) = \mathrm{tr}[\mathbf{X}_i \log \mathbf{X}_i]$ , which is 1-strongly convex with respect to the nuclear norm  $\| \cdot \|_1$  [63], and since  $\| \mathbf{X}_i \|_1 \geq \| \mathbf{X}_i \|_F$  for all  $\mathbf{X}_i \in \mathcal{X}_i$ , we readily get that  $h_i$  is 1-strongly convex with respect to the Frobenius norm, as well.
Letting  $\nabla h_{i}(\mathbf{X}_{i}) = \log \mathbf{X}_{i} + \mathbf{I}$ , by 1-strong convexity, we have for  $\mathbf{P} = (\mathbf{P}_1,\dots ,\mathbf{P}_N)\in \mathcal{X}$  and  $\mathbf{X} = (\mathbf{X}_1,\ldots ,\mathbf{X}_N)\in \mathrm{ri}(\mathcal{X})$ :
$$
\begin{array}{l} h _ {i} (\mathbf {P} _ {i}) \geq h _ {i} (\mathbf {X} _ {i}) + \operatorname {t r} \left[ \nabla h _ {i} (\mathbf {X} _ {i}) (\mathbf {P} _ {i} - \mathbf {X} _ {i}) \right] + \frac {1}{2} \| \mathbf {P} _ {i} - \mathbf {X} _ {i} \| _ {F} ^ {2} \\ = \operatorname {t r} \left[ \mathbf {X} _ {i} \log \mathbf {X} _ {i} \right] + \operatorname {t r} \left[ \left(\mathbf {P} _ {i} - \mathbf {X} _ {i}\right) \log \mathbf {X} _ {i} \right] + \operatorname {t r} \left[ \mathbf {P} _ {i} - \mathbf {X} _ {i} \right] + \frac {1}{2} \| \mathbf {P} _ {i} - \mathbf {X} _ {i} \| _ {F} ^ {2} \\ = \operatorname {t r} \left[ \mathbf {P} _ {i} \log \mathbf {X} _ {i} \right] + \frac {1}{2} \| \mathbf {P} _ {i} - \mathbf {X} _ {i} \| _ {F} ^ {2} \tag {A.12} \\ \end{array}
$$
where we used that  $\operatorname{tr}[\mathbf{P}_i - \mathbf{X}_i] = 0$ . Hence, by reordering, we automatically get that
$$
D _ {i} \left(\mathbf {P} _ {i}, \mathbf {X} _ {i}\right) \geq \frac {1}{2} \left\| \mathbf {P} _ {i} - \mathbf {X} _ {i} \right\| _ {F} ^ {2} \tag {A.13}
$$
$$
D (\mathbf {P}, \mathbf {X}) \geq \frac {1}{2} \sum_ {i = 1} ^ {N} \| \mathbf {P} _ {i} - \mathbf {X} _ {i} \| _ {F} ^ {2} = \frac {1}{2} \| \mathbf {P} - \mathbf {X} \| _ {F} ^ {2} \tag {A.14}
$$
and the proof is completed.
B Omitted proofs from Section 4
In this appendix, we develop the basic scaffolding required for the estimators (2PE) and (1PE). We begin with the construction of the estimators' sampling basis, as encoded in Proposition 1, which we restate below for convenience:
Proposition 1. Let  $\mathbf{E}_j$  be defined as  $\mathbf{E}_j = \frac{1}{\sqrt{j(j + 1)}}\left(\Delta_{11} + \dots + \Delta_{jj} - j\Delta_{j + 1,j + 1}\right)$  for  $j = 1, \ldots, d - 1$ . Then, the set  $\mathcal{E} = \left\{\{\mathbf{E}_j\}_{j = 1}^{d - 1}, \{\mathbf{e}_{k\ell}\}_{k < \ell}, \{\tilde{\mathbf{e}}_{k\ell}\}_{k < \ell}\right\}$  is an orthonormal basis of  $\mathcal{Z}$ .
Proof. First of all, note that
$$
\Delta_ {k \ell} \Delta_ {m n} = \left\{ \begin{array}{l l} 0 & \text {i f} \ell \neq m \\ \Delta_ {k n} & \text {i f} \ell = m \end{array} \right. \tag {B.1}
$$
Unit norm. To begin with, we will show that all elements in  $\mathcal{E}$  have unit norm. Indeed, we have:
- For  $j = 1, \dots, d - 1$ , we have:
$$
\begin{array}{l} \left\| \mathbf {E} _ {j} \right\| _ {F} ^ {2} = \operatorname {t r} \left[ \mathbf {E} _ {j} ^ {\dagger} \mathbf {E} _ {j} \right] = \frac {1}{j (j + 1)} \operatorname {t r} \left[ \left(\sum_ {k = 1} ^ {j} \boldsymbol {\Delta} _ {k k} - j \boldsymbol {\Delta} _ {(j + 1) (j + 1)}\right) \left(\sum_ {k = 1} ^ {j} \boldsymbol {\Delta} _ {k k} - j \boldsymbol {\Delta} _ {(j + 1) (j + 1)}\right) \right] \\ = \frac {1}{j (j + 1)} \operatorname {t r} \left[ \left(\sum_ {k = 1} ^ {j} \boldsymbol {\Delta} _ {k k} + j ^ {2} \boldsymbol {\Delta} _ {(j + 1) (j + 1)}\right) \right] = \frac {1}{j (j + 1)} (j + j ^ {2}) = 1 \tag {B.2} \\ \end{array}
$$
- For  $k < \ell$ , we have:
$$
\begin{array}{l} \left\| \mathbf {e} _ {k \ell} \right\| _ {F} ^ {2} = \operatorname {t r} \left[ \mathbf {e} _ {k \ell} ^ {\dagger} \mathbf {e} _ {k \ell} \right] = \operatorname {t r} \left[ \left(\frac {1}{\sqrt {2}} \Delta_ {\ell k} + \frac {1}{\sqrt {2}} \Delta_ {k \ell}\right) \left(\frac {1}{\sqrt {2}} \Delta_ {k \ell} + \frac {1}{\sqrt {2}} \Delta_ {\ell k}\right) \right] \\ = \operatorname {t r} \left[ \frac {1}{2} \boldsymbol {\Delta} _ {k k} + \frac {1}{2} \boldsymbol {\Delta} _ {\ell \ell} \right] = \frac {1}{2} + \frac {1}{2} = 1 \tag {B.3} \\ \end{array}
$$
- For  $k < \ell$ , we also have:
$$
\begin{array}{l} \left\| \tilde {\mathbf {e}} _ {k \ell} \right\| _ {F} ^ {2} = \operatorname {t r} \left[ \tilde {\mathbf {e}} _ {k \ell} ^ {\dagger} \tilde {\mathbf {e}} _ {k \ell} \right] = \operatorname {t r} \left[ \left(- \frac {i}{\sqrt {2}} \boldsymbol {\Delta} _ {\ell k} + \frac {i}{\sqrt {2}} \boldsymbol {\Delta} _ {k \ell}\right) \left(\frac {i}{\sqrt {2}} \boldsymbol {\Delta} _ {k \ell} - \frac {i}{\sqrt {2}} \boldsymbol {\Delta} _ {\ell k}\right) \right] \\ = \operatorname {t r} \left[ \frac {1}{2} \boldsymbol {\Delta} _ {k k} + \frac {1}{2} \boldsymbol {\Delta} _ {\ell \ell} \right] = \frac {1}{2} + \frac {1}{2} = 1 \tag {B.4} \\ \end{array}
$$
Orthogonality. Now, we will show that any two elements of  $\mathcal{E}$  are orthogonal to each other.
- For  $m < n$ , we have:
$$
\begin{array}{l} \operatorname {t r} \left[ \mathbf {E} _ {m} ^ {\dagger} \mathbf {E} _ {n} \right] = \frac {1}{\sqrt {m (m + 1)} \sqrt {n (n + 1)}} \operatorname {t r} \left[ \left(\sum_ {k = 1} ^ {m} \Delta_ {k k} - m \Delta_ {(m + 1) (m + 1)}\right) \left(\sum_ {k = 1} ^ {n} \Delta_ {k k} - n \Delta_ {(n + 1) (n + 1)}\right) \right] \\ = \frac {1}{\sqrt {m (m + 1)} \sqrt {n (n + 1)}} \operatorname {t r} \left[ \left(\sum_ {k = 1} ^ {m} \boldsymbol {\Delta} _ {k k} - m \boldsymbol {\Delta} _ {(m + 1) (m + 1)}\right) \right] \\ = \frac {1}{\sqrt {m (m + 1)} \sqrt {n (n + 1)}} (m - m) = 0 \tag {B.5} \\ \end{array}
$$
$$
\begin{array}{l} \operatorname {t r} \left[ \mathbf {e} _ {k \ell} ^ {\dagger} \tilde {\mathbf {e}} _ {k \ell} \right] = \operatorname {t r} \left[ \left(\frac {1}{\sqrt {2}} \boldsymbol {\Delta} _ {\ell k} + \frac {1}{\sqrt {2}} \boldsymbol {\Delta} _ {k \ell}\right) \left(\frac {i}{\sqrt {2}} \boldsymbol {\Delta} _ {k \ell} - \frac {i}{\sqrt {2}} \boldsymbol {\Delta} _ {\ell k}\right) \right] \\ = \operatorname {t r} \left[ \frac {i}{2} \boldsymbol {\Delta} _ {\ell \ell} - \frac {i}{2} \boldsymbol {\Delta} _ {k k} \right] = \frac {i}{2} - \frac {i}{2} = 0 \tag {B.6} \\ \end{array}
$$
- For  $(k,\ell)\neq (m,n)$  with  $k < \ell$  and  $m < n$ , we have:
$$
\operatorname {t r} \left[ \mathbf {e} _ {k \ell} ^ {\dagger} \mathbf {e} _ {m n} \right] = \operatorname {t r} \left[ \mathbf {e} _ {k \ell} ^ {\dagger} \tilde {\mathbf {e}} _ {m n} \right] = \operatorname {t r} \left[ \tilde {\mathbf {e}} _ {k \ell} ^ {\dagger} \tilde {\mathbf {e}} _ {m n} \right] = 0 \tag {B.7}
$$
since all the nonzero terms in  $\mathbf{e}_{k\ell}^{\dagger}\mathbf{e}_{mn},\mathbf{e}_{k\ell}^{\dagger}\tilde{\mathbf{e}}_{mn}$  and  $\tilde{\mathbf{e}}_{k\ell}^{\dagger}\tilde{\mathbf{e}}_{mn}$  are of the form  $c\cdot \Delta_{\alpha \beta}$  for some  $c\in \mathbb{C}$ , and  $\alpha ,\beta \in \{k,\ell ,m,n\}$  with  $\alpha \neq \beta$ . Thus,  $\mathrm{tr}\bigl [c\cdot \Delta_{\alpha \beta}\bigr ] = 0$ , since all the diagonal elements are equal to 0. Note that it is not possible to have  $\alpha = \beta$  because this would imply that  $(k,\ell) = (m,n)$ .
- For  $k < \ell$  and  $j = 1, \dots, d - 1$ , we have:
$$
\operatorname {t r} \left[ \mathbf {e} _ {k \ell} ^ {\dagger} \mathbf {E} _ {j} \right] = \operatorname {t r} \left[ \tilde {\mathbf {e}} _ {k \ell} ^ {\dagger} \mathbf {E} _ {j} \right] = 0 \tag {B.8}
$$
since the non-zero terms of both  $\mathbf{e}_{k\ell}^{\dagger}\mathbf{E}_j$  and  $\tilde{\mathbf{e}}_{k\ell}^{\dagger}\mathbf{E}_j$  are of the form  $\Delta_{kn},\Delta_{\ell m}$  for  $k\neq n$  and  $\ell \neq m$
We thus conclude that any two elements of  $\mathcal{E}$  are orthogonal.
Finally, it is clear  $\mathcal{E} \subseteq \operatorname{aff}(\mathcal{X}_0)$ , since  $\mathcal{E} \subseteq \mathbb{H}^d$  and  $\operatorname{tr}[\mathbf{e}_{k\ell}] = \operatorname{tr}[\tilde{\mathbf{e}}_{k\ell}] = \operatorname{tr}\big[\mathbf{E}_j\big] = 0$ , for  $k < \ell$  and  $j = 1,\ldots,d-1$ . Therefore, the elements in  $\mathcal{E}$  form an orthonormal basis of  $\operatorname{aff}(\mathcal{X}_0)$  and  $\dim(\operatorname{aff}(\mathcal{X}_0)) = d^2 - 1$ .
We now proceed with the construction of the precise "safety net" that guarantees that the sampling perturbation of the gradient estimator remains within the problem's feasible region. Again, for convenience, we restate the relevant result below:
Proof. To begin with, it is clear that  $\mathbf{R} \in \mathbb{H}^d$  and  $\operatorname{tr}[\mathbf{R}] = \sum_{j=1}^{d} 1 / d = 1$ . Moreover, for any  $u \in \mathbb{C}^d \setminus \{0\}$ , we have:
$$
u ^ {\dagger} \mathbf {R} u = \frac {1}{d} \sum_ {j = 1} ^ {d} | u _ {j} | ^ {2} > 0 \tag {B.9}
$$
where  $|u_{j}|$  is the modulus of the complex number  $u_{j}\in \mathbb{C}$ . Therefore,  $\mathbf{R}$  is positive definite, i.e., lies in  $\operatorname {ri}(\mathcal{X})$ .
Now, we need to find  $r > 0$  such that
$$
\mathbf {R} + r \mathbf {Z} \in \mathcal {X} \tag {B.10}
$$
for any  $\mathbf{Z}\in \mathcal{E}^{\pm}$
It is clear that for any  $\mathbf{Z} \in \mathcal{E}^{\pm}$ , we have  $\mathrm{tr}[\mathbf{R} + r\mathbf{Z}] = \mathrm{tr}[\mathbf{R}] = 1$ , since  $\mathrm{tr}[\mathbf{Z}] = 0$ . Hence, it remains to consider the positive semi-definite constraint. For this, we will use the following identities, for  $k < \ell$ :
$$
u ^ {\dagger} \left(\Delta_ {k \ell} + \Delta_ {\ell k}\right) u = \bar {u} _ {k} u _ {\ell} + \bar {u} _ {\ell} u _ {k} = 2 \operatorname {R e} \left(\bar {u} _ {k} u _ {\ell}\right) \tag {B.11}
$$
$$
u ^ {\dagger} \left(i \Delta_ {k \ell} - i \Delta_ {\ell k}\right) u = i \left(\bar {u} _ {k} u _ {\ell} - \bar {u} _ {\ell} u _ {k}\right) = - 2 \operatorname {I m} \left(\bar {u} _ {k} u _ {\ell}\right) \tag {B.12}
$$
For  $\mathbf{Z} = \frac{1}{\sqrt{2}} (\Delta_{k\ell} + \Delta_{\ell k})$  and  $u\in \mathbb{C}^d\setminus \{0\}$ , and using (B.11), we have:
$$
\begin{array}{l} u ^ {\dagger} (\mathbf {R} + r \mathbf {Z}) u = \frac {1}{d} \sum_ {j = 1} ^ {d} | u _ {j} | ^ {2} + \frac {r}{\sqrt {2}} 2 \mathrm {R e} (\bar {u} _ {k} u _ {\ell}) \\ = \frac {1}{d} \sum_ {j \neq k, \ell} | u _ {j} | ^ {2} + \frac {1}{d} \left(| u _ {k} | ^ {2} + | u _ {\ell} | ^ {2} + \frac {r d}{\sqrt {2}} 2 \operatorname {R e} \left(\bar {u} _ {k} u _ {\ell}\right)\right) \tag {B.13} \\ \end{array}
$$
If  $\operatorname{Re}(\bar{u}_k u_\ell) > 0$ , we get that  $u^\dagger(\mathbf{R} + r\mathbf{Z})u > 0$ , while if  $\operatorname{Re}(\bar{u}_k u_\ell) \leq 0$  and  $r \leq \sqrt{2}/d$ :
$$
u ^ {\dagger} (\mathbf {R} + r \mathbf {Z}) u \geq \frac {1}{d} \sum_ {j \neq k, \ell} | u _ {j} | ^ {2} + \frac {1}{d} | u _ {k} + u _ {\ell} | ^ {2} \geq 0 \tag {B.14}
$$
Hence, for  $r \leq \sqrt{2} / d$ , and  $\mathbf{Z} = \frac{1}{\sqrt{2}} (\Delta_{k\ell} + \Delta_{\ell k})$ , we have that  $u^{\dagger}(\mathbf{R} + r\mathbf{Z})u \geq 0$  for all  $u \in \mathbb{C}^d$ .
For  $\mathbf{Z} = -\frac{1}{\sqrt{2}} (\Delta_{k\ell} + \Delta_{\ell k})$  we have
$$
\begin{array}{l} u ^ {\dagger} (\mathbf {R} + r \mathbf {Z}) u = \frac {1}{d} \sum_ {j = 1} ^ {d} | u _ {j} | ^ {2} - \frac {r}{\sqrt {2}} 2 \operatorname {R e} \left(\bar {u} _ {k} u _ {\ell}\right) \\ = \frac {1}{d} \sum_ {j \neq k, \ell} | u _ {j} | ^ {2} + \frac {1}{d} \left(| u _ {k} | ^ {2} + | u _ {\ell} | ^ {2} - \frac {r d}{\sqrt {2}} 2 \operatorname {R e} \left(\bar {u} _ {k} u _ {\ell}\right)\right) \tag {B.15} \\ \end{array}
$$
If  $\operatorname{Re}(\bar{u}_k u_\ell) < 0$ , we get that  $u^\dagger(\mathbf{R} + r\mathbf{Z})u > 0$ , while if  $\operatorname{Re}(\bar{u}_k u_\ell) \geq 0$  and  $r \leq \sqrt{2}/d$ :
$$
u ^ {\dagger} (\mathbf {R} + r \mathbf {Z}) u \geq \frac {1}{d} \sum_ {j \neq k, \ell} | u _ {j} | ^ {2} + \frac {1}{d} | u _ {k} - u _ {\ell} | ^ {2} \geq 0 \tag {B.16}
$$