SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The following example shows that the bounds in [19] can be vacuous for our setting. Since, as noted by the authors, Theorem 5 by [41] is equivalent to the results in [19], the following example is only with respect to the latter.
Example 18. For this example, we use the notation from [19]. Consider the game matrix  $M = I_K$ . Observe that  $M$  has a unique Nash-equilibrium, hence, by considering  $p = \frac{1}{K} \mathbf{I}$  and  $q = \delta_i$  it can be seen that  $\delta(M)$  is at most  $\sqrt{\frac{K}{K - 1}} \frac{1}{K} \approx \frac{1}{K}$ . The bound is defined with respect to the condition number  $\kappa(M) = \sqrt{\lambda_{\max} M \tau M} / \delta(M)$ , where  $\lambda_{\max}(M)$  denotes the largest eigenvalue of  $M$ . Hence, for our example,  $\kappa(M)$  is at least of the order of  $K$ , which makes a bound of  $\kappa(M) \log \frac{1}{\varepsilon}$  for this specific example vacuous. Note that for other game matrices, the results give valuable insights into guarantees beyond the worst-case.
B.2 Constant Queries for Games from a Finite Alphabet
Proof. Let  $F$  be the smallest subfield of  $\mathbb{R}$  that contains  $\mathcal{A}$ , then  $F$  is countable and,  $\mathbb{R}$  can be seen as an infinite-dimensional vector space over  $F$ . Recall that a family of real numbers  $(x_{1},\ldots ,x_{n})$
(each seen as a vector over the field  $F$ ) is linearly independent if for any  $\lambda_1, \ldots, \lambda_n \in F$  we have  $\lambda_1 x_1 + \dots + \lambda_n x_n = 0$  if and only if  $\lambda_1 = \dots = \lambda_n = 0$ .
We claim that if a player, say the  $q$ -player, queries an action such that the components  $q_{1}, \ldots, q_{K}$  form a linearly independent family, then they can compute the whole matrix  $M$  with just one observation. Indeed, if  $M, M' \in \mathcal{M}_K(\mathcal{A})$ , yield the same output after one query, then for any  $i \in [K]$ :
$$
\sum_ {i = 1} ^ {K} (M _ {i, j} - M _ {i, j} ^ {\prime}) q _ {i} = 0.
$$
Therefore, by the independence of  $(q_{i})_{i\in [K]}$ , this implies that  $M_{i,j} = M_{i,j}^{\prime}$  for any  $(i,j)$ . In other words, no two different matrices can give the same output after one query under  $q$ .
We are now left to show that there exists a play  $(q_{1},\ldots ,q_{K})$  with coordinates forming a linearly independent family. We prove this using the probabilistic method. Consider a sequence of random variables  $(U_{1},\dots ,U_{K})$  i.i.d. and uniformly distributed over [0, 1]. Then with probability 1, the  $U_{i}$  are independent over  $F$ . Indeed, we can upper bound the probability that they are dependent by a union bound and use of the tower rule as
$$
\begin{array}{l} \mathbb {P} \left[ \exists i \in [ K ] \quad \text {s . t .} \quad U _ {i} \in \operatorname {S p a n} _ {F} \left\{U _ {j} \mid j \neq i \right\} \right] \leqslant \sum_ {i = 1} ^ {K} \mathbb {P} \left[ U _ {i} \in \operatorname {S p a n} _ {F} \left\{U _ {j} \mid j \neq i \right\} \right] \\ = \sum_ {i = 1} ^ {K} \mathbb {E} \left[ \mathbb {P} \left[ U _ {i} \in \operatorname {S p a n} _ {F} \left\{U _ {j} \mid j \neq i \right\} \mid \left\{U _ {j} \mid j \neq i \right\} \right] \right] = 0. \\ \end{array}
$$
The last equality holds because, for any  $i \in [K]$ , conditionally on  $\{U_j \mid j \neq i\}$ , the span of  $\{U_j \mid j \neq i\}$  is a countable set, therefore the probability that  $U_i$  belongs to that set is null. This concludes the proof.
C First-Order Query Complexity of Recovering the Game Matrix
Proof of Theorem 6. Clearly, we can fully reconstruct  $M$  from the queries  $p_t = q_t = e_t$  for  $t = 1, \ldots, K$ , where  $e_t$  denotes the standard basis vector in direction  $t$ . It turns out that this is optimal.
To show this, note that each query  $(p,q)$  provides us with constraints
$$
p ^ {\intercal} M = a, \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad M q = b,
$$
for some loss vectors  $a$  and  $b$ . These may equivalently be expressed as linear constraints in the Hilbert space of matrices  $A \in \mathbb{R}^{K \times K}$ , with inner product  $\langle A, B \rangle = \operatorname{Tr}(A^{\intercal}B)$ :
$$
\langle M ^ {\mathsf {T}}, e _ {i} p ^ {\mathsf {T}} \rangle = a _ {i} \qquad (i = 1, \dots , K),
$$
$$
\langle M ^ {\intercal}, q e _ {j} ^ {\intercal} \rangle = b _ {j} \qquad (j = 1, \dots , K).
$$
Among these  $2K$  constraints on  $M$ , there is (at least) one redundant constraint, because there always exist numbers  $\lambda_1,\ldots ,\lambda_K$  and  $\gamma_{1},\dots,\gamma_{K}$ , at least one of which is nonzero, such that
$$
\sum_ {i = 1} ^ {K} \lambda_ {i} e _ {i} p ^ {\intercal} + \sum_ {i = 1} ^ {K} \gamma_ {j} q e _ {j} ^ {\intercal} = 0.
$$
Specifically, this holds for  $\lambda_{i} = q_{i}$  and  $\gamma_{j} = -p_{j}$ . It follows that a query  $(p_t,q_t)$  in round  $t$  will provide at most  $2(K - t) + 1$  new constraints on top of the queries from rounds  $1,\ldots ,t - 1$ . To see this, note that  $p_t$  will have at least one constraint in common with  $q_{1},\ldots ,q_{t - 1}$  and  $q_{t}$  will have at least one constraint in common with  $p_1,\dots ,p_t$ , so the total number of new constraints is at most  $n_t\coloneqq 2K - (t - 1) - t = 2(K - t) + 1$ .
We now provide the following scenario in which  $M$  cannot be fully determined by strictly less than  $K$  queries. In rounds  $t = 1, \ldots, K - 1$ , the answer to queries  $(p_t, q_t)$  is always the two loss vectors  $\ell_t^{(p)} = \ell_t^{(q)} = \frac{1}{2} \mathbf{1}$ , which are compatible with the possibility that  $M$  equals  $\frac{1}{2} \mathbf{1} \mathbf{1}^{\top}$ , i.e. the matrix with all entries equal to  $1/2$ . We will show by induction that the dimension of the null-space (i.e. the
number of unconstrained dimensions of  $M$ ) of all constraints up to round  $t$  is at least  $(K - t)^2$ . This is true for  $t = 0$ , because the domain of  $M$  has  $K^2$  dimensions, and, whenever it is true for  $t$ , then for  $t + 1$  it is at least
$$
(K - t) ^ {2} - n _ {t + 1} = (K - t) ^ {2} - 2 (K - t - 1) - 1 = (K - t) ^ {2} - 2 (K - t) + 1 = (K - t - 1) ^ {2}.
$$
Thus, after  $K - 1$  rounds, there remains at least one direction  $V \in \mathbb{R}^{K \times K}$  in this null space with  $V \neq 0$ . This means that the learner cannot distinguish the case  $M = \frac{1}{2}\mathbf{11}^{\intercal}$  from the case  $M = \frac{1}{2}\mathbf{11}^{\intercal} + \frac{1}{2}V / \max_{i,j}|V_{i,j}|$ , thus  $K - 1$  queries are not sufficient to fully determine  $M$ .
D Proofs and Technical Results for Section 4
D.1 Approximate Equilibria
D.1.1 Proofs of Main Results
Proof of Corollary 14. If  $(p,q)$  is a common  $\varepsilon$ -NE to all matrices in  $\mathcal{E}_t$ , then we have for any game matrix  $M \in \mathcal{E}_t$ ,
$$
M ^ {\intercal} (p - \bar {p}) \in \operatorname {S p a n} \left(\ell_ {1: t} ^ {(q)}\right)
$$
and therefore for any  $v > 0$
$$
\| v \mathbf {1} - \operatorname {P r o j} _ {\operatorname {S p a n} (\ell_ {1: t} ^ {(q)})} (v \mathbf {1}) \| \leqslant \| v \mathbf {1} - M ^ {\intercal} (p - \bar {p}) \| \leqslant \| v \mathbf {1} - M ^ {\intercal} p \| + \| M ^ {\intercal} \bar {p} \| \leqslant \| v \mathbf {1} - M ^ {\intercal} p \| + \sqrt {K} \| \bar {p} \|.
$$
Now as  $\delta > 0$ , any  $M \in \mathcal{B}_{\varepsilon, \delta}$  has a unique Nash equilibrium  $p^{\star}, q^{\star}$ , which is fully supported. Therefore, the value of  $M$  is  $v = p^{\intercal} M q^{\star}$ , as  $q^{\star}$  is an equalizing strategy. Now, using (1) in the proof of Lemma 13, which is valid for any matrix that admits  $p, q$  as an  $\varepsilon$ -NE,
$$
\left\| v \mathbf {1} - M ^ {\intercal} p \right\| \leqslant \sqrt {K} \left\| v \mathbf {1} - M ^ {\intercal} p \right\| _ {\infty} \leqslant \frac {\sqrt {K} \varepsilon}{\delta}.
$$
Proof of Lemma 13. First note that  $p$  and  $q$  are  $\delta$ -supported, as  $(p,q)$  is an  $\varepsilon$ -equilibrium of at least one matrix in  $\mathcal{E}_t \cap B(\varepsilon, \delta)$ . For  $M \in \mathcal{E}_t$ , pick any  $j^{\star} \in \arg \min_{j \in [K]} (M^{\intercal} p)_j$ . As  $(p,q)$  is an  $\varepsilon$ -equilibrium for  $M$ ,
$$
\left(1 - q (j ^ {\star})\right) \max  _ {j \in [ K ]} (M ^ {\intercal} p) _ {j} + q (j ^ {\star}) \min  _ {j \in [ K ]} (M ^ {\intercal} p) _ {j} \geqslant \sum_ {i = 1} ^ {K} q (j) (M ^ {\intercal} p) _ {j} \geqslant \max  _ {j \in [ K ]} (M ^ {\intercal} p) _ {j} - \varepsilon ,
$$
and therefore, dividing by  $q(j^{\star})\geqslant \delta$
$$
\max  _ {j \in [ K ]} (M ^ {\mathsf {T}} p) _ {j} - \min  _ {j \in [ K ]} (M ^ {\mathsf {T}} p) _ {j} \leqslant \frac {\varepsilon}{q (j ^ {\star})} \leqslant \frac {\varepsilon}{\delta}. \tag {1}
$$
Hence, as  $\min_{j\in [K]}(M^{\intercal}p)_j\leqslant p^{\intercal}Mq_1\leqslant \max_{j\in [K]}(M^{\intercal}p)_j$
$$
\left\| M ^ {\mathsf {T}} p - \left(p ^ {\mathsf {T}} M q _ {1}\right) \mathbf {1} \right\| _ {\infty} \leqslant \frac {\varepsilon}{\delta}.
$$
Since  $p^{\mathsf{T}}Mq_{1} = p^{\mathsf{T}}\ell_{1}^{(p)}$  has the same value for any  $M \in \mathcal{E}_t$ , we apply this inequality for any pair of matrices  $M, M' \in \mathcal{E}_t$  to deduce that
$$
\left\| \left(M - M ^ {\prime}\right) ^ {\intercal} p \right\| _ {\infty} \leqslant \frac {2 \varepsilon}{\delta}. \tag {2}
$$
Let us now instantiate this identity with some well-chosen  $M$  and  $M'$ . Let  $u \in \mathbb{R}^K$  be a vector orthogonal to  $q_1, \ldots, q_t$ , such that  $\| u \|_{\infty} = 1$  (such a  $u$  exists as long as  $q_1, \ldots, q_t$  do not span the whole of  $\mathbb{R}^K$ ). Next consider  $M$  and  $M'$  in  $\mathcal{E}_t$  such that
$$
M - M ^ {\prime} = \frac {r _ {t}}{\| \bar {p} \| _ {\infty}} \bar {p} u ^ {\intercal}.
$$
Such  $M$  and  $M^{\prime}$  are guaranteed to exist in  $\mathcal{E}_t$  as  $\| M - M^{\prime}\| = r_{t}$ . Then, applying (2), we obtain
$$
\frac {2 \varepsilon}{\delta} \geqslant \| (M - M ^ {\prime}) ^ {\intercal} p \| _ {\infty} = \frac {r _ {t}}{\| \bar {p} \| _ {\infty}} \| \bar {p} \| _ {2} ^ {2} \| u \| _ {\infty} \geqslant r _ {t} \| \bar {p} \| _ {2},
$$
from which the claim follows. The same reasoning applies to obtain the bound on  $q$ .
Proof of Lemma 15. Denote by  $\Pi_t$  the projection on the span of the observed losses  $\ell_{1:t}^{(q)}$  for the  $q$ -player at time step  $t$ . We build the sequence  $(M_t)$  incrementally by moving at step  $t + 1$  in directions chosen as a function of the new queries  $p_{t + 1}$  and  $q_{t + 1}$ .
We initialize the sequence at  $M_0$  the center of  $B$ . Now for  $t \geqslant 0$ , if  $p_{t+1}$  is in the span of  $p_{1:t}$ , then set  $M_{t+1} = M_t$ , otherwise set
$$
M _ {t + 1} = M _ {t} + \frac {\bar {p} _ {t + 1}}{\| \bar {p} _ {t + 1} \| ^ {2}} u _ {t} ^ {\intercal},
$$
where  $\bar{p}_{t+1} = p_{t+1} - \mathrm{Proj}_{\mathrm{Span}(p_{1:t})}(p_{t+1})$ , and  $u_t$  is a non-zero vector orthogonal to the vectors  $q_{1:t}$  to  $\ell_{1:t}^{(q)}$ , to  $\mathbf{1}$ , and to  $M_t^\top p_{t+1}$ ; the existence of such a  $u_t$  is guaranteed since the only condition is that it is orthogonal to  $2t + 2 < K$  vectors. (Note that  $u_t$  does not depend on the value of  $v$ .) We set the norm of  $u_t$  at a later stage of the proof. Then,
$$
\ell_ {t + 1} ^ {(q)} = M _ {t + 1} ^ {\intercal} p _ {t + 1} = M _ {t} ^ {\intercal} p _ {t + 1} + u _ {t}.
$$
Then for any  $v \in \mathbb{R}$ , the squared distance from the vector  $v1$  to the space  $\operatorname{Span}(\ell_{1:t+1}^{(q)})$  can be decomposed thanks to the Pythagorean equality, as the squared distance to the previous span minus the squared norm of the projection on the new orthogonal direction  $\ell_{t+1}^{(q)} - \Pi_t(\ell_{t+1}^{(q)})$ :
$$
\| v \mathbf {1} - \Pi_ {t + 1} (v \mathbf {1}) \| ^ {2} = \| v \mathbf {1} - \Pi_ {t} (v \mathbf {1}) \| ^ {2} - \underbrace {\frac {\langle v \mathbf {1} , M _ {t} ^ {\mathsf {T}} p _ {t + 1} + u _ {t} - \Pi_ {t} (M _ {t} ^ {\mathsf {T}} p _ {t + 1} + u _ {t}) \rangle^ {2}}{\| M _ {t} ^ {\mathsf {T}} p _ {t + 1} + u _ {t} - \Pi_ {t} (M _ {t} ^ {\mathsf {T}} p _ {t + 1} + u _ {t}) \| ^ {2}}} _ {:= D _ {t}}.
$$
Observe that for any vectors  $a, b$ , we have  $\langle a - \Pi_t(a), b \rangle = \langle a - \Pi_t(a), b - \Pi_t(b) \rangle = \langle a, b - \Pi_t(b) \rangle$  as  $\Pi_t$  is an orthogonal projection on a linear subspace. Using this identity, as well as the orthogonality conditions used to define  $u_t$  (precisely, that  $\Pi_t(u_t) = 0$ , and that  $u_t$  is orthogonal to  $M_t^\top p_{t+1}$  and  $\Pi_t(M_t^\top p_{t+1})$ ), we obtain after applying Cauchy-Schwarz,
$$
\begin{array}{l} D _ {t} = \frac {\left\langle v \mathbf {1} - \Pi_ {t} (v \mathbf {1}) , M _ {t} ^ {\intercal} p _ {t + 1} + u _ {t} \right\rangle^ {2}}{\| M _ {t} ^ {\intercal} p _ {t + 1} + u _ {t} - \Pi_ {t} \left(M _ {t} ^ {\intercal} p _ {t + 1} + u _ {t}\right) \| ^ {2}} \\ = \frac {\left\langle v \mathbf {1} - \Pi_ {t} (v \mathbf {1}) , M _ {t} ^ {\intercal} p _ {t + 1} + u _ {t} \right\rangle^ {2}}{\| u _ {t} \| ^ {2} + \| M _ {t} ^ {\intercal} p _ {t + 1} - \Pi_ {t} (M _ {t} ^ {\intercal} p _ {t + 1}) \| ^ {2}} \\ = \frac {\left\langle v \mathbf {1} - \Pi_ {t} (v \mathbf {1}) , M _ {t} ^ {\intercal} p _ {t + 1} - \Pi_ {t} \left(M _ {t} ^ {\intercal} p _ {t + 1}\right) \right\rangle^ {2}}{\| u _ {t} \| ^ {2} + \| M _ {t} ^ {\intercal} p _ {t + 1} - \Pi_ {t} \left(M _ {t} ^ {\intercal} p _ {t + 1}\right) \| ^ {2}} \\ \leqslant \| v \mathbf {1} - \Pi_ {t} (v \mathbf {1}) \| ^ {2} \frac {\| M _ {t} ^ {\intercal} p _ {t + 1} - \Pi_ {t} \left(M _ {t} ^ {\intercal} p _ {t + 1}\right) \| ^ {2}}{\| M _ {t} ^ {\intercal} p _ {t + 1} - \Pi_ {t} \left(M _ {t} ^ {\intercal} p _ {t + 1}\right) \| ^ {2} + \| u _ {t} \| ^ {2}} \\ \leqslant \| v \mathbf {1} - \Pi_ {t} (v \mathbf {1}) \| ^ {2} \frac {\| M _ {t} ^ {\intercal} p _ {t + 1} \| ^ {2}}{\| M _ {t} ^ {\intercal} p _ {t + 1} \| ^ {2} + \| u _ {t} \| ^ {2}}. \\ \end{array}
$$
We also used the fact that projections reduce the norm, and the function  $x \mapsto x / (x + 1)$  is increasing on  $(0, +\infty)$  to obtain the final inequality.
Using this bound on  $D_{t}$  to lower bound the distance to the span, we obtain
$$
\left\| \left(I _ {K} - \Pi_ {t + 1}\right) v \mathbf {1} \right\| ^ {2} \geqslant \left\| \left(I _ {K} - \Pi_ {t}\right) v \mathbf {1} \right\| ^ {2} \left(1 - \frac {1}{1 + \| u _ {t} \| ^ {2} / \| M _ {t} ^ {\intercal} p _ {t + 1} \| ^ {2}}\right). \tag {3}
$$
We are now left to choose the norm of  $u_{t}$ ; the objective is to make it as big as possible under the constraint that the sequence  $(M_t)$  stays in  $B$ . We set the norm of  $u_{t}$  to be a constant multiple of  $\| M_t^\top p_{t + 1}\|$ , i.e.,
$$
\left\| u _ {t} \right\| = \sqrt {\alpha} \left\| M _ {t} ^ {\intercal} p _ {t + 1} \right\|.
$$
For  $\alpha$  small enough, we can ensure that the whole sequence  $(M_t)$  stays in the ball  $B$ , as
$$
\begin{array}{l} \| M _ {t} - M _ {0} \| _ {1, \infty} \leqslant \sum_ {s = 1} ^ {t} \frac {1}{\| \bar {p} _ {s + 1} \| ^ {2}} \| \bar {p} _ {s + 1} u _ {s} ^ {\intercal} \| _ {1, \infty} = \sum_ {s = 1} ^ {t} \frac {\| \bar {p} _ {s + 1} \| _ {\infty}}{\| \bar {p} _ {s + 1} \| ^ {2}} \| u _ {s} \| _ {\infty} = \sum_ {s = 1} ^ {t} \frac {\| \bar {p} _ {s + 1} \| _ {\infty}}{\| \bar {p} _ {s + 1} \| ^ {2}} \| u _ {s} \| _ {\infty} \\ \leqslant \sum_ {s = 1} ^ {t} \frac {\| u _ {s} \|}{\| \bar {p} _ {s + 1} \|} = \sqrt {\alpha} \sum_ {t = 1} ^ {t} \frac {\| M _ {t} \bar {p} _ {t + 1} \|}{\| \bar {p} _ {t + 1} \|} \leqslant \sqrt {\alpha} \sqrt {K} t, \\ \end{array}
$$
where we used  $\| M_{t}x\| \leqslant \| M_{t}\|_{2,2}\| x\| \leqslant \sqrt{K}\| x\|$ , and  $M_t^\top \bar{p}_{t + 1} = M_t^\top p_{t + 1}$  since  $M_{t}\in \mathcal{E}_{t}$ . Therefore by taking,  $\alpha = (r / 2)^2 /(KT^2)$ , we ensure that  $M_{t}\in B$  as  $B$  contains a ball of radius  $r$  centered at  $M_0$ . Furthermore, this choice ensures that  $\mathcal{E}_t$  contains the ball of radius  $r / 2$  centered at  $M_{t}$  in its relative interior as for any matrix  $U$  with norm less than  $r / 2$ , we have  $\| M_t + U - M_0\| \leqslant \| M_t - M_0\| +\| U\| \leqslant r / 2$ , so  $M_{t} + U\in B\subset \mathcal{M}_{K}([0,1])$ .
Plugging back the value of  $\alpha$  into (3), we get
$$
\left\| \left(I _ {K} - \Pi_ {t + 1}\right) v \mathbf {1} \right\| ^ {2} \geqslant \left\| \left(I _ {K} - \Pi_ {t}\right) v \mathbf {1} \right\| ^ {2} \frac {\alpha}{1 + \alpha} \geqslant \left\| \left(I _ {K} - \Pi_ {t}\right) v \mathbf {1} \right\| ^ {2} \frac {\alpha}{2}.
$$
After  $T$  time steps, this implies that for any  $v\geqslant 0$
$$
\left\| \left(I _ {K} - \Pi_ {T}\right) v \mathbf {1} \right\| ^ {2} \geqslant \left\| \left(I _ {K} - \Pi_ {0}\right) v \mathbf {1} \right\| ^ {2} \left(\frac {\alpha}{2}\right) ^ {T} = \left\| v \mathbf {1} \right\| ^ {2} \left(\frac {r ^ {2}}{8 K T ^ {2}}\right) ^ {T + 1}.
$$
This is the claimed result.
Proof of Theorem 17. Define  $B_{\varepsilon, \delta} = \mathcal{B}_{\| \cdot \|_{1,\infty}}\left(\frac{1}{2} I_K, \frac{1}{16K^2}\right)$ , which is a ball of radius  $r = 1 / (16K^2)$ . Given an algorithm, consider the sequence of matrices  $M_{1:T}$  generated by Lemma 15, applied with  $B = B_{\varepsilon, \delta}$ ; we know in particular that  $M_t \in B_{\varepsilon, \delta}$ , so  $B_{\varepsilon, \delta} \cap \mathcal{E}_t \neq \emptyset$ .
Assume now that the algorithm outputs a common  $\varepsilon$ -equilibrium to all matrices in  $\mathcal{E}_T$ . The assumptions of Corollary 14 hold, instantiated with  $B_{\varepsilon,\delta}$  and  $r_t = 1/(32K^2)$ . Therefore there exists  $M \in B_{\varepsilon,\delta}$ , with value  $v$  such that
$$
\left\| v \mathbf {1} - \operatorname {P r o j} _ {\operatorname {S p a n} \left(\ell_ {1: t} ^ {(q)}\right)} (v \mathbf {1}) \right\| \leqslant \frac {4 \sqrt {K} \varepsilon}{\delta r _ {t}}.
$$
On the other hand, by Lemma 15 also instantiated with  $B = B_{\varepsilon, \delta}$ , we know that
$$
\left\| v \mathbf {1} - \operatorname {P r o j} _ {\operatorname {S p a n} \left(\ell_ {1: T} ^ {(q)}\right)} (v \mathbf {1}) \right\| \geqslant v \sqrt {K} \left(\frac {r ^ {2}}{8 K T ^ {2}}\right) ^ {(T + 1) / 2}.
$$
$$
\varepsilon \geqslant \frac {v \delta r _ {t}}{4} \left(\frac {r ^ {2}}{8 K T ^ {2}}\right) ^ {(T + 1) / 2}.
$$
Finally, since  $v$  is the value of a matrix  $M \in B_{\varepsilon, \delta}$ ,
$$
v = \min  _ {p \in \Delta_ {K}} \max  _ {q \in \Delta_ {K}} p ^ {\intercal} M q \geqslant \min  _ {p \in \Delta_ {K}} \max  _ {q \in \Delta_ {K}} \frac {p ^ {\intercal} q}{2} - K r \geqslant \frac {1}{2 K} - \frac {K}{1 6 K ^ {2}} \geqslant \frac {1}{4 K}.
$$
We conclude by replacing the constants by their values,  $\delta = 1 / (2K)$ , and  $r_t = r / 2 = 1 / (32K^2)$ . By Lemma 19 in Appendix D, we deduce that for  $(2\varepsilon) \leqslant 1 / (e2^{10}K^4)$ ,
$$
T + 1 \geqslant \frac {- \log (2 ^ {1 1} K ^ {4} \varepsilon)}{\log (2 ^ {1 1 / 2} K ^ {5 / 2}) + \log (- \log (2 ^ {1 1 =} K ^ {4} \varepsilon))}.
$$
D.1.2 Technical Lemmas
Lemma 19. For any  $a, b, x > 0$ , for any  $\varepsilon \leqslant a / e$
$$
i f \quad \varepsilon \geqslant a (1 / (b x)) ^ {x}, \quad t h e n \quad x \geqslant \frac {\log (a / \varepsilon)}{\log (b \log (a / \varepsilon))}.
$$
Proof of Lemma 19. Assume
$$
\varepsilon \geqslant a (1 / (b x)) ^ {x} = e ^ {- x \log (b x)}
$$
$$
\left(\varepsilon / a\right) ^ {b} \geqslant e ^ {- b x \log (b x)}
$$
thus, applying logarithms to both sides,
$$
b \log (\varepsilon / a) \geqslant - b x \log b x.
$$
Reformulate as, since  $a \geqslant e\varepsilon$
$$
b x \log b x \geqslant b \log (a / \varepsilon) \geqslant b > 0.
$$
We now apply Lambert's  $W_0$  function, which is increasing on its main branch  $[-1 / e, +\infty)$ , and use a standard lower bound on  $W_0$  to obtain
$$
\log b x \geqslant W _ {0} (- b \log (\varepsilon / a)) \geqslant \log (- b \log (\varepsilon / a)) - \log \log (- b \log (\varepsilon / a))
$$
$$
b x \geqslant - b \log (\varepsilon / a) / \log (- b \log (\varepsilon / a)),
$$
which is the claimed bound.
Lemma 20. Let  $\alpha \geqslant 0$  and  $s > 0$  be real numbers, let  $M\in \mathcal{M}_K(\mathbb{R})$  be a matrix such that
$$
\max  _ {i, j \in [ K ]} \left| M _ {i, j} - s I _ {K} \right| \leqslant \alpha .
$$
Then for any  $\varepsilon$ -Nash equilibrium  $(p,q)$  of  $M$  we have for any  $i,j\in [K]$
$$
\min  \left(p (i), q (j)\right) \geqslant \frac {1}{K} - \frac {2 (\alpha + \varepsilon) (K - 1)}{s}.
$$
Proof. Let  $(p,q)$  denote an  $\varepsilon$ -NE of  $M$ . Then
$$
\max  _ {j \in [ K ]} (M ^ {\intercal} p) _ {j} - \min  _ {i \in [ K ]} (M q) _ {i} \leqslant 2 \varepsilon . \tag {4}
$$
Now for any  $j\in [K]$ , since  $M$  is close to  $sI_{K}$
$$
(M ^ {\intercal} p) _ {j} \geqslant s p (j) - \alpha
$$
thus, using the fact that  $p$  is a probability vector,
$$
\max  _ {j \in [ K ]} (M ^ {\intercal} p) _ {j} \geqslant s \max  _ {j \in [ K ]} p (j) - \alpha \geqslant \frac {s}{K} - \alpha . \tag {5}
$$
$$
\min  _ {i \in [ K ]} (M q) _ {i} \leqslant s \min  _ {i \in [ K ]} q (i) + \alpha \tag {6}
$$
Combining the equations (4), (5) and (6) above, we have
$$
\frac {s}{K} - \alpha - \varepsilon \leqslant s \min  _ {i \in [ K ]} q (i) + \alpha + \varepsilon ,
$$
i.e., after rearranging,
$$
\min  _ {i \in [ K ]} q (i) \geqslant \frac {1}{K} - \frac {2 (\alpha + \varepsilon)}{s}.
$$
$$
\max  _ {j \in [ K ]} p (j) \leqslant \frac {1}{s} \max  _ {j \in [ K ]} \left(M ^ {\intercal} p\right) _ {j} + \frac {\alpha}{s} \leqslant \min  _ {i \in [ K ]} q (i) + 2 \frac {\alpha + \varepsilon}{s} \leqslant \frac {1}{K} + \frac {2 (\alpha + \varepsilon)}{s}
$$
$$
\begin{array}{l} \min  _ {j \in [ K ]} p (k) \geqslant 1 - (K - 1) \max  _ {j \in [ K ]} p (j) \geqslant 1 - \frac {K - 1}{K} - (K - 1) \frac {2 (\alpha + \varepsilon)}{s} \\ = \frac {1}{K} - (K - 1) \frac {2 (\alpha + \varepsilon)}{s}. \\ \end{array}
$$
Completing the proof.
Corollary 21. For any  $\varepsilon \leqslant 1 / (16K^2)$ , for any game matrix  $M$  such that
$$
\max  _ {i, j \in [ K ]} \left| M _ {i, j} - \frac {1}{2} I _ {K} \right| \leqslant \frac {1}{1 6 K ^ {2}}
$$
all  $\varepsilon$ -NE  $(p,q)$  of  $M$  satisfy for all  $i,j\in [K]$
$$
\min  \left(p (i), q (j)\right) \geqslant \frac {1}{2 K}.
$$
Corollary 22. For any game matrix  $M$  such that
all exact Nash equilibria  $(p,q)$  are fully supported.