SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof of Lemma C.2. Before we delve into the specifics of the proof, we introduce some notation. In order to make sure that  $\mathbf{h}_{\phi}$  is such that  $\| \mathbf{h} - \mathbf{h}_{\phi}\| = R$ , we do the following:  $\mathbf{h}_{\phi}\gets \mathbf{h} + RS_{\phi}$ , where  $\mathbf{S}_{\phi}$  is sampled uniformly at random from the equator  $\mathbb{S}\cap \mathbb{H}$ , where  $\mathbb{S} = \{\mathbf{s}\in \mathbb{R}^m:\| \mathbf{s}\| _2^2 = 1\}$  is the unit sphere and  $\mathbb{H} = \{\mathbf{s}\in \mathbb{R}^m:\langle \mathbf{s},\mathbf{1}\rangle = 0\}$  is an equatorial hyperplane ( $\mathbf{1}\triangleq (1,\dots ,1)\in \mathbb{R}^{m}$ ). Note that this is because we want  $\mathbf{h}_{\phi}$  to remain a valid probability distribution, i.e., that  $\langle \mathbf{h}_{\phi},\mathbf{1}\rangle = 1$  and  $\mathbf{h}_{\phi}\geq 0$  coordinate-wise; indeed, since we already have  $\langle \mathbf{h},\mathbf{1}\rangle = 1$ , we need to make sure that (1)  $\langle \mathbf{S}_{\phi},\mathbf{1}\rangle = 0$ , which is guaranteed by  $\mathbf{S}_{\phi}\in \mathbb{H}$ ; (2)  $\mathbf{h}_{\phi}\geq 0$ , which is guaranteed by returning FALSE whenever  $\mathbf{h}_{\phi}\notin \mathcal{H}_P$ .
For the rest of the proof, we condition on the following success event:
$$
\mathcal {E} \triangleq \left\{\forall [ s, t ] \subseteq [ 1, T ], \forall i \in \mathcal {A} _ {A}, \operatorname {C a l E r r} _ {i} \left(\mathbf {h} _ {s: t}, \mathbf {p} _ {s: t}\right) \leq r _ {\delta} (t - s) \right\}.
$$
Since the agent is  $(\varepsilon, \Pi)$ -adaptively calibrated with rate  $r_{\delta}$ , we have  $\operatorname{Pr}(\mathcal{E}) \geq 1 - \delta$ .
Recall that  $\varepsilon_{\mathrm{cal}} = k\sqrt{mr_{\delta}(l)}$  denote the error that comes from calibration error. We first show that conditioned on  $\mathcal{E},\forall \phi \in [\Phi ],\mathbf{h}_{\phi}\in B_{2}(P_{y_{\phi}},\varepsilon_{\mathrm{cal}})$ . Let  $l_{y_{\phi}}$  be the number of times that agent plays  $y_{\phi}$  during the  $l$  repeats, then we have  $l_{y_{\phi}}\geq l / k$  because  $y_{\phi}$  is the most frequently played action. Then, the calibration error bound in Definition 2.3 guarantees that
$$
\begin{array}{l} \frac {l _ {y _ {\phi}}}{l} \left\| \bar {\mathbf {p}} \left(y _ {\phi}\right) - \mathbf {h} _ {\phi} \right\| _ {\infty} = \operatorname {C a l E r r} _ {y _ {\phi}} \left(\mathbf {h} _ {\phi , 1: l}, \mathbf {p} _ {\phi , 1: l}\right) \leq r _ {\delta} (l) + \varepsilon \\ \Rightarrow \| \bar {\mathbf {p}} (y _ {\phi}) - \mathbf {h} _ {\phi} \| _ {2} \leq \sqrt {m} \| \bar {\mathbf {p}} (y _ {\phi}) - \mathbf {h} _ {\phi} \| _ {\infty} \leq \sqrt {m} k r _ {\delta} (l) = \varepsilon_ {\text {c a l}}. \tag {13} \\ \end{array}
$$
where the first inequality in equation (13) is because of the norm property  $\| x\| _2\leq \sqrt{d}\| x\|_\infty$  for a vector  $x\in \mathbb{R}^d$ . Since  $\bar{\mathbf{p}} (y_{\phi})\in P_{y_{\phi}}$  because the agent always best responds to forecasts, we obtain  $\mathbf{h}_{\phi}\in B_2(P_{y_\phi},\varepsilon_{\mathrm{cal}})$ .
We then prove the following two claims:
(C1) If  $\mathbf{h} \in B_2(P_y^{-\varepsilon_2}, - \varepsilon_1)$ , then APPROXMEM returns TRUE.
(C2) If  $\mathbf{h} \notin B_2(P_y^{-\varepsilon_2}, + \varepsilon_1)$ , then APPROXMEM returns FALSE with probability  $\geq 1 - (\varepsilon_3 - \delta)$ .
Indeed, if the following two claims hold, then we have established that APPROXMEM asserts one of two cases correctly with probability  $\geq 1 - (\varepsilon_3 - \delta)$  conditioned on  $\mathcal{E}$ . Together with the fact that  $\operatorname*{Pr}(\mathcal{E})\geq 1 - \delta$ , proves the lemma.
Proof of (C1). Suppose  $\mathbf{h} \in B_2(P_y^{-\varepsilon_2}, -\varepsilon_1)$ . Then we know that  $B_2(\mathbf{h}, \varepsilon_1) \subseteq P_y^{-\varepsilon_2} = B_2(P_y, -\varepsilon_2)$ , which further implies  $B_2(\mathbf{h}, \varepsilon_1 + \varepsilon_2) \subseteq P_y$ . Therefore, the distance between  $\mathbf{h}$  and any other polytope  $P_{y'}$  ( $y' \neq y$ ) must be lower bounded by  $\varepsilon_1 + \varepsilon_2$ . By triangle inequality,  $\forall \mathbf{h}' \in P_{y'}$  where  $y' \neq y$ , we have
$$
\forall \phi \in [ \Phi ], \quad \| \mathbf {h} _ {\phi} - \mathbf {h} ^ {\prime} \| _ {2} \geq \| \mathbf {h} - \mathbf {h} ^ {\prime} \| _ {2} - \| \mathbf {h} - \mathbf {h} _ {\phi} \| _ {2} \geq \varepsilon_ {1} + \varepsilon_ {2} - R \geq \varepsilon_ {\mathrm {c a l}},
$$
where the last step follows from (Condition 1).
Since this holds for all  $\mathbf{h}' \in P_{y'}$ , it implies  $\mathbf{h}_{\phi} \notin B_2(P_{y'}, \varepsilon_{\mathrm{cal}})$  whenever  $y' \neq y$ . Together with the fact that  $\mathbf{h}_{\phi} \in B_2(P_{y_{\phi}}, \varepsilon_{\mathrm{cal}})$ , we must have  $y_{\phi} = y$  for all epochs  $\phi \in [\Phi]$ . Therefore, APPROXMEM always returns TRUE.
Proof of (C2). Suppose  $\mathbf{h} \notin B_2(P_y^{-\varepsilon_2}, + \varepsilon_1)$ . We first analyze the probability of returning FALSE for a fixed epoch  $\phi \in [\Phi]$ . Let  $\partial S$  be the boundary of a convex set  $S$ , then by triangle inequality, we have
$$
d (\mathbf {h}, \partial B _ {2} (P _ {y}, \varepsilon_ {\mathrm {c a l}})) \leq d (\mathbf {h}, \partial P _ {y}) + d (\partial P _ {y}, \partial B _ {2} (P _ {y}, \varepsilon_ {\mathrm {c a l}})).
$$
For the first term, we have  $d(\mathbf{h},\partial P_y)\leq \varepsilon_2$  since
$$
\mathbf {h} \notin B _ {2} \left(P _ {y} ^ {- \varepsilon_ {2}}, + \varepsilon_ {1}\right) \Rightarrow h \notin P _ {y} ^ {- \varepsilon_ {2}} = B _ {2} \left(P _ {y}, - \varepsilon_ {2}\right) \Rightarrow d (\mathbf {h}, P _ {y}) \leq \varepsilon_ {2}.
$$
For the second term, we have
$$
d \left(\partial P _ {y}, \partial B _ {2} \left(P _ {y}, \varepsilon_ {\mathrm {c a l}}\right)\right) \leq \varepsilon_ {\mathrm {c a l}},
$$
because  $P_{y}$  is a convex set. Combining the two bounds, we know that  $d(\mathbf{h},\partial B_2(P_y,\varepsilon_{\mathrm{cal}}))\leq \varepsilon_{\mathrm{cal}} + \varepsilon_2$
Since  $\mathbf{h}_{\phi}$  is uniformly sampled from the sphere of radius  $R$  around  $\mathbf{h}$ , by convexity of  $B_{2}(P_{y},\varepsilon_{\mathrm{cal}})$  and the rotation invariance property of a unit sphere, we have
$$
\Pr \left[ \mathbf {h} _ {\phi} \notin B _ {2} \left(P _ {y}, \varepsilon_ {\text {c a l}}\right) \right] \geq \Pr \left[ \langle R \mathbf {S} _ {\phi}, \mathbf {v} \rangle \geq \varepsilon_ {\text {c a l}} + \varepsilon_ {2} \right] = \Pr \left[ \langle \mathbf {S} _ {\phi}, \mathbf {e} _ {1} \rangle \geq \varepsilon_ {\text {c a l}} + \varepsilon_ {2} / R \right],
$$
where  $\mathbf{v}$  is a unit vector pointing in the direction of (one of) the projection from  $\mathbf{h}$  to  $\partial B_2(P_y, \varepsilon_{\mathrm{cal}})$ , and  $\mathbf{e}_1 = (1, 0, \dots, 0) \in \mathbb{R}^m$ . According to [23, Lemma 9], we can further lower bound the probability by
$$
\Pr [ \mathbf {h} _ {\phi} \notin B _ {2} (P _ {y}, \varepsilon_ {\mathrm {c a l}}) ] \geq \frac {1}{2 \sqrt {m}} \left(1 - \left(\frac {\varepsilon_ {\mathrm {c a l}} + \varepsilon_ {2}}{R}\right) ^ {2}\right) ^ {\frac {m - 1}{2}}.
$$
Finally, the probability that no epoch returns FALSE (failure of APPROXMEM) is at most
$$
\left(1 - \delta^ {\prime}\right) ^ {\Phi} \leq \varepsilon_ {3} - \delta , \text {w h e r e} \delta^ {\prime} = \frac {1}{2 \sqrt {m}} \left(1 - \left(\frac {\varepsilon_ {\mathrm {c a l}} + \varepsilon_ {2}}{R}\right) ^ {2}\right) ^ {\frac {m - 1}{2}}.
$$
Since  $\delta' \ll 1$ , the above inequality holds as long as
$$
\Phi \geq \frac {2}{\delta^ {\prime}} \log \left(\frac {1}{\varepsilon_ {3} - \delta}\right) = 4 \sqrt {m} \left(1 - \left(\frac {\varepsilon_ {\mathrm {c a l}} + \varepsilon_ {2}}{R}\right) ^ {2}\right) ^ {\frac {m - 1}{2}} \log \left(\frac {1}{\varepsilon_ {3} - \delta}\right),
$$
which follows from (Condition 2). Therefore,  $\mathbf{h} \notin B_2(P_y^{-\varepsilon_2}, + \varepsilon_1)$ , then APPROXMEM returns FALSE with probability  $\geq 1 - (\varepsilon_3 - \delta)$ .
C.3 Complexity of the explore algorithm
Lemma C.4 (Sample complexity of Algorithm 2). Using Algorithm 2, the principal can find with probability at least  $1 - (\delta + \varepsilon_{opt} + T^{-1})$ , a strategy  $\tilde{\mathbf{h}} \in \mathcal{H}_P$  that satisfies the two properties in Lemma C.5. We refer the reader to Theorems C.8 and C.13 for the sample complexity under two different instantiations of Algorithm 2.
Lemma C.5 (Optimality and robustness of output policy). Under the conditions of Lemma C.4, the output  $\hat{\mathbf{h}}$  of Algorithm 2 satisfies the following two properties:
(P1)  $U_P(\tilde{\mathbf{h}}, y^{\star}) \geq V^{\star} - \varepsilon_{opt}$ , i.e.,  $\tilde{\mathbf{h}}$  is an approximate Stackelberg equilibrium.
(P2)  $\tilde{\mathbf{h}}\in B_2(P_{y^{\star}}, - \varepsilon_{\text{robust}})$ , i.e.,  $\tilde{\mathbf{h}}$  lies robustly within the best-response polytope for  $y^{\star}$ .
We refer the reader to Theorems C.8 and C.13 for the exact values of  $\varepsilon_{\text{robust}}, \varepsilon_{\text{opt}}$  under two different instantiations of Algorithm 2.
Before we formally state the proof of Lemmas C.4 and C.5, we first show the sample complexity guarantee of the exploration phase of Algorithm 2 (Lemma C.6), then state a useful lemma from prior work on convex optimization from membership queries [42] (Lemma C.7).
Lemma C.6 (Initialization set of Algorithm 2). Let  $V$  be the volume of an  $\ell_2$  ball of radius  $\frac{\eta}{2}$  in  $\mathbb{R}^m$ . Suppose the principal samples  $\mathbf{h}$  uniformly from  $\mathcal{H}_P$  for  $O(V^{-1}\log T)$  times and calls  $\mathrm{APPROXMEM}(\varepsilon_1 = \varepsilon_2 = \eta /4, \varepsilon_3 = \delta +T^{-2} / 2)$  for the membership of each of them, then with probability at least  $1 - \delta -T^{-1}$ , the initialization set  $\mathcal{I}$  contains  $(\mathbf{h}_0,y_0)$  where  $y_{0} = y^{\star}$  is the optimal target, and  $\mathbf{h}_0$  is  $\frac{\eta}{2}$ -centered in  $P_{y^{\star}}$ , i.e.,  $\mathbf{h}_0\in B_2(P_{y^{\star}}, - \frac{\eta}{2})$ .
The total number of samples required for the initialization phase is  $O\left(V^{-1} 1.25^{m} r_{\delta}^{-1}\left(\frac{\eta}{k \sqrt{m}}\right) \log^{2} T\right)$ .
Proof of Lemma C.6. By regularity assumption, there exists  $\mathbf{h} \in P_{y^*}$ , s.t.  $B_2(\mathbf{h}, \eta) \in P_{y^*}$ . Therefore,  $\forall \mathbf{h}' \in B_2(\dot{\mathbf{h}}, \frac{\eta}{2})$ , we have  $\dot{\mathbf{h}} \in B_2(P_{y^*}, -\frac{\eta}{2})$ . Moreover, since  $\mathbf{h}'$  lies robustly inside  $P_{y^*}$ , on the success event of  $\mathrm{APPROXMEM}(\varepsilon_1 = \varepsilon_2 = \frac{\eta}{4})$ ,  $\mathrm{APPROXMEM}(\mathbf{h}')$  must return membership  $y^*$ , otherwise we must have  $\mathbf{h}' \notin B_2(P_{y^*}^{-\frac{\eta}{4}}, -\frac{\eta}{4}) = B_2(P_{y^*}, -\frac{\eta}{2})$ , a contradiction. Since the set of all such  $\mathbf{h}'$  takes up nontrivial volume  $V$  in  $\mathcal{H}_P$ , we know that  $O(V^{-1} \log T)$  uniform samples are guaranteed to hit one with probability  $1 - \frac{1}{(2T)}$ .
Now we upper bound the failure probability of the initialization phase. Since the agent is miscalibrated with probability  $\delta$ , random sampling fails to discover centered  $\mathbf{h}'$  with probability  $\frac{1}{(2T)}$ , and the
probability that one of the answers from APPROXMEM is wrong with probability  $\frac{1}{(2T)}$ , the total failure probability is  $1 - \delta - T^{-1}$  as desired.
As for the sample complexity, note that according to Lemma C.2 and proposition C.3, each call to APPROXMEM takes  $O\left(1.25^{m}r_{\delta}^{-1}\left(\frac{\eta}{k\sqrt{m}}\right)\log T \right)$  samples, and the initialization phase calls APPROXMEM for  $O(V^{-1}\log T)$  times, the total sample complexity is  $O\left(V^{-1}1.25^{m}r_{\delta}^{-1}\left(\frac{\eta}{k\sqrt{m}}\right)\log^{2}T\right)$ .
Lemma C.7 (LSV performance guarantee, Theorems 14 and 15 in [42]). Let  $K$  be a convex set specified by a membership oracle, a point  $x_0 \in \mathbb{R}^n$ , and  $\eta > 0$  such that  $B_{2}(x_{0},\eta /2) \subseteq K \subseteq B_{2}(x_{0},1)$ . There exists a universal constant  $\gamma_0 > 1$  such that for any convex function  $f$  given by an evaluation oracle and any  $\varepsilon' > 0$ , there is a randomized algorithm that computes a point  $z \in B_{2}(K,\varepsilon')$  such that with probability  $\geq 1 - \varepsilon'$ ,
$$
f (z) \leq \min  _ {x \in K} f (x) + \varepsilon^ {\prime} \cdot \left(\max  _ {x \in K} f (x) - \min  _ {x \in K} f (x)\right),
$$
with constant probability using  $O\left(n^{2}\log^{O(1)}(n / \varepsilon^{\prime}\eta)\right)$  calls to the  $\varepsilon_{1}$ -approximate membership oracle and evaluation oracle, where  $\varepsilon_{1} = (\varepsilon^{\prime}\eta /n)^{\gamma_{0}}$ .
Note that in Lemma C.7, the required accuracy  $(\varepsilon_{1})$  of the approximate membership oracle is orders of magnitudes smaller than the suboptimality  $(\varepsilon)$  of the output solution, i.e.,  $\varepsilon_{1} \ll \varepsilon$ . Let  $\tilde{\mathbf{h}}$  denote the  $\varepsilon$ -optimal solution returned by LSV. Since we use APPROXMEM which is an  $\varepsilon_{1}$ -approximate membership oracle to  $K = P_{y}^{-\varepsilon_{2}}$ ,  $\tilde{\mathbf{h}}$  will lie in  $B_{2}(K,\varepsilon) = B_{2}(P_{y}^{-\varepsilon_{2}},\varepsilon) = B_{2}(P_{y}, -\varepsilon_{2} + \varepsilon)$ . In order to find a near-optimal solution that lies robustly within  $P_{y}$ , we can either simulate APPROXMEM with more epochs to guarantee  $\varepsilon_{1} \ll \varepsilon \approx \varepsilon_{2}$  (see Appendix C.4 for more details), or perform a post-process to  $\tilde{\mathbf{h}}$  that pushes it further inside the polytope (see Appendix C.5 for more details).
C.4 More epochs to guarantee robustness
In this section, we show how to use more epochs in the simulation of each APPROXMEM oracle to obtain higher accuracy feedbacks. This approach directly guarantees that the output of LSV lies robustly within the polytope.
Theorem C.8. If the principal uses Algorithm 2 with parameters  $\mathrm{APPROXMEM}(\varepsilon_1 = (\frac{\varepsilon' \eta}{n})^{\gamma_0}, \varepsilon_2 = 2\varepsilon', \varepsilon_3 = \delta + T^{-2})$  and  $\mathrm{LSV}(\varepsilon')$ , then with probability at least  $1 - \delta - T^{-1} - \varepsilon'$ , the final solution  $\tilde{\mathbf{h}}$  satisfies the following two properties:
(P1)  $U_P(\tilde{\mathbf{h}}, y^{\star}) \geq V^{\star} - 3\varepsilon'$ , i.e.,  $\tilde{\mathbf{h}}$  is an approximate Stackelberg equilibrium.
(P2)  $\tilde{\mathbf{h}}\in B_2(P_{y^\star}, - \varepsilon ')$ , i.e.,  $\tilde{\mathbf{h}}$  lies robustly within the best-response polytope for  $y^\star$ .
The total number of samples needed is  $\tilde{O}\left(V^{-1}m^{\frac{5 + m\gamma_0}{2}}(\eta \varepsilon')^{-\frac{m\gamma_0}{2}}r_{\delta}^{-1}\left(\frac{(\varepsilon'\eta)^{\gamma_0}}{km^{\gamma_0}\sqrt{m}}\right)\right)$ .
Proof. We first analyze the failure probability of the algorithm. Since the agent's adaptive calibration error is uniformly bounded by  $r_{\delta}(\cdot)$  with probability  $1 - \delta$ , the probability that there exists an incorrect answer from APPROXMEM is bounded by  $T \cdot T^{-2} = T^{-1}$  conditioned on agent's calibration error being bounded, and the probability that LSV returns a bad solution is bounded by  $\varepsilon'$ , the total failure probability of the described algorithm is at most  $\delta + T^{-1} + \varepsilon'$ .
Now we consider the optimization problem for the optimal polytope  $P_{y^{\star}}$ , with initial point  $\mathbf{h}_0$  that is  $\frac{\eta}{2}$ -centered in  $P_{y^{\star}}$ . There exists such a pair  $(\mathbf{h}_0, y^{\star}) \in \mathcal{I}$  according to Lemma C.6. Let  $\tilde{\mathbf{h}}$  be the solution output by LSV. According to Lemma C.7,  $\tilde{\mathbf{h}} \in B_2(P_{y^{\star}}^{-\varepsilon_2}, +\varepsilon') \subseteq B_2(P_{y^{\star}}, -\varepsilon')$ , which proves (P2). For (P1), note that Lemma C.7 also guarantees
$$
V ^ {\star} - U _ {P} (\tilde {\mathbf {h}}, y ^ {\star}) \leq \varepsilon^ {\prime} + \left(V ^ {\star} - \max  _ {\mathbf {h} \in P _ {y ^ {\star}} ^ {- \varepsilon_ {2}}} U _ {P} (\mathbf {h}, y ^ {\star})\right) \leq \varepsilon^ {\prime} + 2 \varepsilon^ {\prime} = 3 \varepsilon^ {\prime}.
$$
Finally, we compute the total number of samples. By Lemma C.6, the initialization epoch takes  $O\left(V^{-1}1.25^{m}r_{\delta}^{-1}\left(\frac{\eta}{k\sqrt{m}}\right)\log^{2}T\right)$  samples. According to Lemma C.2 and proposition C.3, each call to the APPROXMEM oracle requires  $O\left(\sqrt{m}\left(\frac{\varepsilon_{2}}{\varepsilon_{1}}\right)^{\frac{m}{2}}r_{\delta}^{-1}\left(\frac{\varepsilon_{1}}{k\sqrt{m}}\right)\log(T)\right) = O\left(\sqrt{m}\left(\frac{m}{\eta\varepsilon^{\prime}}\right)^{\frac{m\gamma_{0}}{2}}r_{\delta}^{-1}\left(\frac{(\varepsilon^{\prime}\eta)^{\gamma_{0}}}{km^{\gamma_{0}}\sqrt{m}}\right)\log(T)\right)$  samples, and Lemma C.7 suggests that there LSV makes  $O\left(m^{2}\log^{O(1)}\left(\frac{m}{\varepsilon^{\prime}\eta}\right)\right)$  oracle calls, where we run  $O(V^{-1}\log T)$  LSV instances for each initial point in the initialization set. Putting together, the total number of samples is
$$
\tilde {O} \left(V ^ {- 1} m ^ {\frac {5 + m \gamma_ {0}}{2}} \left(\eta \varepsilon^ {\prime}\right) ^ {- \frac {m \gamma_ {0}}{2}} r _ {\delta} ^ {- 1} \left(\frac {\left(\varepsilon^ {\prime} \eta\right) ^ {\gamma_ {0}}}{k m ^ {\gamma_ {0}} \sqrt {m}}\right)\right),
$$
where  $\tilde{O}$  hides logarithm terms in  $T, m, 1/\eta, 1/\varepsilon'$ .
C.5 Representation length and post-processing
In this section, we show a post-processing of the output of LSV [42] under the representation length assumption.
Assumption C.9 (Utility Representation Length). Suppose the utility functions  $U_{P}$  and  $U_{A}$  are rational with denominator at most  $a$ , and normalized to be [0,1]. Therefore, the game's utility representation length is  $L = 2mn\log a$ .
The next lemma shows that under Assumption C.9, the optimal solution  $\mathbf{h}^{\star}$  also has a finite representation length.
Lemma C.10. For any  $y \in \mathcal{A}_A$ , let  $\mathbf{h}_y^\star \in \mathbb{R}^m$  be the principal's optimal strategy in  $P_y$  that achieves  $\max_{\mathbf{h}^\star \in P_y} U_P(\mathbf{h}^\star, y)$ . Suppose the utility functions  $U_P, U_A$  satisfy Assumption C.9. Then for all  $i \in \mathcal{A}_P$ , the  $i$ -th coordinate of  $\mathbf{h}_y^\star$  is a rational number with denominator at most  $2^{2mL}$ .
Proof. The proof of this lemma is similar to [12, Lemma 10], which follows from well-known results in linear programming. We first note that  $\mathbf{h}^{\star}$  is the solution of the linear programming:
$$
\max  _ {\mathbf {h}} U _ {P} (\mathbf {h}, y), \quad \text {s u b j e c t t o} \mathbf {h} ^ {\star} \in P _ {y},
$$
where  $U_P(\mathbf{h}, y)$  is linear in  $\mathbf{h}$  and the best response polytope  $P_y$  can be represented as a system  $\{\mathbf{h}: A\mathbf{h}^T \succeq \mathbf{b}\}$ , where the set of constraints are the ones that define probability simplex  $\mathcal{H}_P$ , together with the constraints of the form  $U_P(\mathbf{h}, y) - U_P(\mathbf{h}, y') \geq 0, \forall y' \neq y$ . Suppose  $A$  is normalized so that each entry is an integer. By Assumption C.9, each coefficient is at most  $2^L$ . Since the solution to the above LP is the intersection of  $m$  independent constraints of  $A$  (denoted with  $D$ ), we have  $\mathbf{h}_{y,i}^* = \frac{\operatorname*{det}(D_i)}{\operatorname*{det}(D)}$  by Cramer's rule, where  $D_i$  is  $D$  with the  $i$ -th column replaced by  $\mathbf{b}$ . According to Hardamard's inequality,
$$
\det  (D) \leq \prod_ {i \in [ m ]} \sqrt {\prod_ {j \in [ m ]} d _ {i j} ^ {2}} \leq \prod_ {i \in [ m ]} \sqrt {m} 2 ^ {L} = m ^ {\frac {m}{2}} 2 ^ {L m} \leq 2 ^ {2 L m}.
$$
We also make the following assumption on the optimal strategy  $\mathbf{h}^{\star}$ : any near-optimal strategy  $\mathbf{h}$  must lie in the neighborhood of  $\mathbf{h}^{\star}$ .
Assumption C.11 (Near-optimal strategies). There exists a constant  $c$  s.t. for all  $\varepsilon \leq 2^{-4mL}$  and all strategy  $\tilde{\mathbf{h}}$  such that  $U_{P}(\tilde{\mathbf{h}}.y^{\star}) \leq V^{\star} - \varepsilon$ , we have  $\| \tilde{\mathbf{h}} - \mathbf{h}^{\star} \|_{\infty} \leq c2^{L} \varepsilon$ .
Now we are ready to define the post-processing algorithm.
Lemma C.12 (Post-processing). Suppose that Assumptions C.9 and C.11 hold. Assume  $\varepsilon \leq 2^{-4mL}$ ,  $\tilde{\mathbf{h}}$  is an  $\varepsilon$ -approximate optimal strategy in  $P_{\tilde{y}}$ , and  $\mathbf{h}_0$  is  $\eta/2$ -centered in  $P_{\tilde{y}}$ , i.e.,  $B_2(\mathbf{h}_0, \eta/2) \subseteq P_{\tilde{y}}$ , then using no more than  $O\left(2^{2m} r_{\delta}^{-1} \left(\frac{\lambda}{k \sqrt{m}}\right) \log T\right)$  rounds of interactions with the agent, POSTPROCESS (Algorithm 5) outputs a strategy  $\mathbf{h}$  that satisfies with probability  $\geq 1 - \delta - T^{-1}$ :
Algorithm 5: Post-Processing (POSTPROCESS)
Input: non-robust strategy  $(\tilde{\mathbf{h}},\tilde{y})$  s.t.  $\tilde{\mathbf{h}}\in B_2(P_{\tilde{y}},\varepsilon)$  , robustness parameter  $\lambda$  , initial point  $\mathbf{h}_0$    
Output: robust strategy h   
 $S$  -set of strategies  $\mathbf{h}^{\prime}\in \mathcal{H}_{P}$  s.t. each coordinate  $\mathbf{h}_i^\prime$  is a rational number with denominator at most  $2^{2mL}$  and  $|\tilde{\mathbf{h}}_i - \mathbf{h}_i|\leq c2^L\varepsilon$    
for  $\mathbf{h}^j\in S$  do   
 $\mathbf{h}_{\mathrm{query}}^j\gets \left(1 - \frac{2\lambda}{\eta}\right)\mathbf{h}^j +\frac{2\lambda}{\eta}\mathbf{h}_0$    
Query APPROXMEM  $\tilde{\mathbf{y}} (\mathbf{h}_{\mathrm{query}}^j)$  with  $\varepsilon_{1} = \varepsilon_{2} = \frac{\lambda}{2},\varepsilon_{3} = \delta +T^{-2}$    
if APPROXMEM returns TRUE then   
 $\mathbf{h}\gets \left(1 - \frac{2\lambda}{\eta}\right)\mathbf{h}_{\mathrm{query}}^j +\frac{2\lambda}{\eta}\mathbf{h}_0$    
RETURN h
(i)  $\mathbf{h}$  lies robustly inside  $P_{\tilde{y}}\colon \mathbf{h}\in B_2(P_{\tilde{y}}, - \lambda)$
(ii)  $\mathbf{h}$  is close to  $\tilde{\mathbf{h}}$ :  $\| \mathbf{h} - \tilde{\mathbf{h}} \|_2 \leq \sqrt{m} \left( c 2^L \varepsilon + \frac{4\lambda}{\eta} \right)$ .
Proof. To prove the lemma, we make the following claim: For all  $\mathbf{h}_1 \in \mathcal{H}_P$ , if  $\mathbf{h}_1 \in P_{\tilde{y}}$  and  $\mathbf{h}_2 = \left(1 - \frac{2\lambda}{\eta}\right)\mathbf{h}_1 + \frac{2\lambda}{\eta}\mathbf{h}_0$ , then we have  $\mathbf{h}_2 \in B_2(P_{\tilde{y}}, -\lambda)$ . In fact, for all unit vectors  $\mathbf{s} \in \mathbb{R}^m$ , we have
$$
\mathbf {h} _ {2} + \lambda \mathbf {s} = \left(1 - \frac {2 \lambda}{\eta}\right) \mathbf {h} _ {1} + \frac {2 \lambda}{\eta} \left(\mathbf {h} _ {0} + \frac {\eta}{2} \mathbf {s}\right).
$$
Since  $\mathbf{h}_1 \in P_{\tilde{y}}$  and  $\mathbf{h}_0 + \frac{\eta}{2}\mathbf{s} \in B_2(\mathbf{h}_0, \eta/2) \subseteq P_{\tilde{y}}$  from the assumption that  $\mathbf{h}_0$  is  $\eta/2$ -centered, their convex combination must also lie in  $P_{\tilde{y}}$  because  $P_{\tilde{y}}$  is convex. Thus, we have  $\mathbf{h}_2 + \lambda \mathbf{s} \in P_{\tilde{y}}$ . Since the above holds for all unit vectors  $\mathbf{s}$ , we conclude that  $B_2(\mathbf{h}_2, \lambda) \subseteq P_{\tilde{y}}$ . We have thus proved the claim.
With the above claim, we first show that when all oracle calls return correctly,  $S$  contains at least one point  $\mathbf{h}^j$  such that APPROXMEM returns TRUE on  $\mathbf{h}_{\mathrm{query}}^j$ , and the post-processed version of this strategy satisfies (i) and (ii).