SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Theorem 5.2 (Restated). For continuous CSGs satisfying Assumption 5.1, for all  $\varepsilon_0 > 0$ , there exists a finite binning  $\Pi_0$  such that if the agent is  $(0,\Pi_0)$  - adaptively calibrated and the principal runs an appropriately parametrized instance of LAZYGDwOG (Algorithm 3) then:
$$
\lim_{\substack{\Phi \to \infty \\ M\to \infty}}\frac{1}{\Phi M}\sum_{\phi \in [\Phi ]}\sum_{i\in [M]}U_{P}(\mathbf{h}_{\phi},y_{\phi ,i})\geq V^{\star} - \varepsilon_{0}.
$$
Moreover, for any sequence of the principal's actions  $\mathbf{h}_{[1:T]}$ , it holds that:
$$
\lim  _ {T \rightarrow \infty} \frac {1}{T} \sum_ {t \in [ T ]} U _ {P} \left(\mathbf {h} _ {t}, y _ {t}\right) \leq V ^ {\star} + \varepsilon_ {0}.
$$
E.1.1 Proof of Lower Bound
Proof. Before delving into the proof of the lower bound, we first introduce some notations. Let  $C(\mathbf{h}) \triangleq U_P(\mathbf{h}, \mathsf{BR}(\mathbf{h}))$ . Let  $V_{\delta}^{\star} \triangleq \max_{\mathbf{h} \in B_2(\mathcal{A}_P, -\delta)} C(\mathbf{h})$  be the optimal utility restricted in the smaller strategy set  $B_2(\mathcal{A}_P, -\delta)$ . We use  $\overline{y}_{\phi} \triangleq \frac{1}{M} \sum_{s \in [M]} y_{\phi,s}$  to denote the average feedback that LAZYGDwOG uses to update the strategies.
We first consider any fixed  $\varepsilon > 0$ . Combining the guarantees of Lemmas E.4 and E.5, we conclude that there exists a finite binning  $\Pi_0$  and  $M_{\varepsilon} < \infty$ , such that if the agent is  $(0,\Pi_0)$ -adaptively calibrated, then  $\forall M \geq M_{\varepsilon}$ , the following two inequalities are satisfied at the same time:
$$
\sup  _ {\phi \in [ \Phi ]} \left\| \bar {y} _ {\phi} - \operatorname {B R} \left(\mathbf {h} _ {\phi}\right) \right\| _ {2} \leq \varepsilon \quad (\text {b y})
$$
$$
\sup  _ {\phi \in \Phi} \frac {1}{M} \sum_ {s \in [ M ]} U _ {P} \left(\mathbf {h} _ {\phi}, y _ {\phi , s}\right) \geq C \left(\mathbf {h} _ {\phi}\right) - \varepsilon ; \quad (\text {b y}) \tag {22}
$$
Set the parameters according to  $\gamma_{\phi} = \gamma_0m^{-\frac{1}{2}}\phi^{-\frac{3}{4}}$  and  $\delta_{\phi}\equiv \delta = \delta_0m^{\frac{1}{2}}\Phi^{-\frac{1}{4}}$  in Algorithm 3, then similar arguments to [51, Theorem 3.1] guarantee that
$$
\begin{array}{l} V _ {\delta} ^ {\star} - \frac {1}{\Phi} \sum_ {\phi \in [ \Phi ]} \mathbb {E} \left[ C \left(\mathbf {h} _ {\phi}\right) \right] \leq \left(\frac {D _ {P} ^ {2}}{2 \gamma_ {0}} + \frac {2 W _ {P} ^ {2}}{\delta_ {0} ^ {2}}\right) \sqrt {m} \Phi^ {- \frac {1}{4}} + L _ {\mathrm {B R}} D _ {P} \frac {1}{\Phi} \sum_ {\phi \in [ \Phi ]} \| \bar {y} _ {\phi} - \mathrm {B R} \left(\mathbf {h} _ {\phi}\right) \| _ {2} \\ \stackrel {(a)} {\leq} \left(\frac {D _ {P} ^ {2}}{2 \gamma_ {0}} + \frac {2 W _ {P} ^ {2}}{\delta_ {0} ^ {2}}\right) \sqrt {m} \Phi^ {- \frac {1}{4}} + L _ {\mathbb {B R}} D _ {P} \cdot \varepsilon , \\ \end{array}
$$
where (a) is from Equation (21).
Now we upper bound the difference between  $V^{\star}$  and  $V_{\delta}^{\star} = \max_{\mathbf{h}\in B_2(\mathcal{A}_P, - \delta)}C(\mathbf{h})$  , then we have  $V^{\star} - V_{\delta}^{\star}\leq \max_{\mathbf{h}^{\star}\in \mathcal{A}_{P}}\min_{\mathbf{h}^{\prime}\in B_{2}(\mathcal{A}_{P}, - \delta)}C(\mathbf{h}^{\star}) - C(\mathbf{h}^{\prime})\leq L_{U}\max_{\mathbf{h}^{\star}\in \mathcal{A}_{P}}\min_{\mathbf{h}^{\prime}\in B_{2}(\mathcal{A}_{P}, - \delta)}\| \mathbf{h}^{\star} - \mathbf{h}^{\prime}\|_{2}\leq L_{U}\delta$
where the second inequality follows from Assumption 5.1 that  $C(\mathbf{h})$  is  $L_{U}$ -Lipschitz.
The next step is to upper bound the difference between the actual average utility and  $\frac{1}{\Phi}\sum_{\phi \in [\Phi]}\mathbb{E}[C(\mathbf{h}_{\phi})]$ . From Equation (22), we have
$$
\frac {1}{\Phi} \sum_ {\phi \in [ \Phi ]} \mathbb {E} [ C (\mathbf {h} _ {\phi}) ] - \frac {1}{\Phi M} \sum_ {\phi \in [ \Phi ]} \sum_ {i \in [ M ]} U _ {P} (\mathbf {h} _ {\phi}, y _ {\phi , i}) \leq \varepsilon .
$$
Finally, putting the above inequalities together, we obtain
$$
\begin{array}{l} V ^ {\star} - \frac {1}{\Phi M} \sum_ {\phi \in [ \Phi ]} \sum_ {i \in [ M ]} U _ {P} \left(\mathbf {h} _ {\phi}, y _ {\phi , i}\right) \\ \leq \left(V ^ {\star} - V _ {\delta} ^ {\star}\right) + \left(V _ {\delta} ^ {\star} - \frac {1}{\Phi} \sum_ {\phi \in [ \Phi ]} \mathbb {E} [ C (\mathbf {h} _ {\phi}) ]\right) + \left(\frac {1}{\Phi} \sum_ {\phi \in [ \Phi ]} \mathbb {E} [ C (\mathbf {h} _ {\phi}) ] - \frac {1}{\Phi M} \sum_ {\phi \in [ \Phi ]} \sum_ {i \in [ M ]} U _ {P} (\mathbf {h} _ {\phi}, y _ {\phi , i})\right) \\ \leq L _ {U} \delta_ {0} m ^ {\frac {1}{2}} \Phi^ {- \frac {1}{4}} + \left(\frac {D _ {P} ^ {2}}{2 \gamma_ {0}} + \frac {2 W _ {P} ^ {2}}{\delta_ {0} ^ {2}}\right) \sqrt {m} \Phi^ {- \frac {1}{4}} + L _ {\mathrm {B R}} D _ {P} \cdot \varepsilon + \varepsilon . \\ \end{array}
$$
Taking the limit of  $\Phi \to \infty$ , the above inequalities imply
$$
\lim_{\substack{\Phi \to \infty \\ M\to \infty}}\frac{1}{\Phi M}\sum_{\phi \in [\Phi ]}\sum_{i\in [M]}U_{P}(\mathbf{h}_{\phi},y_{\phi ,i})\geq V^{\star} - \varepsilon \left(L_{\mathbb{BR}}D_{P} + 1\right).
$$
Since the above arguments hold for all  $\varepsilon > 0$ , taking  $\varepsilon = \frac{\varepsilon_0}{L_{\mathrm{BR}}D_P + 1}$  proves the theorem.
E.1.2 Proof of Upper Bound
For a fixed  $\varepsilon > 0$ , let  $D_{\varepsilon} = \{x_1, \dots, x_I\}$  be an  $\varepsilon$ -grid of  $\mathcal{F}_P$  under  $\ell_2$  distance, and let  $\Pi_0$  be the continuous binning specified by Equation (26). We have:
$$
\begin{array}{l} \sum_ {t \in [ T ]} U _ {P} \left(\mathbf {h} _ {t}, y _ {t}\right) = \sum_ {i \in [ I ]} \sum_ {t \in [ T ]} w _ {i} \left(\mathbf {p} _ {t}\right) U _ {P} \left(\mathbf {h} _ {t}, \mathbb {R} \left(\mathbf {p} _ {t}\right)\right) \\ \stackrel {\mathrm {(a)}} {\leq} \sum_ {i \in [ I ]} \sum_ {t \in [ T ]} w _ {i} (\mathbf {p} _ {t}) \left(U _ {P} \left(\mathbf {h} _ {t}, \mathbb {B R} (x _ {i})\right) + L _ {2} \cdot L _ {\mathbb {B R}} \underbrace {\| \mathbf {p} _ {t} - x _ {i} \| _ {2}} _ {\leq 2 \varepsilon}\right) \\ \stackrel {(b)} {\leq} \sum_ {i \in [ I ]} \left(\sum_ {t \in [ T ]} w _ {i} (\mathbf {p} _ {t})\right) U _ {P} \left(\frac {\sum_ {t \in [ T ]} w _ {i} (\mathbf {p} _ {t}) \mathbf {h} _ {t}}{\sum_ {t \in [ T ]} w _ {i} (\mathbf {p} _ {t})}, \mathrm {B R} (x _ {i})\right) + 2 L _ {2} L _ {\mathrm {B R}} \varepsilon T \\ \stackrel {(c)} {=} \sum_ {i \in [ I ]} n _ {T} (i) U _ {P} (\overline {{\mathbf {h}}} _ {T} (i), \mathbb {B R} (x _ {i})) + 2 L _ {2} L _ {\mathbb {B R}} \varepsilon T \\ \stackrel {\text {(d)}} {\leq} \sum_ {i \in [ I ]} n _ {T} (i) \left(U _ {P} \left(\overline {{\mathbf {p}}} _ {T} (i), \mathbb {B R} \left(x _ {i}\right)\right) + L _ {1} \left\| \overline {{\mathbf {p}}} _ {T} (i) - \overline {{\mathbf {h}}} _ {T} (i) \right\| _ {2}\right) + 2 L _ {2} L _ {\mathbb {B R}} \varepsilon T \\ = \underbrace {\sum_ {i \in [ I ]} n _ {T} (i) U _ {P} \left(\overline {{\mathbf {p}}} _ {T} (i) , \mathbb {R} \left(x _ {i}\right)\right)} _ {(A)} + \underbrace {L _ {1} \sum_ {i \in [ I ]} n _ {T} (i) \left\| \overline {{\mathbf {p}}} _ {T} (i) - \overline {{\mathbf {h}}} _ {T} (i) \right\| _ {2}} _ {(B)} + 2 L _ {2} L _ {\mathbb {B R}} \varepsilon T \tag {23} \\ \end{array}
$$
In the above inequalities that lead to (23), step (a) is because  $U_{P}$  is  $L_{2}$ -Lipschitz in the second argument and  $\mathsf{BR}(\cdot)$  is  $L_{\mathsf{BR}}$ -Lipschitz, and the fact that  $w_{i}(\mathbf{p}_{t}) > 0$  only when  $\| \mathbf{p}_t - x_i\| _2 < 2\varepsilon$ . In step (b), we used Jensen's inequality because  $U_{P}$  in concave in the first argument. Step (c) follows from the definition of  $n_T(i)$  and  $\overline{\mathbf{h}}_T(i)$  in Definition 2.3. The last inequality (d) uses the fact that  $U_{P}$  is  $L_{1}$ -Lipschitz in the first argument to decompose  $U_{P}(\overline{\mathbf{p}}_{T}(i),\mathsf{BR}(x_{i}))$  into calibration error (i.e., term (B)) and  $U_{P}(\overline{\mathbf{p}}_{T}(i),\mathsf{BR}(x_{i}))$  where the strategy that the agent best responds to is close to the principal's strategy (i.e., term (A)).
We can further bound  $(A)$  and  $(B)$  in Equation (23) respectively as follows:
$$
(A) \leq \sum_ {i \in [ I ]} n _ {T} (i) \left(U _ {P} \left(x _ {i}, \mathbb {R} \left(x _ {i}\right)\right) + L _ {1} \| x _ {i} - \overline {{\mathbf {p}}} _ {T} (i) \| _ {2}\right) \leq V ^ {\star} T + L _ {1} (2 \varepsilon) T,
$$
$$
(B) \leq L _ {1} T \sum_ {i \in [ I ]} \operatorname {C a l E r r} _ {i} \left(\mathbf {h} _ {1: T}, \mathbf {p} _ {1: T}\right) \leq L _ {1} \left| D _ {\varepsilon} \right| r _ {\delta} (T) T \quad \mathrm {w . p .} \geq 1 - \delta .
$$
Therefore, putting the above bounds together, we obtain that with probability  $\geq 1 - \delta$
$$
\frac {1}{T} \sum_ {t \in [ T ]} U _ {P} (\mathbf {h} _ {t}, y _ {t}) \leq V ^ {\star} + (L _ {1} | D _ {\varepsilon} |) r _ {\delta} (T) + 2 (L _ {1} + L _ {2} L _ {\mathrm {B R}}) \varepsilon .
$$
Since the above derivation holds for any  $\varepsilon > 0$ , it suffices to take  $\varepsilon$  such that  $2(L_{1} + L_{2}L_{\mathrm{BR}})\varepsilon = \varepsilon_{0}$ . Finally, since  $|D_{\varepsilon}| < \infty$  and  $r_{\delta}(T) = o(1)$ , taking the limit of  $T \to \infty$  proves the upper bound:
E.2 Key lemma: asymptotically correct forecast
In this section, we state and prove the key lemma for establishing Theorem 5.2. Intuitively, this lemma states that for any strategy  $\mathbf{h} \in \mathcal{A}_P$ , as long as the principal repeatedly plays  $\mathbf{h}$  for enough rounds, the fraction of times where the agent's forecast is close to  $\mathbf{h}$  will converge to 1.
Lemma E.1. For any  $\varepsilon_0 > 0$ , there exists a finite binning  $\Pi_0$ , such that if the principal repeatedly plays any  $\mathbf{h} \in \mathcal{A}_P$  for  $M$  rounds and the agent's forecasts  $\mathbf{p}_{1:M}$  are  $(0, \Pi_0)$ -adaptively calibrated, then:
$$
\lim  _ {M \rightarrow \infty} \quad \frac {1}{M} \left|\left\{s \in [ M ]: \| \mathbf {p} _ {s} - \mathbf {h} \| _ {2} \geq \varepsilon_ {0} \right\}\right| = 0 \tag {24}
$$
In particular, if the calibration error (defined in Definition 2.3) has rate  $r(\cdot) \in o(1)$  with respect to  $\Pi_0$ , then
$$
\left. \frac {1}{M} \right| \left\{s \in [ M ]: \| \mathbf {p} _ {s} - \mathbf {h} \| _ {2} \geq \varepsilon_ {0} \right\} \leq \frac {8 \sqrt {m} | \Pi_ {0} | ^ {2}}{\varepsilon_ {0}} r (M). \tag {25}
$$
Proof of Lemma E.1. We first describe the construction of  $\Pi_0$ . For  $\varepsilon = \frac{1}{4}\varepsilon_0$ , let  $D_{\varepsilon} = \{x_{1},\dots ,x_{I}\}$  be an  $\varepsilon$ -grid of  $\mathcal{F}_P$  under  $\ell_2$  distance, and  $\Lambda (\mathbf{p};x,R)\triangleq (R - \| \mathbf{p} - x\| _2)_+$  be the tent function with center  $x$  and radius  $R$ . Consider the following binning
$$
\Pi_ {0} = \left\{w _ {i} (\mathbf {p}) \triangleq \frac {\Lambda (\mathbf {p} ; x _ {i} , 2 \varepsilon)}{\sum_ {j \in [ I ]} \Lambda (\mathbf {p} ; x _ {j} , 2 \varepsilon)}: x _ {i} \in D _ {\varepsilon} \right\}. \tag {26}
$$
Clearly,  $|\Pi_0| = I < \infty$  because the diameter of  $\mathcal{F}_P$  is bounded as stated in Theorem 5.1. We can also verify that  $\Pi_0$  satisfies  $\sum_{i\in [I]}w_i(\mathbf{p}) = 1$  for all  $\mathbf{p}\in \mathcal{F}_P$  because  $w_{i}(\mathbf{p})$  is defined as the normalized tent function.
Now we prove that  $\Pi_0$  satisfies the desired property. Since the agent is adaptively calibrated to  $\Pi_0$ , we have that  $\forall i \in [I]$ ,
$$
\frac {n _ {[ M ]} (i)}{M} \left\| \bar {\mathbf {p}} _ {[ M ]} (i) - \mathbf {h} \right\| _ {2} \leq \sqrt {m} \lim _ {M \to \infty} \frac {n _ {[ M ]} (i)}{M} \left\| \bar {\mathbf {p}} _ {[ M ]} (i) - \mathbf {h} \right\| _ {\infty} \leq \sqrt {m} r (M).
$$
Now, for  $\delta = 3\varepsilon = \frac{3}{4}\varepsilon_0$ , let  $D_{\varepsilon}^{(\delta)} \subseteq D_{\varepsilon}$  be defined as
$$
D _ {\varepsilon} ^ {(\delta)} = \left\{x _ {i} \in D _ {\varepsilon}: \| x _ {i} - \mathbf {h} \| \geq \delta \right\}. \tag {27}
$$
Since  $|D_{\varepsilon}^{(\delta)}| \leq |D_{\varepsilon}| = I < \infty$ , taking the sum of calibration error over bins in  $D_{\varepsilon}^{(\delta)}$ , we obtain
$$
\sum_ {x _ {i} \in D _ {\varepsilon} ^ {(\delta)}} \frac {n _ {[ M ]} (i)}{M} \| \bar {\mathbf {p}} _ {[ M ]} (i) - \mathbf {h} \| _ {2} = \frac {1}{M} \sum_ {x _ {i} \in D _ {\varepsilon} ^ {(\delta)}} \left\| \sum_ {s \in [ M ]} w _ {i} \left(\mathbf {p} _ {s}\right) \left(\mathbf {h} - \mathbf {p} _ {s}\right) \right\| _ {2} \leq \sqrt {m} I r (M). \tag {28}
$$
We can further lower bound (28) and get:
$$
\begin{array}{l} \frac {1}{M} \sum_ {x _ {i} \in D _ {\varepsilon} ^ {(\delta)}} \left\| \sum_ {s \in [ M ]} w _ {i} (\mathbf {p} _ {s}) (\mathbf {h} - \mathbf {p} _ {s}) \right\| _ {2} = \frac {1}{M} \sum_ {x _ {i} \in D _ {\varepsilon} ^ {(\delta)}} \left\| \sum_ {s \in [ M ]} w _ {i} (\mathbf {p} _ {s}) \left((\mathbf {h} - x _ {i}) + (x _ {i} - \mathbf {p} _ {s})\right) \right\| _ {2} \\ \stackrel {\mathrm {(a)}} {\geq} \frac {1}{M} \sum_ {x _ {i} \in D _ {\varepsilon} ^ {(\delta)}} \left(\left\| \sum_ {s \in [ M ]} w _ {i} (\mathbf {p} _ {s}) (\mathbf {h} - x _ {i}) \right\| _ {2} - \left\| \sum_ {s \in [ M ]} w _ {i} (\mathbf {p} _ {s}) (x _ {i} - \mathbf {p} _ {s}) \right\| _ {2}\right) \\ \stackrel {(b)} {\geq} \frac {1}{M} \sum_ {x _ {i} \in D _ {\varepsilon} ^ {(\delta)}} \sum_ {s \in [ M ]} w _ {i} \left(\left\| \mathbf {h} - x _ {i} \right\| _ {2} - \left\| x _ {i} - \mathbf {p} _ {s} \right\| _ {2}\right) \\ \stackrel {\mathrm {(c)}} {\geq} \frac {1}{M} \sum_ {x _ {i} \in D _ {\varepsilon} ^ {(\delta)}} n _ {[ M ]} (i) (\delta - 2 \varepsilon) \geq \frac {\varepsilon_ {0}}{4 M} \sum_ {x _ {i} \in D _ {\varepsilon} ^ {(\delta)}} n _ {[ M ]} (i). \\ \end{array}
$$
In the above inequalities, (a) and (b) are both due to triangle inequalities, and (c) is because  $\| \mathbf{h} - x_i \|_2 \geq \delta$  from the definition of  $D_{\varepsilon}^{(\delta)}$  in (27) and  $\| x_i - \mathbf{p}_s \|_2 < 2\varepsilon$  whenever  $w_i(\mathbf{p}_s) > 0 \iff \Lambda(\mathbf{p}_s; x_i, 2\varepsilon) > 0$ . Together with (28), the above set of inequalities imply
$$
\frac {1}{M} \sum_ {x _ {i} \in D _ {\varepsilon} ^ {(\delta)}} n _ {[ t ]} (i) \leq \left(\frac {4}{\varepsilon_ {0}}\right) \frac {1}{M} \sum_ {x _ {i} \in D _ {\varepsilon} ^ {(\delta)}} \left\| \sum_ {s \in [ M ]} w _ {i} \left(\mathbf {p} _ {s}\right) \left(\mathbf {h} - \mathbf {p} _ {s}\right) \right\| _ {2} \leq \frac {4 \sqrt {m} I}{\varepsilon_ {0}} r (M). \tag {29}
$$
On the other hand, since  $D_{\varepsilon}$  is an  $\varepsilon$ -grid of  $\mathcal{F}_P$ , if  $\| \mathbf{p}_s - \mathbf{h}\| _2\geq \varepsilon_0$ , there must exist  $x_{i}\in D_{\varepsilon}$  such that  $\| x_{i} - \mathbf{p}_{s}\|_{2}\leq \varepsilon$ , which implies
$$
\| x _ {i} - \mathbf {h} \| _ {2} \geq \| \mathbf {p} _ {s} - \mathbf {h} \| _ {2} - \| x _ {i} - \mathbf {p} _ {s} \| _ {2} \geq \varepsilon_ {0} - \varepsilon = \frac {3}{4} \varepsilon_ {0} = \delta \quad \Rightarrow \quad x _ {i} \in D _ {\varepsilon} ^ {(\delta)}.
$$
As for the weight that  $w_{i}$  assigns to  $\mathbf{p}_s$ , we also have
$$
w _ {i} (\mathbf {p} _ {s}) = \frac {\Lambda (\mathbf {p} _ {s} ; x _ {i} , 2 \varepsilon)}{\sum_ {j \in [ I ]} \Lambda (\mathbf {p} _ {s} ; x _ {j} , 2 \varepsilon)} \geq \frac {2 \varepsilon - \varepsilon}{I \cdot 2 \varepsilon} = \frac {1}{2 I}.
$$
$$
\frac {1}{M} \left| \left\{s \in [ M ]: \| \mathbf {p} _ {s} - \mathbf {h} \| \geq \varepsilon_ {0} \right\} \right| \leq \frac {1}{M} \sum_ {x _ {i} \in D _ {\varepsilon} ^ {(\delta)}} \sum_ {s \in [ M ]} (2 I) w _ {i} \left(\mathbf {p} _ {s}\right) = \frac {2 I}{t} \sum_ {x _ {i} \in D _ {\varepsilon} ^ {(\delta)}} n _ {[ M ]} (i) \tag {30}
$$
Finally, combining inequalities (29) and (30), we conclude that
$$
\frac {1}{M} \left| \left\{s \in [ M ]: \| \mathbf {p} _ {s} - \mathbf {h} \| \geq \varepsilon_ {0} \right\} \right| \leq (2 I) \lim  _ {M \to \infty} \frac {1}{M} \sum_ {x _ {i} \in D _ {\varepsilon} ^ {(\delta)}} n _ {[ M ]} (i) \leq \frac {8 \sqrt {m} I ^ {2}}{\varepsilon_ {0}} r (M),
$$
which proves (25). The proof is complete by taking the limit of  $M \to 0$ , which guarantees  $r(M) \to 0$  and immediately implies the convergence result in (24).
Note that the rate in Equation (25) does not depend on strategy  $\mathbf{h}$ . Therefore, in the context of running LAZYGDwOG (Algorithm 3), we can turn Lemma E.1 into the following uniform convergence result across epochs:
Proposition E.2. For any  $\varepsilon_0 > 0$ , there exists a finite binning  $\Pi_0$ , such that  $\forall \Phi > 0$ , if the principal runs LAZYGDwOG for  $\Phi$  epochs where each epoch has length  $M$ , and the agent's forecasts  $(\mathbf{p}_{\phi, s})_{\phi \in [\Phi], s \in [M]}$  are adaptively calibrated with respect to  $\Pi_0$ , then we have the following uniform convergence guarantee:
$$
\lim  _ {M \rightarrow \infty} \sup  _ {\phi \in [ \Phi ]} \frac {1}{M} \left| \{s \in [ M ]: \| \mathbf {p} _ {\phi , s} - \mathbf {h} _ {\phi} \| _ {2} \geq \varepsilon_ {0} \} \right| = 0 \tag {31}
$$
Remark E.3. Note that the rate in (25) has a polynomial dependency on  $|\Pi_0|$ , which, due to the construction in the proof of Lemma E.1, ends up being exponential in  $m$  because it is the size of a  $\frac{\varepsilon_0}{4}$  grid of the domain  $\mathcal{A}_P$ . To improve on this exponential dependency, one possible approach is to design an adaptive calibration algorithm for the agent that achieves the stronger notion of  $\ell_1$  calibration, which is more common in recent literature. For example, Hart [36], Foster and Vohra [29, 30] are defined using  $\ell_1$  calibration error rather than  $\ell_{\infty}$ . Another approach is to avoid using naive conversion from  $\ell_{\infty}$  to  $\ell_1$  calibration error in (28), which leads to a polynomial dependency on the number of bins. These two approaches are equivalent ways of formulating the problem, and they both lead to interesting open directions.
E.3 More auxiliary lemmas: approximate best response and closeness in utility
In this section, we use the results in Appendix E.2 to show that the average feedback  $\frac{1}{M}\sum_{s\in [M]}y_{\phi ,s}$  in epoch  $\phi \in [\Phi ]$  is close to the best response  $\mathsf{BR}(\mathbf{h}_{\phi})$  (Lemma E.4), and that the principal's average utility in this epoch is close to  $U_{P}(\mathbf{h}_{\phi},\mathsf{BR}(\mathbf{h}_{\phi}))$  (Lemma E.5).
Lemma E.4. For any  $\varepsilon_1 > 0$ , there exists a finite binning  $\Pi_0$  and  $M_0 < \infty$  such that when agent's forecasts  $\mathbf{p}_{1:t}$  are adaptively calibrated with respect to  $\Pi_0$ , then we have that  $\forall M \geq M_0$ ,
$$
\sup  _ {\phi \in [ \Phi ]} \left\| \frac {1}{M} \sum_ {s \in [ M ]} y _ {\phi , s} - \mathsf {B R} (\mathbf {h} _ {\phi}) \right\| _ {2} \leq \varepsilon_ {1}.
$$
Proof. Let  $\varepsilon_0 = \frac{\varepsilon_1}{2L_{\mathrm{BR}}}$  and  $\Pi_0$  be the binning that satisfies Proposition E.2 for parameter  $\varepsilon_0$ . Therefore, we know from Equation (31) in Proposition E.2 that for  $\varepsilon_2 = \frac{\varepsilon_1}{2\cdot D_P\cdot L_{\mathrm{BR}}}$  there exists  $M_0$  such that  $\forall M \geq M_0$ ,
$$
\sup  _ {\phi \in [ \Phi ]} \frac {1}{M} \left| \left\{s \in [ M ]: \| \mathbf {p} _ {\phi , s} - \mathbf {h} _ {\phi} \| _ {2} \geq \varepsilon_ {0} \right\} \right| \leq \varepsilon_ {2}. \tag {32}
$$
Using Lipschitzness of the best response mapping  $\mathsf{BR}(\cdot)$ , we have that  $\forall \phi \in [\Phi]$
$$
\begin{array}{l} \left\| \frac {1}{M} \sum_ {s \in [ M ]} y _ {\phi , s} - \mathbb {B R} (\mathbf {h} _ {\phi}) \right\| \\ \leq \frac {1}{M} \sum_ {s \in [ M ]} \| y _ {\phi , s} - \mathrm {B R} (\mathbf {h} _ {\phi}) \| _ {2} \quad \text {(T r i a n g l e i n e q u a l i t y)} \\ \leq L _ {\mathrm {B R}} \frac {1}{M} \sum_ {s \in [ M ]} \| \mathbf {p} _ {\phi , s} - \mathbf {h} _ {\phi} \| _ {2} \quad (\mathrm {B R} (\cdot) \text {i s} L _ {\mathrm {B R}} \text {- L i p s c h i t z}) \\ \leq L _ {\mathbb {B R}} \frac {1}{M} \left(\sum_ {s \in [ M ]: \| \mathbf {p} _ {\phi , s} - \mathbf {h} _ {\phi} \| \geq \varepsilon_ {0}} \operatorname {d i a m} \left(\mathcal {H} _ {P}\right) + \sum_ {s \in [ M ]: \| \mathbf {p} _ {\phi , s} - \mathbf {h} _ {\phi} \| <   \varepsilon_ {0}} \varepsilon_ {0}\right) \\ \leq L _ {\mathrm {B R}} \frac {1}{M} \left(\varepsilon_ {2} M \cdot D _ {P} + M \cdot \varepsilon_ {0}\right) \quad (\text {E q .} (3 2) \& \operatorname {d i a m} \left(\mathcal {H} _ {P}\right) \leq D _ {P}) \\ \leq D _ {P} \cdot L _ {\mathrm {B R}} \cdot \varepsilon_ {2} + L _ {\mathrm {B R}} \cdot \varepsilon_ {0} = \frac {\varepsilon_ {1}}{2} + \frac {\varepsilon_ {1}}{2} = \varepsilon_ {1}. \\ \end{array}
$$
Lemma E.5. For any  $\varepsilon_1 > 0$ , there exists a finite binning  $\Pi_0$  and  $M_0 < \infty$  such that when agent's forecasts  $\mathbf{p}_{1:t}$  are adaptively calibrated with respect to  $\Pi_0$ , then we have that  $\forall M \geq M_0$ ,
$$
\sup  _ {\phi \in [ \Phi ]} \left| \frac {1}{M} \sum_ {s \in [ M ]} U _ {P} \left(\mathbf {h} _ {\phi}, y _ {\phi , s}\right) - U _ {P} \left(\mathbf {h} _ {\phi}, \mathbf {B R} \left(\mathbf {h} _ {\phi}\right)\right) \right| \leq \varepsilon_ {1}. \tag {33}
$$
Proof. The proof of this lemma is very similar to that of Lemma E.4, with a different choice of constants  $\varepsilon_0$  and  $\varepsilon_{2}$ . Note that since  $U_{P}$  is  $L_{2}$ -Lipschitz in the second argument, we have
$$
\begin{array}{l} \left| \frac {1}{M} \sum_ {s \in [ M ]} U _ {P} \left(\mathbf {h} _ {\phi}, y _ {\phi , s}\right) - U _ {P} \left(\mathbf {h} _ {\phi}, \mathbb {R} \left(\mathbf {h} _ {\phi}\right)\right) \right| \leq \frac {1}{M} \sum_ {s \in [ M ]} \| U _ {P} \left(\mathbf {h} _ {\phi}, y _ {\phi , s}\right) - U _ {P} \left(\mathbf {h} _ {\phi}, \mathbb {R} \left(\mathbf {h} _ {\phi}\right)\right) \| _ {2} \\ \leq L _ {2} \cdot \frac {1}{M} \sum_ {s \in [ M ]} \| y _ {\phi , s} - \mathbb {B R} (\mathbf {h} _ {\phi}) \| _ {2}. \\ \end{array}
$$
The rest of the proof follows from Lemma E.4 by choosing  $\varepsilon_0 = \frac{\varepsilon_1}{2L_2L_{\mathrm{BR}}}$  and  $\varepsilon_{2} = \frac{\varepsilon_{1}}{2\cdot D_{P}\cdot L_{\mathrm{BR}}L_{2}}$
F Future Directions
Adaptive Calibration Although the results in this paper are all stated in terms of the  $\ell_{\infty}$ -calibration error (e.g., maximum instead of sum over bins), a lot of the existing calibration literature focuses on  $\ell_{1}$ -calibration error [29, 30]. It is an interesting problem whether we can get  $\ell_{1}$ -adaptive calibration error bounds without a polynomial dependency in the number of binning functions, where obtaining such bounds lead to polynomial improvements on the dependency of  $m$  (the number of agent's actions). In the case of continuous calibration, it is an open problem to obtain uniform (adaptive) calibration error bounds for parametric or nonparametric continuous binning function classes. Resolving this open problem could lead to a better rate for the learning direction of Theorem 5.2, as the current result uses naive  $\ell_{\infty}$ -to- $\ell_{1}$  conversion of calibration error that leads to linear dependency on the number of binning functions, which turns out to be exponential in the dimension of the principal's action space. See Remark E.3 for more details.
Convex optimization from membership oracles In the case of finite Stackelberg games, we use the results of Lee et al. [42] to solve the constrained convex optimization problem in each polytope from approximate membership queries, where the key step is a polynomial reduction from approximate separation oracles to approximate membership oracles. However, the precision of the constructed separation oracle is worse than the precision of the membership oracle, which naturally leads to a