SlowGuess/NeurIPS_2023_OCR huggingface dataset:
- For any  $\pmb{v},\pmb{v}^{\prime}\in (\mathcal{K}_{\epsilon}^{-})^{*}$  we have  $\left\| |\pmb{a}_i^\top \pmb {v}| - |\pmb {a}_i^\top \pmb {v}'|\right\|_{\psi_2}\leq \| \pmb {a}_i^\top (\pmb {v} - \pmb {v}')\|_{\psi_2} = O(1)\| \pmb {v} - \pmb {v}'\| _2$  Similarly as before, we assume  $\| \pmb {a}_i\| _2 = O(\sqrt{n})$  , which gives  $|\pmb {a}_i^\top \pmb {v} - \pmb {a}_i^\top \pmb {v}'|\leq \| \pmb {a}_i\| _2\| \pmb {v} -$ $\pmb{v}^{\prime}\|_{2} = O(\sqrt{n})\| \pmb {v} - \pmb{v}^{\prime}\|_{2}$  . Moreover,  $(\mathcal{K}_{\epsilon}^{-})^{*}\subset \mathbb{B}_{2}^{n}$  implies  $\left\| |\pmb {a}_i^\top \pmb {v}|\right\|_{\psi_2} = O(1)$  and  $|\pmb {a}_i^\top \pmb {v}|\leq$ $\| \pmb {a}_i\| _2\| \pmb {v}\| _2 = O(\sqrt{n})$  holds for all  $\pmb {v}\in (\mathcal{K}_{\epsilon}^{-})^{*}$
Combined with Assumption 3,  $\mathcal{R}_{u2,c}$  satisfies the conditions of Theorem 2 with
$$
M _ {g} \asymp L _ {0} + \frac {B _ {0}}{\beta}, A _ {g} = A _ {g} ^ {(2)}, M _ {h} \asymp 1, A _ {h} \asymp O (1)
$$
$$
L _ {g} \asymp \sqrt {n} \big (L _ {0} + \frac {B _ {0}}{\beta} \big), U _ {g} = U _ {g} ^ {(2)}, L _ {h} \asymp \sqrt {n}, U _ {h} \asymp \sqrt {n}
$$
and  $P_0 = P_0^{(2)} + 2\exp (-\Omega (n))$  . Suppose we have
$$
m \gtrsim \mathscr {H} \left(\mathcal {K}, \frac {A _ {g} ^ {(2)}}{\sqrt {m} n \left[ L _ {0} + \frac {B _ {0}}{\beta} \right]}\right) + \mathscr {H} \left(\left(\mathcal {K} _ {\epsilon} ^ {-}\right) ^ {*}, \frac {A _ {g} ^ {(2)}}{\sqrt {m} \left(A _ {g} ^ {(2)} + \sqrt {n} U _ {g} ^ {(2)}\right)}\right),
$$
which can be guaranteed (from Lemma 6) by
$$
m \gtrsim k \log \left(\frac {L r \sqrt {m}}{A _ {g} ^ {(2)}} \left[ n \left(L _ {0} + \frac {B _ {0}}{\beta}\right) + \frac {T \left(A _ {g} ^ {(2)} + \sqrt {n} U _ {g} ^ {(2)}\right)}{\epsilon} \right]\right). \tag {C.7}
$$
Then, we can invoke Theorem 2 to obtain that the following bound holds with probability at least  $1 - mP_0^{(2)} - 2m\exp (-\Omega (n)) - C\exp (-\Omega (k))$
$$
\begin{array}{l} \left| \mathscr {R} _ {u 2, c} \right| \lesssim \frac {A _ {g} ^ {(2)}}{\sqrt {m}} \sqrt {\mathscr {H} \left(\mathcal {K} , \frac {A _ {g} ^ {(2)}}{\sqrt {m} n \left[ L _ {0} + \frac {B _ {0}}{\beta} \right]}\right) + \mathscr {H} \left(\left(\mathcal {K} _ {\epsilon} ^ {-}\right) ^ {*}, \frac {A _ {g} ^ {(2)}}{\sqrt {m} \left(A _ {g} ^ {(2)} + \sqrt {n} U _ {g} ^ {(2)}\right)}\right)} \tag {C.8} \\ \lesssim A _ {g} ^ {(2)} \sqrt {\frac {k}{m} \log \left(\frac {L r \sqrt {m}}{A _ {g} ^ {(2)}} \Big [ n \big (L _ {0} + \frac {B _ {0}}{\beta} \big) + \frac {T (A _ {g} ^ {(2)} + \sqrt {n} U _ {g} ^ {(2)})}{\epsilon} \Big ]\right)}. \\ \end{array}
$$
(2.3) Bounding the expectation terms  $\mathcal{R}_{u1,e},\mathcal{R}_{u2,e}$
Recall that  $\xi_{i,\beta}(a) = f_{i,\beta}(a) - Ta$  and  $\varepsilon_{i,\beta}(a) = f_{i,\beta}(a) - f_i(a)$ , and so  $\xi_{i,\beta}(a) = \varepsilon_{i,\beta}(a) + f_i(a) - Ta$ . Hence, by using  $\mathbb{E}[\pmb{a}_ia_i^\top ] = \pmb{I}_n$  and  $\| \pmb {v}\| _2 = 1$ , we have
$$
\begin{array}{l} \mathscr {R} _ {u 1, e} \leq \sup  _ {\boldsymbol {x}, \boldsymbol {v}} \mathbb {E} \left[ \left(f _ {i} \left(\boldsymbol {a} _ {i} ^ {\top} \boldsymbol {x}\right) - T \boldsymbol {a} _ {i} ^ {\top} \boldsymbol {x}\right) \left(\boldsymbol {a} _ {i} ^ {\top} \boldsymbol {v}\right) \right] + \sup  _ {\boldsymbol {x}, \boldsymbol {v}} \mathbb {E} \left[ \left[ \varepsilon_ {i, \beta} \left(\boldsymbol {a} _ {i} ^ {\top} \boldsymbol {x}\right) \right] \boldsymbol {a} _ {i} ^ {\top} \boldsymbol {v} \right] \\ \leq \sup  _ {\boldsymbol {x} \in \mathcal {X}} \left\| \mathbb {E} \left[ f _ {i} \left(\boldsymbol {a} _ {i} ^ {\top} \boldsymbol {x}\right) \boldsymbol {a} _ {i} \right] - T \boldsymbol {x} \right\| _ {2} + \sup  _ {\boldsymbol {x}, \boldsymbol {v}} \mathbb {E} \left[ | \varepsilon_ {i, \beta} \left(\boldsymbol {a} _ {i} ^ {\top} \boldsymbol {x}\right) | | \boldsymbol {a} _ {i} ^ {\top} \boldsymbol {v} | \right] \tag {C.9} \\ \leq \sup  _ {\boldsymbol {x} \in \mathcal {X}} \rho (\boldsymbol {x}) + \mathcal {R} _ {u 2, e}, \\ \end{array}
$$
where  $\rho(\pmb{x})$  is the model mismatch defined in (2.3), and  $\mathcal{R}_{u2,e}$  is defined in (C.3). It remains to bound  $\mathcal{R}_{u2,e}$ , for which we first apply Cauchy-Schwarz (with  $\| \pmb{v} \|_2 \leq 1$ ) and then use Lemma 13 to obtain
$$
\begin{array}{l} \mathscr {R} _ {u 2, e} = \sup  _ {\boldsymbol {x}, \boldsymbol {v}} \mathbb {E} \left[ | \varepsilon_ {i, \beta} (\boldsymbol {a} _ {i} ^ {\top} \boldsymbol {x}) | | \boldsymbol {a} _ {i} ^ {\top} \boldsymbol {v} | \right] \\ \leq \sup  _ {\boldsymbol {x}, \boldsymbol {v}} \sqrt {\mathbb {E} [ | \varepsilon_ {i , \beta} (\boldsymbol {a} _ {i} ^ {\top} \boldsymbol {x}) | ^ {2} ]} \sqrt {\mathbb {E} [ | \boldsymbol {a} _ {i} ^ {\top} \boldsymbol {v} | ^ {2} ]} \\ \leq \sup  _ {\boldsymbol {x} \in \mathcal {K}} \sqrt {\mathbb {E} \left[ | \varepsilon_ {i , \beta} \left(\boldsymbol {a} _ {i} ^ {\top} \boldsymbol {x}\right) | ^ {2} \mathbb {1} \left(\boldsymbol {a} _ {i} ^ {\top} \boldsymbol {x} \in \mathscr {D} _ {f i} + \left[ - \frac {\beta}{2} , \frac {\beta}{2} \right]\right) \right]} \tag {C.10} \\ \leq \left(\frac {3 L _ {0} \beta}{2} + B _ {0}\right) \sup  _ {\boldsymbol {x} \in \mathcal {K}} \sqrt {\mathbb {P} \left(\boldsymbol {a} _ {i} ^ {\top} \boldsymbol {x} \in \mathscr {D} _ {f _ {i}} + \left[ - \frac {\beta}{2} , \frac {\beta}{2} \right]\right)} \\ \leq \left(\frac {3 L _ {0} \beta}{2} + B _ {0}\right) \sup  _ {\boldsymbol {x} \in \mathcal {K}} \sqrt {\mu_ {\beta} (\boldsymbol {x})}, \\ \end{array}
$$
where we use Lemma 13 in the third and fourth line, and  $\mu_{\beta}(\pmb{x})$  is defined in (2.4).
(3) Combining everything to conclude the proof.
Recall that in Assumption 4, we assume that
$$
\sup  _ {\boldsymbol {x} \in \mathcal {X}} \rho (\boldsymbol {x}) \lesssim (A _ {g} ^ {(1)} \vee A _ {g} ^ {(2)}) \sqrt {\frac {k}{m}},
$$
and we take sufficiently small  $\beta_{1}$  such that
$$
\left(L _ {0} \beta_ {1} + B _ {0}\right) \sup  _ {\boldsymbol {x} \in \mathcal {K}} \sqrt {\mu_ {\beta_ {1}} (\boldsymbol {x})} \lesssim \left(A _ {g} ^ {(1)} \vee A _ {g} ^ {(2)}\right) \sqrt {\frac {k}{m}},
$$
then by setting  $\beta = \beta_{1}$ , the derived bound of  $\mathcal{R}_{u1,c} + \mathcal{R}_{u2,c}$  (see (C.6) and (C.8)) dominates that of  $\mathcal{R}_{u1,e} + \mathcal{R}_{u2,e}$  (see (C.9) and (C.10)), and so  $\mathcal{R}_{u1} + \mathcal{R}_{u2} \lesssim \mathcal{R}_{u1,c} + \mathcal{R}_{u2,c}$ .
Recall that (C.6) and (C.8) are guaranteed by the sample size of (C.5) and (C.7), while (C.5) and (C.7) hold as long as
$$
\begin{array}{l} m \gtrsim k \log \left(\frac {L r \sqrt {m}}{A _ {g} ^ {(1)} \wedge A _ {g} ^ {(2)}} \left[ n \left(L _ {0} + T + \frac {B _ {0}}{\beta_ {1}}\right) + \frac {T (\sqrt {n} \left(U _ {g} ^ {(1)} \vee U _ {g} ^ {(2)}\right) + \left(A _ {g} ^ {(1)} \vee A _ {g} ^ {(2)}\right))}{\epsilon} \right]\right) \\ := k \mathscr {L} \quad (\text {h e r e w e u s e} \mathscr {L} \text {t o a b b r e v i a t e t h e l o g f a c t o r s}) \tag {C.11} \\ \end{array}
$$
with probability at least  $1 - m(P_0^{(1)} + P_0^{(2)}) - m\exp (-\Omega (n)) - C\exp (-\Omega (k))$  we have
$$
\mathscr {R} _ {u 1, c} + \mathscr {R} _ {u 2, c} \lesssim (A _ {g} ^ {(1)} \vee A _ {g} ^ {(2)}) \sqrt {\frac {k \mathscr {L}}{m}}.
$$
Therefore, the right-hand side of (3.1) can be uniformly bounded by
$$
O \left(\| \hat {\boldsymbol {x}} - T \boldsymbol {x} ^ {*} \| _ {2} \cdot \left(A _ {g} ^ {(1)} \vee A _ {g} ^ {(2)}\right) \sqrt {\frac {k \mathscr {L}}{m}}\right). \tag {C.12}
$$
Combining with the uniform lower bound for the left-hand side of (3.1), i.e.,  $\Omega (\| \hat{x} -Tx^{*}\|_{2}^{2})$ , we obtain the following bound uniformly for all  $x^{*}$ :
$$
\left\| \hat {\boldsymbol {x}} - T \boldsymbol {x} ^ {*} \right\| _ {2} \lesssim \left(A _ {g} ^ {(1)} \vee A _ {g} ^ {(2)}\right) \sqrt {\frac {k \mathcal {L}}{m}}.
$$
Hence, as long as
$$
m \gtrsim \left(A _ {g} ^ {(1)} \vee A _ {g} ^ {(2)}\right) ^ {2} \frac {k \mathscr {L}}{\epsilon^ {2}},
$$
we again obtain  $\| \hat{\pmb{x}} - T\pmb{x}^* \|_2 \leq 3\epsilon$ , which completes the proof.
C.2 Proof of Theorem 2
Proof. Step 1. Control the process over finite nets.
Recall that  $\mathcal{X}$  and  $\mathcal{V}$  are the index sets of  $\pmb{x}$  and  $\pmb{v}$ , as stated in Theorem 2. We first establish the desired concentration for a fixed pair  $(\pmb{x},\pmb{v})\in \mathcal{X}\times \mathcal{V}$ . By Lemma 2,  $\| g_{\pmb{x}}(\pmb{a}_i)h_{\pmb{v}}(\pmb{a}_i)\|_{\psi_1}\leq \| g_{\pmb{x}}(\pmb{a}_i)\|_{\psi_2}\| h_{\pmb{v}}(\pmb{a}_i)\|_{\psi_2}\leq A_gA_h$ . Furthermore, centering (Lemma 3) gives
$$
\left\| g _ {\boldsymbol {x}} \left(\boldsymbol {a} _ {i}\right) h _ {\boldsymbol {v}} \left(\boldsymbol {a} _ {i}\right) - \mathbb {E} \left[ g _ {\boldsymbol {x}} \left(\boldsymbol {a} _ {i}\right) h _ {\boldsymbol {v}} \left(\boldsymbol {a} _ {i}\right) \right] \right\| _ {\psi_ {1}} = O \left(A _ {g} A _ {h}\right).
$$
Thus, for fixed  $(\pmb {x},\pmb {v})\in \mathcal{X}\times \mathcal{V}$  we define
$$
I _ {\boldsymbol {x}, \boldsymbol {v}} = \frac {1}{m} \sum_ {i = 1} ^ {m} g _ {\boldsymbol {x}} (\boldsymbol {a} _ {i}) h _ {\boldsymbol {v}} (\boldsymbol {a} _ {i}) - \mathbb {E} \left[ g _ {\boldsymbol {x}} (\boldsymbol {a} _ {i}) h _ {\boldsymbol {v}} (\boldsymbol {a} _ {i}) \right].
$$
Then, we can invoke Bernstein's inequality (Lemma 4) to obtain for any  $t \geq 0$  that
$$
\mathbb {P} \left(| I _ {\boldsymbol {x}, \boldsymbol {v}} | \geq t\right) \leq 2 \exp \left(- c m \min  \left\{\left(\frac {t}{A _ {g} A _ {h}}\right) ^ {2}, \frac {t}{A _ {g} A _ {h}} \right\}\right). \tag {C.13}
$$
We construct  $\mathcal{G}_1$  as an  $\eta_1$ -net of  $\mathcal{X}$ , and  $\mathcal{G}_2$  as an  $\eta_2$ -net of  $\mathcal{V}$ , with both nets being minimal in that  $\log |\mathcal{G}_1| = \mathcal{H}(\mathcal{X}, \eta_1)$ ,  $\log |\mathcal{G}_2| = \mathcal{H}(\mathcal{V}, \eta_2)$ , and where  $\eta_1, \eta_2$  are to be chosen later. Then, we take a union bound of (C.13) over  $(\boldsymbol{x}, \boldsymbol{v}) \in \mathcal{G}_1 \times \mathcal{G}_2$  to obtain
$$
\mathbb {P} \left(\sup  _ {\substack {\boldsymbol {x} \in \mathcal {G} _ {1} \\ \boldsymbol {v} \in \mathcal {G} _ {2}}} | I _ {\boldsymbol {x}, \boldsymbol {v}} | \geq t\right) \leq 2 \exp \left(\mathcal {H} (\mathcal {X}, \eta_ {1}) + \mathcal {H} (\mathcal {V}, \eta_ {2}) - c m \min  \left\{\left(\frac {t}{A _ {g} A _ {h}}\right) ^ {2}, \frac {t}{A _ {g} A _ {h}} \right\}\right). \tag{C.14}
$$
Now we set  $t \asymp A_gA_h\sqrt{\frac{\mathcal{H}(\mathcal{X},\eta_1) + \mathcal{H}(\mathcal{V},\eta_2)}{m}}$  for a sufficiently large hidden constant. Then, if  $m \geq C(\mathcal{H}(\mathcal{X},\eta_1) + \mathcal{H}(\mathcal{V},\eta_2))$  for large enough  $C$  so that  $\frac{t}{A_gA_h} \leq 1$  (we assume this now and will confirm it in (C.21) after specifying  $\eta_1,\eta_2$ ), (C.14) gives
$$
\mathbb{P}\left(\sup_{\substack{\boldsymbol {x}\in \mathcal{G}_{1}\\ \boldsymbol {v}\in \mathcal{G}_{2}}}|I_{\boldsymbol {x},\boldsymbol{v}}|\gtrsim A_{g}A_{h}\sqrt{\frac{\mathcal{H}(\mathcal{X},\eta_{1}) + \mathcal{H}(\mathcal{V},\eta_{2})}{m}}\right)\leq 2\exp \big(-\Omega \big(\mathcal{H}(\mathcal{X},\eta_{1}) + \mathcal{H}(\mathcal{V},\eta_{2})\big)\big)
$$
Hence, from now on we proceed with the proof on the event
$$
\sup  _ {\substack {\boldsymbol {x} \in \mathcal {G} _ {1} \\ \boldsymbol {v} \in \mathcal {G} _ {2}}} | I _ {\boldsymbol {x}, \boldsymbol {v}} | \lesssim A _ {g} A _ {h} \sqrt {\frac {\mathcal {H} (\mathcal {X} , \eta_ {1}) + \mathcal {H} (\mathcal {V} , \eta_ {2})}{m}}, \tag{C.15}
$$
which holds within the promised probability.
Step 2. Control the approximation error of the nets.
We have derived a bound for  $\sup_{\boldsymbol{x} \in \mathcal{G}_1, \boldsymbol{v} \in \mathcal{G}_2} |I_{\boldsymbol{x}, \boldsymbol{v}}|$ , while we want to control  $I = \sup_{\boldsymbol{x} \in \mathcal{X}, \boldsymbol{v} \in \mathcal{V}} |I_{\boldsymbol{x}, \boldsymbol{v}}|$  so we further investigate how close these two quantities are. We define the event as
$$
\mathcal {E} _ {1} = \left\{\text {t h e e v e n t s i n (3 . 7) h o l d f o r a l l} \boldsymbol {a} _ {i}, i \in [ m ] \right\},
$$
then by assumption (A2.) in the theorem statement, a union bound gives  $\mathbb{P}(\mathcal{E}_1) \geq 1 - mP_0$ . In the following, we proceed with the analysis of the event  $\mathcal{E}_1$ . Combining with (C.15) we now bound  $|I_{\boldsymbol{x},\boldsymbol{v}}|$  for any given  $\boldsymbol{x} \in \mathcal{X}, \boldsymbol{v} \in \mathcal{V}$ . Specifically, we pick  $\boldsymbol{x}' \in \mathcal{G}_1$ ,  $\boldsymbol{v}' \in \mathcal{G}_2$  such that  $\| \boldsymbol{x}' - \boldsymbol{x} \|_2 \leq \eta_1$ ,  $\| \boldsymbol{v}' - \boldsymbol{v} \|_2 \leq \eta_2$ , and thus we have
$$
\left| I _ {\boldsymbol {x}, \boldsymbol {v}} \right| \leq \left| I _ {\boldsymbol {x} ^ {\prime}, \boldsymbol {v} ^ {\prime}} \right| + \left| I _ {\boldsymbol {x}, \boldsymbol {v}} - I _ {\boldsymbol {x} ^ {\prime}, \boldsymbol {v} ^ {\prime}} \right| \leq O \left(A _ {g} A _ {h} \sqrt {\frac {\mathcal {H} (\mathcal {X} , \eta_ {1}) + \mathcal {H} (\mathcal {V} , \eta_ {2})}{m}}\right) + \left| I _ {\boldsymbol {x}, \boldsymbol {v}} - I _ {\boldsymbol {x} ^ {\prime}, \boldsymbol {v} ^ {\prime}} \right|. \tag {C.16}
$$
$$
\begin{array}{l} \left| I _ {\boldsymbol {x}, \boldsymbol {v}} - I _ {\boldsymbol {x} ^ {\prime}, \boldsymbol {v} ^ {\prime}} \right| \\ = \frac {1}{m} \left| \sum_ {i = 1} ^ {m} \left(g _ {\boldsymbol {x}} (\boldsymbol {a} _ {i}) h _ {\boldsymbol {v}} (\boldsymbol {a} _ {i}) - g _ {\boldsymbol {x} ^ {\prime}} (\boldsymbol {a} _ {i}) h _ {\boldsymbol {v} ^ {\prime}} (\boldsymbol {a} _ {i})\right) - m \cdot \mathbb {E} \left(g _ {\boldsymbol {x}} (\boldsymbol {a} _ {i}) h _ {\boldsymbol {v}} (\boldsymbol {a} _ {i}) - g _ {\boldsymbol {x} ^ {\prime}} (\boldsymbol {a} _ {i}) h _ {\boldsymbol {v} ^ {\prime}} (\boldsymbol {a} _ {i})\right) \right| \\ \leq \underbrace {\frac {1}{m} \sum_ {i = 1} ^ {m} \left| g _ {\boldsymbol {x}} \left(\boldsymbol {a} _ {i}\right) h _ {\boldsymbol {v}} \left(\boldsymbol {a} _ {i}\right) - g _ {\boldsymbol {x} ^ {\prime}} \left(\boldsymbol {a} _ {i}\right) h _ {\boldsymbol {v} ^ {\prime}} \left(\boldsymbol {a} _ {i}\right) \right|} _ {\text {e r r} _ {1}} + \underbrace {\mathbb {E} \left| g _ {\boldsymbol {x}} \left(\boldsymbol {a} _ {i}\right) h _ {\boldsymbol {v}} \left(\boldsymbol {a} _ {i}\right) - g _ {\boldsymbol {x} ^ {\prime}} \left(\boldsymbol {a} _ {i}\right) h _ {\boldsymbol {v} ^ {\prime}} \left(\boldsymbol {a} _ {i}\rvert\right) \right|} _ {\text {e r r} _ {2}} \tag {C.17} \\ \end{array}
$$
We bound  $\mathrm{err}_1$  using the event  $\mathcal{E}_1$  as follows:
$$
\begin{array}{l} \operatorname {e r r} _ {1} \leq \frac {1}{m} \sum_ {i = 1} ^ {m} \left| \left| g _ {\boldsymbol {x}} (\boldsymbol {a} _ {i}) - g _ {\boldsymbol {x} ^ {\prime}} (\boldsymbol {a} _ {i}) \right| \cdot \left| h _ {\boldsymbol {v}} (\boldsymbol {a} _ {i}) \right| + \left| h _ {\boldsymbol {v}} (\boldsymbol {a} _ {i}) - h _ {\boldsymbol {v} ^ {\prime}} (\boldsymbol {a} _ {i}) \right| \cdot \left| g _ {\boldsymbol {x} ^ {\prime}} (\boldsymbol {a} _ {i}) \right| \right| \\ \leq \frac {1}{m} \sum_ {i = 1} ^ {m} \left| L _ {g} \cdot \| \boldsymbol {x} - \boldsymbol {x} ^ {\prime} \| _ {2} \cdot U _ {h} + L _ {h} \cdot \| \boldsymbol {v} - \boldsymbol {v} ^ {\prime} \| _ {2} \cdot U _ {g} \right| \tag {C.18} \\ \leq \frac {1}{m} \sum_ {i = 1} ^ {m} \left| L _ {g} U _ {h} \eta_ {1} + L _ {h} U _ {g} \eta_ {2} \right| = L _ {g} U _ {h} \eta_ {1} + L _ {h} U _ {g} \eta_ {2}. \\ \end{array}
$$
On the other hand, we bound  $\mathrm{err}_2$  using assumption (A1.) in the theorem statement. Noting that  $\mathbb{E}|X| = O\big(\| X\|_{\psi_1}\big)$  [50, Proposition 2.7.1(b)], and further applying Lemma 2 we obtain
$$
\begin{array}{l} \operatorname {e r r} _ {2} \lesssim \| g _ {\boldsymbol {x}} (\boldsymbol {a} _ {i}) h _ {\boldsymbol {v}} (\boldsymbol {a} _ {i}) - g _ {\boldsymbol {x} ^ {\prime}} (\boldsymbol {a} _ {i}) h _ {\boldsymbol {v} ^ {\prime}} (\boldsymbol {a} _ {i}) \| _ {\psi_ {1}} \\ \leq \left\| \left(g _ {\boldsymbol {x}} \left(\boldsymbol {a} _ {i}\right) - g _ {\boldsymbol {x} ^ {\prime}} \left(\boldsymbol {a} _ {i}\right)\right) h _ {\boldsymbol {v}} \left(\boldsymbol {a} _ {i}\right) \right\| _ {\psi_ {1}} + \left\| g _ {\boldsymbol {x} ^ {\prime}} \left(\boldsymbol {a} _ {i}\right) \left(h _ {\boldsymbol {v}} \left(\boldsymbol {a} _ {i}\right) - h _ {\boldsymbol {v} ^ {\prime}} \left(\boldsymbol {a} _ {i}\right)\right) \right\| _ {\psi_ {1}} \\ \leq \left\| g _ {\boldsymbol {x}} \left(\boldsymbol {a} _ {i}\right) - g _ {\boldsymbol {x} ^ {\prime}} \left(\boldsymbol {a} _ {i}\right) \right\| _ {\psi_ {2}} \left\| h _ {\boldsymbol {v}} \left(\boldsymbol {a} _ {i}\right) \right\| _ {\psi_ {2}} + \left\| g _ {\boldsymbol {x} ^ {\prime}} \left(\boldsymbol {a} _ {i}\right) \right\| _ {\psi_ {2}} \left\| h _ {\boldsymbol {v}} \left(\boldsymbol {a} _ {i}\right) - h _ {\boldsymbol {v} ^ {\prime}} \left(\boldsymbol {a} _ {i}\right) \right\| _ {\psi_ {2}} \tag {C.19} \\ \leq M _ {g} \cdot \| \boldsymbol {x} - \boldsymbol {x} ^ {\prime} \| _ {2} \cdot A _ {h} + A _ {g} \cdot M _ {h} \cdot \| \boldsymbol {v} - \boldsymbol {v} ^ {\prime} \| _ {2} \leq M _ {g} A _ {h} \eta_ {1} + A _ {g} M _ {h} \eta_ {2}. \\ \end{array}
$$
Note that the bounds (C.18) and (C.19) hold uniformly for all  $(\boldsymbol{x}, \boldsymbol{v}) \in \mathcal{X} \times \mathcal{V}$ , and hence, we can substitute them into (C.16) and (C.17) to obtain
$$
\sup  _ {\substack {\boldsymbol {x} \in \mathcal {X} \\ \boldsymbol {v} \in \mathcal {V}}} | I _ {\boldsymbol {x}, \boldsymbol {v}} | \leq O \left(A _ {g} A _ {h} \sqrt {\frac {\mathscr {H} (\mathcal {X} , \eta_ {1}) + \mathscr {H} (\mathcal {V} , \eta_ {2})}{m}}\right) + \left(L _ {g} U _ {h} + M _ {g} A _ {h}\right) \eta_ {1} + \left(L _ {h} U _ {g} + M _ {h} A _ {g}\right) \eta_ {2}. \tag{C.20}
$$
Recall that we use the shorthand  $S_{g,h} \coloneqq L_gU_h + M_gA_h$  and  $T_{g,h} \coloneqq L_hU_g + M_hA_g$ . We set  $\eta_1 \asymp \frac{A_gA_h}{\sqrt{m}S_{g,h}}$ ,  $\eta_2 \asymp \frac{A_gA_h}{\sqrt{m}T_{g,h}}$  so that the right-hand side of (C.20) is dominated by the first term. Overall, with a sample size satisfying
$$
m = \Omega \left(\mathcal {H} \left(\mathcal {X}, \frac {A _ {g} A _ {h}}{\sqrt {m} S _ {g , h}}\right) + \mathcal {H} \left(\mathcal {V}, \frac {A _ {g} A _ {h}}{\sqrt {m} T _ {g , h}}\right)\right), \tag {C.21}
$$
we can bound  $I = \sup_{\boldsymbol{x} \in \mathcal{X}} \sup_{\boldsymbol{v} \in \mathcal{V}} |I_{\boldsymbol{x}, \boldsymbol{v}}|$  (defined in the theorem statement) as
$$
I \lesssim A _ {g} A _ {h} \sqrt {\frac {\mathcal {H} \left(\mathcal {X} , A _ {g} A _ {h} m ^ {- 1 / 2} S _ {g , h} ^ {- 1}\right) + \mathcal {H} \left(\mathcal {V} , A _ {g} A _ {h} m ^ {- 1 / 2} T _ {g , h} ^ {- 1}\right)}{m}} \tag {C.22}
$$
with probability at least
$$
1 - m P _ {0} - 2 \exp \left[ - \Omega \left(\mathcal {H} \left(\mathcal {X}, \frac {A _ {g} A _ {h}}{\sqrt {m} S _ {g , h}}\right) + \mathcal {H} \left(\mathcal {V}, \frac {A _ {g} A _ {h}}{\sqrt {m} T _ {g , h}}\right)\right) \right].
$$
D Other Omitted Proofs
D.1 Proof of Lemma 1 (Lipschitz continuity of  $f_{i,\beta}$  and  $\varepsilon_{i,\beta}$ ).
Proof. It is straightforward to check that  $f_{i,\beta}$  and  $|\varepsilon_{i,\beta}|$  are piece-wise continuous functions; hence, it suffices to prove that they are Lipschitz with the claimed Lipschitz constant over each piece. In any interval contained in the part of  $x \notin \mathcal{D}_{f_i} + [-\frac{\beta}{2}, \frac{\beta}{2}]$ ,  $f_{i,\beta} = f_i$  and  $|\varepsilon_{i,\beta}| = 0$  trivially satisfy the claim. In any interval contained in  $[x_0 - \frac{\beta}{2}, x_0]$  for some  $x_0 \in \mathcal{D}_{f_i}$ ,  $f_{i,\beta}$  is linear with slope  $\frac{2}{\beta} (f_i^a(x_0) - f_i(x_0 - \frac{\beta}{2}))$ , combined with the bound  $|f_i^a(x_0) - f_i(x_0 - \frac{\beta}{2})| \leq |f_i^a(x_0) - f_i^-(x_0)| + |f_i^-(x_0) - f_i(x_0 - \frac{\beta}{2})| \leq \frac{|f_i^+(x_0) - f_i^-(x_0)|}{2} + \frac{L_0\beta}{2} \leq \frac{1}{2} (B_0 + L_0\beta)$ , we know that  $f_{i,\beta}$  is  $(L_0 + \frac{B_0}{\beta})$ -Lipschitz. Further,  $|\varepsilon_{i,\beta}| = |f_{i,\beta} - f_i|$ , and  $f_i$  is  $L_0$ -Lipschitz over this interval, so  $|\varepsilon_{i,\beta}|$  is  $(2L_0 + \frac{B_0}{\beta})$ -Lipschitz continuous. A similar argument applies to an interval contained in  $[x_0, x_0 + \frac{\beta}{2}]$ .
D.2 Proof of Lemma 6 (Metric entropy of constraint sets).
Proof. Bounding  $\mathcal{H}(\mathcal{K},\eta)$
By [50, Corollary 4.2.13], there exists an  $\left(\frac{\eta}{Lr}\right)$ -net  $\mathcal{G}_1$  of  $\mathbb{B}_2^k$  such that
$$
\log | \mathcal {G} _ {1} | \leq k \log \left(\frac {2 L r}{\eta} + 1\right) \leq k \log \frac {3 L r}{\eta},
$$
where we use  $\eta \leq Lr$ . Note that  $r\mathcal{G}_1$  is an  $\left(\frac{\eta}{L}\right)$ -net of  $\mathbb{B}_2^k (r)$ , and because  $G$  is  $L$ -Lipschitz,  $G(r\mathcal{G}_1)$  is an  $\eta$ -net of  $\mathcal{K}$ , thus yielding  $\mathcal{H}(\mathcal{K},\eta) \leq k\log \frac{3Lr}{\eta}$ .
Bounding  $\mathcal{H}(\mathcal{K}^{-},\eta)$  and  $\mathcal{H}(\mathcal{K}_{\epsilon}^{-},\eta)$
We construct  $\mathcal{G}_2$  as an  $\left(\frac{\eta}{2}\right)$ -net of  $\kappa$  satisfying  $\log |\mathcal{G}_2| \leq k\log \frac{6Lr}{\eta}$ . Then, it is easy to see that  $\mathcal{G}_2 - \mathcal{G}_2$  is an  $\eta$ -net of  $\mathcal{K}^{-} = \mathcal{K} - \mathcal{K}$ , showing that
$$
\mathcal {H} \left(\mathcal {K} ^ {-}, \eta\right) \leq \log | \mathcal {G} _ {2} | ^ {2} \leq 2 k \log \frac {6 L r}{\eta}.
$$
For a given  $T > 0$ , this directly implies  $\mathcal{H}(TK^{-},\eta) \leq 2k\log \frac{6TLr}{\eta}$ . Moreover, because  $\mathcal{K}_{\epsilon}^{-} \subset TK^{-}$ , by [50, Exercise 4.2.10] (which states that  $\mathcal{H}(\mathcal{K}_1,r) \leq \mathcal{H}(\mathcal{K}_2,\frac{r}{2})$  holds for any  $r > 0$  if  $\mathcal{K}_1 \subset \mathcal{K}_2$ ) we obtain
$$
\mathcal {H} \left(\mathcal {K} _ {\epsilon} ^ {-}, \eta\right) \leq \mathcal {H} \left(T \mathcal {K} ^ {-}, \frac {\eta}{2}\right) \leq 2 k \log \frac {1 2 T L r}{\eta}.
$$
Bounding  $\mathcal{H}\big((\mathcal{K}_\epsilon^-)^*,\eta \big)$
We construct  $\mathcal{G}_3$  as an  $(\epsilon \eta)$ -net of  $\kappa_{\epsilon}^{-}$  satisfying  $\log |\mathcal{G}_3| \leq 2k \log \frac{12TLr}{\epsilon\eta}$ , then we consider  $(\mathcal{G}_3)^* := \{\frac{z}{\|\pmb{z}\|_2} : \pmb{z} \in \mathcal{G}_3\}$ . We aim to prove that  $(\mathcal{G}_3)^*$  is an  $\eta$ -net of  $(\mathcal{K}_{\epsilon}^{-})^{*}$ . Note that any  $x_1 \in (\mathcal{K}_{\epsilon}^{-})^{*}$  can be written as  $\frac{z_1}{\|\pmb{z}_1\|_2}$  for some  $z_1 \in \mathcal{K}_{\epsilon}^{-}$  and recall that  $\| z_1 \|_2 \geq 2\epsilon$ . Moreover, by construction, there exists some  $z_2 \in \mathcal{G}_3$  such that  $\| z_1 - z_2 \|_2 \leq \epsilon \eta$ . Note that  $\frac{z_2}{\|\pmb{z}_2\|_2} \in (\mathcal{G}_3)^*$ , and moreover we have
$$
\begin{array}{l} \left\| \frac {z _ {1}}{\| z _ {1} \| _ {2}} - \frac {z _ {2}}{\| z _ {2} \| _ {2}} \right\| _ {2} \leq \left\| \frac {z _ {1}}{\| z _ {1} \| _ {2}} - \frac {z _ {2}}{\| z _ {1} \| _ {2}} \right\| _ {2} + \left\| \frac {z _ {2}}{\| z _ {1} \| _ {2}} - \frac {z _ {2}}{\| z _ {2} \| _ {2}} \right\| _ {2} \\ = \frac {\| z _ {1} - z _ {2} \| _ {2}}{\| z _ {1} \| _ {2}} + \frac {\| \| z _ {2} \| _ {2} - \| z _ {1} \| _ {2} |}{\| z _ {1} \| _ {2}} \\ \leq \frac {2 \| \boldsymbol {z} _ {1} - \boldsymbol {z} _ {2} \| _ {2}}{\| \boldsymbol {z} _ {1} \| _ {2}} \leq \frac {2 \epsilon \eta}{2 \epsilon} = \eta . \\ \end{array}
$$
$$
\mathscr {H} \left(\left(\mathcal {K} _ {\epsilon} ^ {-}\right) ^ {*}, \eta\right) \leq \log | (\mathcal {G} _ {3}) ^ {*} | \leq \log | \mathcal {G} _ {3} | \leq 2 k \log \frac {1 2 T L r}{\epsilon \eta},
$$
D.3 Proof of Lemma 8 (Choice of  $T$  in 1-bit GCS).