SlowGuess/NeurIPS_2023_OCR huggingface dataset:
So the Rademacher complexity of  $\mathcal{L} \circ \mathcal{G}$  given  $n$  i.i.d. samples drawn from distribution with density  $p(x, y, m)$  can be defined as
$$
\mathfrak {R} _ {n} (\mathcal {L} \circ \mathcal {G}) = \mathbb {E} _ {p (\boldsymbol {x}, y, m)} \mathbb {E} _ {\boldsymbol {\sigma}} \left[ \sup  _ {g \in \mathcal {G}} \frac {1}{n} \sum_ {i = 1} ^ {n} \sigma_ {i} h (\boldsymbol {x} _ {i}, y _ {i}, m _ {i}) \right].
$$
Denote by  $\hat{R}_{L_{\tilde{\psi}}}(\pmb {g}) = \frac{1}{n}\sum_{i = 1}^{n}L_{\tilde{\psi}}(\pmb {g}(\pmb {x}_i),y_i,m_i)$  the empirical risk and  $\hat{g}$  the empirical risk minimizer. We have the following theorem:
Lemma 1.
$$
R _ {L _ {\tilde {\psi}}} (\hat {\boldsymbol {g}}) - R _ {L _ {\tilde {\psi}}} (\boldsymbol {g} ^ {*}) \leq 2 \sup  _ {\boldsymbol {g} \in \mathcal {G}} | \hat {R} _ {L _ {\tilde {\psi}}} (\boldsymbol {g}) - R _ {L _ {\tilde {\psi}}} (\boldsymbol {g}) |
$$
$$
\begin{array}{l} R _ {L _ {\tilde {\psi}}} (\hat {\boldsymbol {g}}) - R _ {L _ {\tilde {\psi}}} (\boldsymbol {g} ^ {*}) = R _ {L _ {\tilde {\psi}}} (\hat {\boldsymbol {g}}) - \hat {R} _ {L _ {\tilde {\psi}}} (\hat {\boldsymbol {g}}) + \hat {R} _ {L _ {\tilde {\psi}}} (\hat {\boldsymbol {g}}) - R _ {L _ {\tilde {\psi}}} (\boldsymbol {g} ^ {*}) \\ \leq R _ {L _ {\tilde {\psi}}} (\hat {\boldsymbol {g}}) - \hat {R} _ {L _ {\tilde {\psi}}} (\hat {\boldsymbol {g}}) + \hat {R} _ {L _ {\tilde {\psi}}} (\boldsymbol {g} ^ {*}) - R _ {L _ {\tilde {\psi}}} (\boldsymbol {g} ^ {*}) \\ \leq 2 \sup  _ {\boldsymbol {g} \in \mathcal {G}} \left| \hat {R} _ {L _ {\hat {\psi}}} (\boldsymbol {g}) - R _ {L _ {\hat {\psi}}} (\boldsymbol {g}) \right| \\ \end{array}
$$
Then it is routing to get the estimation error bound with Rademacher complexity and McDiarmid's inequality:
Theorem 4. Suppose  $L_{\tilde{\psi}}(\pmb{g}(\pmb{x}), y, m) \leq M$  and  $L_{\tilde{\psi}}$  is  $L$ -Lipschitz w.r.t.  $\pmb{g}(\pmb{x})$ , then with probability at least  $1 - \delta$ :
$$
R _ {L _ {\tilde {\psi}}} (\hat {\boldsymbol {g}}) - R _ {L _ {\tilde {\psi}}} (\boldsymbol {g} ^ {*}) \leq 4 \sqrt {2} L \sum_ {y = 1} ^ {K + 1} \Re_ {n} (\mathcal {G} _ {y}) + 4 M \sqrt {\frac {\ln (2 / \delta)}{2 n}}
$$
Proof. We will only discuss a one-sided bound on  $\sup_{\pmb{g} \in \mathcal{G}} \left( \hat{R}(\pmb{g}) - R(\pmb{g}) \right)$  that holds with probability at least  $1 - \frac{\delta}{2}$ . Altering  $(\pmb{x}[i], y_i, m_i)$  to  $(\pmb{x}'[i], y_i', m_i')$  and we can get a perturbed empirical risk  $\hat{R}'(\pmb{g})$ . We can learn that
$$
\left| \sup  _ {\boldsymbol {g} \in \mathcal {G}} \left(\hat {R} ^ {\prime} (\boldsymbol {g}) - R (\boldsymbol {g})\right) - \sup  _ {\boldsymbol {g} \in \mathcal {G}} \left(\hat {R} (\boldsymbol {g}) - R (\boldsymbol {g})\right) \right| \leq \left| \sup  _ {\boldsymbol {g} \in \mathcal {G}} \left(\hat {R} ^ {\prime} (\boldsymbol {g}) - \hat {R} (\boldsymbol {g})\right) \right| \leq \frac {M}{n}.
$$
Then we can use Mcdiarmids' inequality and vector-valued Talagrand inequality to learn:
$$
\sup  _ {\boldsymbol {g} \in \mathcal {G}} \left| \hat {R} _ {L _ {\bar {\psi}}} (\boldsymbol {g}) - R _ {L _ {\bar {\psi}}} (\boldsymbol {g}) \right| \leq 2 \sqrt {2} L \sum_ {y = 1} ^ {K + 1} \Re_ {n} (\mathcal {G} _ {y}) + 2 M \sqrt {\frac {\ln (2 / \delta)}{2 n}}
$$
Using the lemma 1 and we can conclude our proof.