SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Again by the 1-Lipschitz property of the ReLU activation function, and by the distributive property of the convolution operation,
$$
\begin{array}{l} \left\| \phi \left(K ^ {(\ell)} * X ^ {(\ell - 1)}\right) - \phi \left(K ^ {(\ell)} * \tilde {X} ^ {(\ell - 1)}\right) \right\| _ {\max } \\ \left. \right. + \left\| \phi \left(K ^ {(\ell)} * \tilde {X} ^ {(\ell - 1)}\right) - \phi \left(L ^ {(2 \ell)} * \phi \left(\tilde {L} ^ {(2 \ell - 1)} * \tilde {X} ^ {(\ell - 1)}\right)\right)\right\| _ {\max } \\ \leq \left\| K ^ {(\ell)} * X ^ {(\ell - 1)} - K ^ {(\ell)} * \tilde {X} ^ {(\ell - 1)} \right\| _ {\max } \\ + \left\| K ^ {(\ell)} * \tilde {X} ^ {(\ell - 1)} - L ^ {(2 \ell)} * \phi \left(\tilde {L} ^ {(2 \ell - 1)} * \tilde {X} ^ {(\ell - 1)}\right) \right\| _ {\max } \\ = \left\| K ^ {(\ell)} * \left(X ^ {(\ell - 1)} - \tilde {X} ^ {(\ell - 1)}\right) \right\| _ {\max } \\ + \left\| K ^ {(\ell)} * \tilde {X} ^ {(\ell - 1)} - L ^ {(2 \ell)} * \phi \left(\tilde {L} ^ {(2 \ell - 1)} * \tilde {X} ^ {(\ell - 1)}\right) \right\| _ {\max } . \\ \end{array}
$$
Lemma 20 and the hypothesis  $\left\| K^{(\ell)}\right\| _1\leq 1$  imply that
$$
\left\| K ^ {(\ell)} * \left(X ^ {(\ell - 1)} - \tilde {X} ^ {(\ell - 1)}\right) \right\| _ {\max }
$$
$$
\begin{array}{l} + \left\| K ^ {(\ell)} * \tilde {X} ^ {(\ell - 1)} - L ^ {(2 \ell)} * \phi \left(\tilde {L} ^ {(2 \ell - 1)} * \tilde {X} ^ {(\ell - 1)}\right) \right\| _ {\max } \\ \leq \left\| K ^ {(\ell)} \right\| _ {1} \cdot \left\| \left(X ^ {(\ell - 1)} - \tilde {X} ^ {(\ell - 1)}\right) \right\| _ {\max } \\ + \left\| K ^ {(\ell)} * \tilde {X} ^ {(\ell - 1)} - L ^ {(2 \ell)} * \phi \left(\tilde {L} ^ {(2 \ell - 1)} * \tilde {X} ^ {(\ell - 1)}\right) \right\| _ {\max } \\ \leq \left\| \left(X ^ {(\ell - 1)} - \tilde {X} ^ {(\ell - 1)}\right) \right\| _ {\max } \\ + \left\| K ^ {(\ell)} * \tilde {X} ^ {(\ell - 1)} - L ^ {(2 \ell)} * \phi \left(\tilde {L} ^ {(2 \ell - 1)} * \tilde {X} ^ {(\ell - 1)}\right) \right\| _ {\max } . \\ \end{array}
$$
Now, we first apply Eq. 41 and then we unroll the recurrence for all layers (as, with probability  $1 - \varepsilon$ , Eq. 41 holds for all layers and for all choices of target filters  $K^{(i)} \in \mathbb{R}^{d_i \times d_i \times c_{i-1} \times c_i}$ ), obtaining
$$
\begin{array}{l} \left\| \left(X ^ {(\ell - 1)} - \tilde {X} ^ {(\ell - 1)}\right) \right\| _ {\max } \\ + \left\| K ^ {(\ell)} * \tilde {X} ^ {(\ell - 1)} - L ^ {(2 \ell)} * \phi \left(\tilde {L} ^ {(2 \ell - 1)} * \tilde {X} ^ {(\ell - 1)}\right) \right\| _ {\max } \\ \leq \left\| X ^ {(\ell - 1)} - \tilde {X} ^ {(\ell - 1)} \right\| _ {\max } + \frac {\varepsilon}{2 \ell} \cdot \left(1 + \frac {\varepsilon}{2 \ell}\right) ^ {\ell - 1} \\ \leq \sum_ {j = 1} ^ {\ell} \frac {\varepsilon}{2 \ell} \cdot \left(1 + \frac {\varepsilon}{2 \ell}\right) ^ {j - 1}. \\ \end{array}
$$
By summing the geometric series and observing that  $\varepsilon < 1$ , we conclude that
$$
\begin{array}{l} \sum_ {j = 1} ^ {\ell} \frac {\varepsilon}{2 \ell} \cdot \left(1 + \frac {\varepsilon}{2 \ell}\right) ^ {j - 1} = \left(1 + \frac {\varepsilon}{2 \ell}\right) ^ {\ell} - 1 \\ <   e ^ {\frac {\varepsilon}{2}} - 1 \\ \leq \varepsilon . \\ \end{array}
$$
$$
\left\| X ^ {(\ell)} - \tilde {X} ^ {(\ell)} \right\| _ {\max } \leq \varepsilon ,
$$
yielding the thesis.