SlowGuess/NeurIPS_2023_OCR huggingface dataset:
To prove (F.23), we assume that  $(Z(i_{1},i_{2},\ldots ,i_{\ell_{0}})\cap \mathcal{Y})\cap B_{H^{1}(D)}(g_{j},\varepsilon_{0})\neq \emptyset$  and  $g\in Z(i_1,i_2,\dots ,i_{\ell_0})\cap \mathcal{V}$ . By the assumption, there exists  $g^{*}\in (Z(i_{1},i_{2},\dots ,i_{\ell_{0}})\cap \mathcal{V})\cap B_{H^{1}(D)}(g_{j},\varepsilon_{0})$ . Using (F.31), we have
$$
\left\| g - g _ {j} \right\| _ {H ^ {1} (D)} \leq \left\| g - g ^ {*} \right\| _ {H ^ {1} (D)} + \left\| g ^ {*} - g _ {j} \right\| _ {H ^ {1} (D)} \leq 2 \epsilon_ {0}.
$$
Thus,  $g \in B_{H^1(D)}(g_j, 2\varepsilon_0)$ , which implies that the property (F.23) follows.
We next consider the case when the parameter  $a$  of the leaky relu function  $\sigma_{a}$  is zero. Again, we assume that  $D = [0,1]$  and  $y_{\ell} = h\ell$ , where  $h = 1 / \ell_0$  and  $\ell = 0,1,\ldots ,\ell_0$ . We consider  $g\in \mathcal{V}\cap Z(i_1,i_2,\dots ,i_{\ell_0})\subset \sigma_a(\overline{B}_{C^{1,\alpha}(\overline{D})}(0,R))$  of the form  $g = \sigma_{a}(v)$  and an interval  $[\ell_1h,(\ell_1 + 1)h]\subset D$ , where  $1\leq \ell_{1}\leq \ell_{0} - 2$ . We will consider four cases. First, if  $g$  does not obtain the value zero on the interval  $[\ell_1h,(\ell_1 + 1)h]$  the mean value theorem implies that there is  $x^{\prime}\in [\ell_{1}h,(\ell_{1} + 1)h]$  such that  $\frac{dg}{dx} (x') = \frac{dv}{dx} (x')$  is equal to  $d = (g(\ell_1h) - g([(\ell_1 - 1)h)) / h$ . Second, if  $g$  does not obtain the value zero on either of the intervals  $[( \ell _ { 1 } - 1 ) h ,   \ell _ { 1 } h ]$  or  $[( \ell _ { 1 } + 1 ) h ,   (\ell _ { 1 } + 2 ) h ]$ , we can use the mean value theorem to estimate the derivatives of  $g$  and  $v$  at some point of these intervals similarly to the first case. Third, if  $g$  does not vanish identically on the interval  $[\ell_1h,(\ell_1 + 1)h]$  but it obtains the value zero on the both intervals  $[( \ell _ { 1 } - 1 ) h ,   \ell _ { 1 } h ]$  and  $[( \ell _ { 1 } + 1 ) h ,   (\ell _ { 1 } + 2 ) h ]$ , the function  $v$  has two zeros on the interval  $[( \ell _ { 1 } - 1 ) h ,   (\ell _ { 1 } + 2 ) h ]$  and the mean value theorem implies that there is  $x^{\prime}\in [( \ell _ { 1 } - 1 ) h ,   (\ell _ { 1 } + 2 ) h ]$  such that  $\frac{dv}{dx} (x') = 0$ . Fourth, if none of the above cases are valid,  $g$  vanishes identically on the interval  $[\ell_1h,(\ell_1 + 1)h]$ . In all these cases the fact that  $\| v\|_{C^{1,\alpha}(\overline{D})}\leq R$  implies that the derivative of  $g$  can be estimated on the whole interval  $[\ell_1h,(\ell_1 + 1)h]$  within a small error. Using these observations we see for any  $\varepsilon_2,\varepsilon_3 > 0$  that if  $y_{\ell}\in D = [d_{1},d_{2}]\subset \mathbb{R}$ ,  $\ell = 1,2,\ldots ,\ell_0$  are a sufficiently dense grid in  $D$  and  $\varepsilon_{1}$  to be small enough, then the derivatives of any two functions  $g,g^{\prime}\in \mathcal{V}\cap Z(i_1,i_2,\dots ,i_{\ell_0})\subset \sigma_a(\overline{B}_{C^{1,\alpha}(\overline{D})}(0,R))$  of the form  $g = \sigma_a(v),g' = \sigma_a(v')$  satisfy  $\| g - g'\|_{H^1 ([d_1 + \varepsilon_3,d_2 - \varepsilon_3])} <   \varepsilon_2$ . As the embedding  $C^1 ([d_1 + \varepsilon_3,d_2 - \varepsilon_3])\to H^1 ([d_1 + \varepsilon_3,d_2 - \varepsilon_3])$  is continuous,
$$
\left\| \sigma_ {a} (v) \right\| _ {H ^ {1} ([ d _ {1}, d _ {1} + \varepsilon_ {3} ])} \leq c _ {a} \| v \| _ {C ^ {1, \alpha} (\overline {{D}})} \sqrt {\varepsilon_ {3}},
$$
$$
\left\| \sigma_ {a} (v) \right\| _ {H ^ {1} ([ d _ {2} - \varepsilon_ {3}, d _ {2} ])} \leq c _ {a} \| v \| _ {C ^ {1, \alpha} (\overline {{D}})} \sqrt {\varepsilon_ {3}},
$$
and  $\varepsilon_{2}$  and  $\varepsilon_{3}$  can be chosen to be arbitrarily small, we see that the property (F.23) follows. Thus the property (F.23) is shown in all cases.
By our assumptions  $\mathcal{Y} \subset \sigma_a(B_{C^{1,\alpha}(\overline{D})}(0,R))$  and hence  $g \in \mathcal{Y}$  implies that  $\| g\|_{C(\overline{D})} \leq AR$ . This implies that  $\mathcal{Y} \cap Z(i_1,i_2,\ldots ,i_{\ell_0})$  is empty if there is  $\ell$  such that  $|i_{\ell}| > 2AR / \varepsilon_{1} + 1$ . Thus, there is a finite set  $\mathcal{I} \subset \mathbb{Z}^{\ell_0}$  such that
$$
\mathcal {Y} \subset \bigcup_ {\vec {i} \in \mathcal {I}} Z (\vec {i}), \tag {F.32}
$$
$$
Z (\vec {i}) \cap \mathcal {Y} \neq \emptyset , \quad \text {f o r a l l} \vec {i} \in \mathcal {I}, \tag {F.33}
$$
where we use notation  $\vec{i} = (i_1, i_2, \ldots, i_{\ell_0}) \in \mathbb{Z}^{\ell_0}$ . On the other hand, we have chosen  $g_j \in \mathcal{V}$  such that  $B_{H^1(D)}(g_j, \varepsilon_0)$ ,  $j = 1, \ldots, J$  cover  $\mathcal{V}$ . This implies that for all  $\vec{i} \in \mathcal{I}$  there is  $j = j(\vec{i}) \in \{1, 2, \ldots, j\}$  such that there exists  $g \in Z(\vec{i}) \cap B_{H^1(D)}(g_j, \varepsilon_0)$ . By (F.23), this implies that
$$
Z (\vec {i}) \subset B _ {H ^ {1} (D)} \left(g _ {j (\vec {i})}, 2 \varepsilon_ {0}\right). \tag {F.34}
$$
Thus, we see that  $Z(\vec{i})$ ,  $\vec{i} \in \mathcal{I}$  is a disjoint covering of  $\mathcal{V}$ , and by (F.34), in each set  $Z(\vec{i}) \cap \mathcal{V}$ ,  $\vec{i} \in \mathcal{I}$  the map  $g \to \mathcal{R}_j(g)$  we have constructed a right inverse of the map  $F_1$ .
Below, we denote  $s(\vec{i},\ell) = i_{\ell}\varepsilon_1$ . Next we construct a partition of unity in  $\mathcal{V}$  using maps
$$
k _ {z, s, h} (x, y, v (x), w (y)) = v (x) \mathbf {1} _ {[ s - \frac {1}{2} h, s + \frac {1}{2} h)} (w (y)) \delta (y - z).
$$
$$
F _ {z, s, h} (v, w) (x) = \left\{ \begin{array}{c l} v (x), & \text {i f - \frac {1}{2} h \leq w (z) - s <   \frac {1}{2} h}, \\ 0, & \text {o t h e r w i s e}. \end{array} \right.
$$
Next, for all  $\vec{i} \in \mathcal{I}$  we define the operator  $\Phi_{\vec{i}}: H^{1}(D) \times \mathcal{Y} \to H^{1}(D)$ ,
$$
\Phi_ {\vec {i}} (v, w) = \pi_ {1} \circ \phi_ {\vec {i}, 1} \circ \phi_ {\vec {i}, 2} \circ \dots \circ \phi_ {\vec {i}, \ell_ {0}} (v, w),
$$
where  $\phi_{\vec{i},\ell}:H^{1}(D)\times \mathcal{V}\to H^{1}(D)\times \mathcal{V}$  are the maps
$$
\phi_ {\vec {i}, \ell} (v, w) = \left(F _ {y _ {\ell}, s (\vec {i}, \ell), \varepsilon_ {1}} (v, w), w\right),
$$
and  $\pi_1(v,w) = v$  maps a pair  $(v,w)$  to the first function  $v$ . It satisfies
$$
\Phi_ {\vec {i}} (v, w) = \left\{ \begin{array}{l l} v, & \text {i f - \frac {1}{2} \varepsilon_ {1} \leq w (y _ {\ell}) - s (\vec {i} , \ell) <   \frac {1}{2} \varepsilon_ {1} f o r a l l \ell}, \\ 0, & \text {o t h e r w i s e}. \end{array} \right.
$$
Observe that here  $s(\vec{i},\ell) = i_{\ell}\varepsilon_1$  is close to the value  $g_{j(\vec{i})}(y_{\ell})$ . Now we can write for  $g\in Y$
$$
F _ {1} ^ {- 1} (g) = \sum_ {\vec {i} \in \mathcal {I}} \Phi_ {\vec {i}} (\mathcal {R} _ {j (\vec {i})} (g), g),
$$
with suitably chosen  $j(\vec{i})\in \{1,2,\dots ,J\}$
Let us finally consider  $A_{u_0}^{-1}$  where  $u_0 \in C(\overline{D})$ . Let us denote
$$
\widetilde {K} _ {u _ {0}} w = \int_ {D} u _ {0} (y) \frac {\partial k}{\partial u} (x, y, u _ {0} (y)) w (y) d y,
$$
and  $J_{u_0} = K_{u_0} + \widetilde{K}_{u_0}$  be the integral operator with kernel
$$
j _ {u _ {0}} (x, y) = k (x, y, u _ {0} (y)) + u _ {0} (y) \frac {\partial k}{\partial u} (x, y, u _ {0} (y)).
$$
$$
(I + J _ {u _ {0}}) ^ {- 1} = I - J _ {u _ {0}} + J _ {u _ {0}} (I + J _ {u _ {0}}) ^ {- 1} J _ {u _ {0}},
$$
so that when we write the linear bounded operator
$$
A _ {u _ {0}} ^ {- 1} = \left(I + J _ {u _ {0}}\right) ^ {- 1}: H ^ {1} (D) \to H ^ {1} (D),
$$
as an integral operator
$$
(I + J _ {u _ {0}}) ^ {- 1} v (x) = v + \int_ {D} m _ {u _ {0}} (x, y) v (y) d y,
$$
$$
\begin{array}{l} (I + J _ {u _ {0}}) ^ {- 1} v (x) \\ = v (x) - J _ {u _ {0}} v (x) \\ + \int_ {D} \left(\int_ {D} \left\{j _ {u _ {0}} (x, y ^ {\prime}) j _ {u _ {0}} (y, y ^ {\prime}) + \left(\int_ {D} j _ {u _ {0}} (x, y ^ {\prime}) m _ {u _ {0}} \left(y ^ {\prime}, x ^ {\prime}\right) j _ {u _ {0}} \left(x ^ {\prime}, y\right) d x ^ {\prime}\right) \right\} d y ^ {\prime}\right) v (y) d y \\ = v (x) - \int_ {D} \widetilde {j} _ {u _ {0}} (x, y) v (y) d y, \\ \end{array}
$$
$$
\widetilde {j} _ {u _ {0}} (x, y) = - j _ {u _ {0}} (x, y) + \int_ {D} (j _ {u _ {0}} (x, y ^ {\prime}) j _ {u _ {0}} (y, y ^ {\prime}) d y ^ {\prime} + \int_ {D} \int_ {D} j _ {u _ {0}} (x, y ^ {\prime}) m _ {u _ {0}} (y ^ {\prime}, x ^ {\prime}) j _ {u _ {0}} (x ^ {\prime}, y) d x ^ {\prime} d y ^ {\prime}.
$$
This implies that the operator  $A_{u_0}^{-1} = (I + J_{u_0})^{-1}$  is a neural operator, too. Observe that  $\widetilde{j}_{u_0}(x,y), \partial_x \widetilde{j}_{u_0}(x,y) \in C(\overline{D} \times \overline{D})$ .
This proves Theorem 3.