SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\frac {\left\lfloor N ^ {1 / d} \right\rfloor^ {2} \left\lfloor L ^ {2 / d} \right\rfloor}{(N + 1) ^ {3 d n L}} \leq \frac {N ^ {2} L ^ {2}}{(N + 1) ^ {3 d n L}} \leq \frac {L ^ {2}}{(N + 1) ^ {3 d n L - 2}} \leq \frac {L ^ {2}}{2 ^ {d n L}} \leq 1.
$$
In the final of this subsection, we establish three lemmas for  $\{\Omega_{\pmb{m}}\}_{\pmb{m}\in \{1,2\}^{d}},\{g_{\pmb{m}}\}_{\pmb{m}\in \{1,2\}^{d}}$  and  $\{\phi_{\pmb{m}}\}_{\pmb{m}\in \{1,2\}^{d}}$  defined in Subsection 3.1.
Lemma 5. For  $\{\Omega_{m}\}_{m\in \{1,2\}^{d}}$  defined in Definition 5, we have
$$
\bigcup_ {\boldsymbol {m} \in \{1, 2 \} ^ {d}} \Omega_ {\boldsymbol {m}} = [ 0, 1 ] ^ {d}.
$$
Proof. We prove this lemma via induction.  $d = 1$  is valid due to  $\Omega_1 \cup \Omega_2 = [0,1]$ . Assume that the lemma is true for  $d - 1$ , then
$$
\begin{array}{l} \bigcup_ {\boldsymbol {m} \in \{1, 2 \} ^ {d}} \Omega_ {\boldsymbol {m}} = [ 0, 1 ] ^ {d} = \bigcup_ {\boldsymbol {m} \in \{1, 2 \} ^ {d - 1}} \Omega_ {\boldsymbol {m}} \times \Omega_ {1} + \bigcup_ {\boldsymbol {m} \in \{1, 2 \} ^ {d - 1}} \Omega_ {\boldsymbol {m}} \times \Omega_ {2} \\ = ([ 0, 1 ] ^ {d - 1} \times \Omega_ {1}) \bigcup ([ 0, 1 ] ^ {d - 1} \times \Omega_ {2}) = [ 0, 1 ] ^ {d}, \tag {40} \\ \end{array}
$$
hence the case of  $d$  is valid, and we finish the proof of the lemma.
Lemma 6.  $\{g_m\}_{m\in \{1,2\}^d}$  defined in Definition 6 satisfies:
(i):  $\sum_{\pmb{m}\in \{1,2\}^d}g_{\pmb{m}}(\pmb {x}) = 1$  for every  $x\in [0,1]^d$
(ii): supp  $g_{\pmb{m}}\cap [0,1]^d\subset \Omega_{\pmb{m}}$  , where  $\Omega_{\pmb{m}}$  is defined in Definition 5.
(ii): For any  $\mathbf{m} = (m_1, m_2, \ldots, m_d) \in \{1, 2\}^d$  and  $\mathbf{x} = (x_1, x_2, \ldots, x_d) \in [0, 1]^d \backslash \Omega_{\mathbf{m}}$ , there exists  $j$  such as  $g_{m_j}(x_j) = 0$  and  $\frac{\mathrm{d}g_{m_j}(x_j)}{\mathrm{d}x_j} = 0$ .
Proof. (i) can be proved via induction as Lemma 5, and we leave it to readers.
As for (ii) and (iii), without loss of generality, we show the proof for  $\pmb{m}_{*} := (1, 1, \dots, 1)$ . For any  $\pmb{x} \in [0, 1]^d \backslash \Omega_{\pmb{m}_*}$ , there is  $x_j \in [0, 1] \backslash \Omega_1$ . Then  $g_1(x_j) = 0$  and  $g_{\pmb{m}_*}(\pmb{x}) = \prod_{j=1}^d g_1(x_j) = 0$ , therefore  $\operatorname{supp} g_{\pmb{m}_*} \cap [0, 1]^d \subset \Omega_{\pmb{m}_*}$ . Furthermore,  $\frac{\mathrm{d}g_{m_j}(x_j)}{\mathrm{d}x_j} = 0$  for  $x_j \in [0, 1] \in \Omega_1$  due to the definition of  $g_1$  (Definition 6), then we finish this proof.
The following lemma demonstrates that  $\phi_{m}$ , as defined in Proposition 1, can restrict the Sobolev norm of the entire space to  $\Omega_{m}$ .
Lemma 7. For any  $\chi (\pmb {x})\in W^{1,\infty}((0,1)^d)$  , denote
$$
M = \max  \left\{\| \chi \| _ {W ^ {1, \infty} ((0, 1) ^ {d})}, \| \phi_ {\boldsymbol {m}} \| _ {W ^ {1, \infty} ((0, 1) ^ {d})} \right\},
$$
$$
\left\| \phi_ {\boldsymbol {m}} (\boldsymbol {x}) \cdot \chi (\boldsymbol {x}) \right\| _ {W ^ {1, \infty} ((0, 1) ^ {d})} = \left\| \phi_ {\boldsymbol {m}} (\boldsymbol {x}) \cdot \chi (\boldsymbol {x}) \right\| _ {W ^ {1, \infty} \left(\Omega_ {\boldsymbol {m}}\right)}
$$
$$
\left\| \phi_ {\boldsymbol {m}} (\boldsymbol {x}) \cdot \chi (\boldsymbol {x}) - \phi_ {M} \left(\phi_ {\boldsymbol {m}} (\boldsymbol {x}), \chi (\boldsymbol {x})\right) \right\| _ {W ^ {1, \infty} \left((0, 1) ^ {d}\right)} = \left\| \phi_ {\boldsymbol {m}} (\boldsymbol {x}) \cdot \chi (\boldsymbol {x}) - \phi_ {M} \left(\phi_ {\boldsymbol {m}} (\boldsymbol {x}), \chi (\boldsymbol {x})\right) \right\| _ {W ^ {1, \infty} \left(\Omega_ {\boldsymbol {m}}\right)} \tag {41}
$$
for any  $\pmb{m} \in \{1, 2\}^d$ , where  $\phi_{\pmb{m}}(\pmb{x})$  and  $\Omega_{\pmb{m}}$  is defined in Proposition 1 and Definition 5, and  $\phi_M$  is from Proposition 4 (choosing  $a = M$  in the proposition).
Proof. For the first equality, we only need to show that
$$
\left\| \phi_ {\boldsymbol {m}} (\boldsymbol {x}) \cdot \chi (\boldsymbol {x}) \right\| _ {W ^ {1, \infty} \left((0, 1) ^ {d} \backslash \Omega_ {\boldsymbol {m}}\right)} = 0. \tag {42}
$$
According to the Proposition 1, we have  $\phi_{\pmb{m}}(\pmb{x}) = \phi_{\mathrm{prod}}(g_{m_1}, g_{m_2}, \ldots, g_{m_d})$ , and for any  $\pmb{x} = (x_1, x_2, \ldots, x_d) \in (0, 1)^d \backslash \Omega_{\pmb{m}}$ , there is  $m_j$  such as  $g_{m_j}(x_j) = 0$  and  $\frac{\mathrm{d}g_{m_j}(x_j)}{\mathrm{d}x_j} = 0$  due to Lemma 6. Based on Eq. (31) in Proposition 5, we have
$$
\phi_ {\boldsymbol {m}} (\boldsymbol {x}) = \frac {\partial \phi_ {\boldsymbol {m}} (\boldsymbol {x})}{\partial x _ {s}} = 0, x \in (0, 1) ^ {d} \backslash \Omega_ {\boldsymbol {m}}, s \neq j.
$$
$$
\frac {\partial \phi_ {\boldsymbol {m}} (\boldsymbol {x})}{\partial x _ {j}} = \frac {\partial \phi_ {\operatorname {p r o d}} \left(g _ {m _ {1}} , g _ {m _ {2}} , \dots , g _ {m _ {d}}\right)}{\partial g _ {m _ {j}}} \frac {\mathrm {d} g _ {m _ {j}} \left(x _ {j}\right)}{\mathrm {d} x _ {j}} = 0. \tag {43}
$$
$$
\left| \phi_ {\boldsymbol {m}} (\boldsymbol {x}) \cdot \chi (\boldsymbol {x}) \right| + \sum_ {q = 1} ^ {d} \left| \frac {\partial \left[ \phi_ {\boldsymbol {m}} (\boldsymbol {x}) \cdot \chi (\boldsymbol {x}) \right]}{\partial x _ {q}} \right| = 0 \tag {44}
$$
for all  $\pmb{x} \in (0,1)^d \backslash \Omega_{\pmb{m}}$ .
Similarly, for the second equality in this lemma, we have
$$
\begin{array}{l} | \phi_ {M} (\phi_ {\boldsymbol {m}} (\boldsymbol {x}), \chi (\boldsymbol {x})) | + \sum_ {q = 1} ^ {d} \left| \frac {\partial [ \phi_ {M} (\phi_ {\boldsymbol {m}} (\boldsymbol {x}) , \chi (\boldsymbol {x})) ]}{\partial x _ {q}} \right| \\ = | \phi_ {M} (0, \chi (\boldsymbol {x})) | + \sum_ {q = 1} ^ {d} \left[ \left| \frac {\partial [ \phi_ {M} (0 , \chi (\boldsymbol {x})) ]}{\partial \chi (\boldsymbol {x})} \cdot \frac {\partial \chi (\boldsymbol {x})}{\partial x _ {q}} \right| + \left| \frac {\partial [ \phi_ {M} (\phi_ {\boldsymbol {m}} (\boldsymbol {x}) , \chi (\boldsymbol {x})) ]}{\partial \phi_ {\boldsymbol {m}} (\boldsymbol {x})} \cdot \frac {\partial \phi_ {\boldsymbol {m}} (\boldsymbol {x})}{\partial x _ {q}} \right| \right] \\ = 0, \tag {45} \\ \end{array}
$$
for all  $\pmb{x} \in (0,1)^{d} \backslash \Omega_{m}$  based on
$$
\phi_ {M} (0, y) = \frac {\partial \phi_ {M} (0 , y)}{\partial y} = 0, y \in (- M, M),
$$
and  $\frac{\partial\phi_m(\pmb{x})}{\partial x_q} = 0$  . Hence we finish our proof.
7.2.2 An approximation of functions in Sobolev spaces based on the Bramble-Hilbert Lemma [7, Lemma 4.3.8]
In this subsection, we establish  $\{f_{K,m}\}_{m\in \{1,2\}^d}$  as mentioned in Subsection 3.1, which is presented in Theorem 6. To prove this result, we build upon the work of [18], which leverages the average Taylor polynomials and the Bramble-Hilbert Lemma to approximate functions in Sobolev spaces.
Before we show Theorem 6, we define subsets of  $\Omega_{m}$  for simplicity notations.
For any  $m\in \{1,2\} ^d$  , we define
$$
\Omega_ {\boldsymbol {m}, \boldsymbol {i}} := [ 0, 1 ] ^ {d} \cap \prod_ {j = 1} ^ {d} \left[ \frac {2 i _ {j} - 1 _ {m _ {j} \leq 2}}{2 K}, \frac {3 + 4 i _ {j} - 2 \cdot 1 _ {m _ {j} \leq 2}}{4 K} \right] \tag {46}
$$
$\pmb{i} = (i_{1},i_{2},\dots ,i_{d})\in \{0,1\dots ,K\}^{d}$ , and it is easy to check  $\bigcup_{i\in \{0,1\dots ,K\}^d}\Omega_{\pmb {m},\pmb {i}} = \Omega_{\pmb{m}}$
Theorem 6. Let  $K \in \mathbb{N}_{+}$  and  $n \geq 2$ . Then for any  $f \in W^{n,\infty}((0,1)^d)$  with  $\| f \|_{W^{n,\infty}((0,1)^d)} \leq 1$  and  $m \in \{1,2\}^d$ , there exist piece-wise polynomials function  $f_{K,m} = \sum_{|\alpha| \leq n-1} g_{f,\alpha,m}(x) x^\alpha$  on  $\Omega_m$  (Definition 5) with the following properties:
$$
\begin{array}{l} \left\| f - f _ {K, \boldsymbol {m}} \right\| _ {W ^ {1, \infty} \left(\Omega_ {\boldsymbol {m}}\right)} \leq C _ {1} (n, d) K ^ {- (n - 1)}, \\ \left\| f - f _ {K, \boldsymbol {m}} \right\| _ {L ^ {\infty} \left(\Omega_ {\boldsymbol {m}}\right)} \leq C _ {1} (n, d) K ^ {- n}. \tag {47} \\ \end{array}
$$
Furthermore,  $g_{f,\alpha,m}(\pmb{x}) : \Omega_m \to \mathbb{R}$  is a constant function with on each  $\Omega_{m,i}$  for  $i \in \{0,1,\dots,K\}^d$ . And
$$
\left| g _ {f, \boldsymbol {\alpha}, \boldsymbol {m}} (\boldsymbol {x}) \right| \leq C _ {2} (n, d) \tag {48}
$$
for all  $\pmb{x} \in \Omega_{m}$ , where  $C_1$  and  $C_2$  are constants independent with  $K$ .
This proof is similar to that of [18, Lemma C.4.], but we provide detailed proof as follows for readability. Before the proof, we must introduce the partition of unity, average Taylor polynomials, and a lemma.
Definition 7 (The partition of unity). Let  $d, K \in \mathbb{N}_{+}$ , then
$$
\Psi = \left\{h _ {i}: i \in \{0, 1, \dots , K \} ^ {d} \right\}
$$
with  $h_i: \mathbb{R}^d \to \mathbb{R}$  for all  $i \in \{0, 1, \ldots, K\}^d$  is called the partition of unity  $[0, 1]^d$  if it satisfies
(i):  $0 \leq h_{i}(\pmb{x}) \leq 1$  for every  $h_{i} \in \Psi$ .
(ii):  $\sum_{h_i\in \Psi}h_i = 1$  for every  $x\in [0,1]^d$
Definition 8. Let  $n \geq 1$  and  $f \in W^{n,\infty}((0,1)^d)$ ,  $\pmb{x}_0 \in ((0,1)^d)$  and  $r > 0$  such that for the ball  $B(\pmb{x}_0) := B(\pmb{x}_0)_{r,|\cdot|}$  which is a compact subset of  $((0,1)^d)$ . The corresponding Taylor polynomial of order  $n$  of  $f$  averaged over  $B$  is defined for
$$
Q ^ {n} f (x) := \int_ {B} T _ {\boldsymbol {y}} ^ {n} f (\boldsymbol {x}) b _ {r} (\boldsymbol {y}) \mathrm {d} \boldsymbol {y} \tag {49}
$$
$$
\begin{array}{l} T _ {\boldsymbol {y}} ^ {n} f (\boldsymbol {x}) := \sum_ {| \boldsymbol {\alpha} | \leq n - 1} \frac {1}{\boldsymbol {\alpha} !} D ^ {\boldsymbol {\alpha}} f (\boldsymbol {y}) (\boldsymbol {x} - \boldsymbol {y}) ^ {\boldsymbol {\alpha}}, \\ b _ {r} (\boldsymbol {x}) := \left\{ \begin{array}{l l} \frac {1}{c _ {r}} e ^ {- \left(1 - (| \boldsymbol {x} - \boldsymbol {x} _ {0} | / r) ^ {2}\right) ^ {- 1}}, & | \boldsymbol {x} - \boldsymbol {x} _ {0} | <   r, \\ 0, & | \boldsymbol {x} - \boldsymbol {x} _ {0} | \leq r, \end{array} \right. \\ c _ {r} = \int_ {\mathbb {R} ^ {d}} e ^ {- \left(1 - \left(\left| \boldsymbol {x} - \boldsymbol {x} _ {0} \right| / r\right) ^ {2}\right) ^ {- 1}} \mathrm {d} x. \tag {50} \\ \end{array}
$$
Lemma 8. Let  $n \geq 1$  and  $f \in W^{n,\infty}((0,1)^d)$ ,  $\pmb{x}_0 \in \Omega$  and  $r > 0$  such that for the ball  $B(\pmb{x}_0) \coloneqq B_{r,|\cdot|}(\pmb{x}_0)$  which is a compact subset of  $((0,1)^d)$ . The corresponding Taylor polynomial of order  $n$  of  $f$  averaged over  $B$  can be read as
$$
Q ^ {n} f (\boldsymbol {x}) = \sum_ {| \boldsymbol {\alpha} | \leq n - 1} c _ {f, \boldsymbol {\alpha}} \boldsymbol {x} ^ {\boldsymbol {\alpha}}.
$$
$$
\left| c _ {f, \alpha} \right| \leq C _ {2} (n, d) \| f \| _ {W ^ {n - 1, \infty} (B)}. \tag {51}
$$
where  $C_2(n,d) = \sum_{|\pmb {\alpha} + \pmb {\beta}|\leq n - 1}\frac{1}{\pmb{\alpha}! \pmb{\beta}!}$ .
Proof. Based on [18, Lemma B.9.],  $Q^n f(x)$  can be read as
$$
Q ^ {n} f (\boldsymbol {x}) = \sum_ {| \boldsymbol {\alpha} | \leq n - 1} c _ {f, \boldsymbol {\alpha}} \boldsymbol {x} ^ {\boldsymbol {\alpha}} \tag {52}
$$
$$
c _ {f, \alpha} = \sum_ {| \alpha + \beta | \leq n - 1} \frac {1}{(\beta + \alpha) !} a _ {\beta + \alpha} \int_ {B} D ^ {\alpha + \beta} f (\boldsymbol {x}) \boldsymbol {y} ^ {\beta} b _ {r} (\boldsymbol {y}) d \boldsymbol {y} \tag {53}
$$
for  $a_{\beta + \alpha} \leq \frac{(\alpha + \beta)!}{\alpha! \beta!}$ . Note that
$$
\left| \int_ {B} D ^ {\boldsymbol {\alpha} + \beta} f (\boldsymbol {x}) \boldsymbol {y} ^ {\boldsymbol {\beta}} b _ {r} (\boldsymbol {y}) \mathrm {d} \boldsymbol {y} \right| \leq \| f \| _ {W ^ {n - 1, \infty} (B)} \| b _ {r} (x) \| _ {L ^ {1} (B)} = \| f \| _ {W ^ {n - 1, \infty} (B)}. \tag {54}
$$
$$
\left| c _ {f, \alpha} \right| \leq C _ {2} (n, d) \| f \| _ {W ^ {n - 1, \infty} \left(B _ {m, N}\right)}. \tag {55}
$$
where  $C_2(n,d) = \sum_{|\alpha +\beta |\leq n - 1}\frac{1}{\alpha!\beta!}$ .
The proof of Theorem 6 is based on average Taylor polynomials and the Bramble-Hilbert Lemma [7, Lemma 4.3.8].
Definition 9. Let  $\Omega$ ,  $B \in \mathbb{R}^d$ . Then  $\Omega$  is called stared-shaped with respect to  $B$  if
$$
\overline {{c o n v}} \left(\left\{\boldsymbol {x} \right\} \cup B \subset \Omega\right), f o r a l l \boldsymbol {x} \in \Omega .
$$
Definition 10. Let  $\Omega \in \mathbb{R}^d$  be bounded, and define
$$
\mathcal {R} := \left\{r > 0: \begin{array}{l} \text {t h e r e e x i s t s} \boldsymbol {x} _ {0} \in \Omega \text {s u c h t h a t} \Omega \text {i s} \\ \text {s t a r - s h a p e d w i t h r e s p e c t t o} B _ {r, | \cdot |} \left(\boldsymbol {x} _ {0}\right) \end{array} \right\}.
$$
$$
r _ {\max } ^ {\star} := \sup  \mathcal {R} \quad a n d c a l l \quad \gamma := \frac {\operatorname {d i a m} (\Omega)}{r _ {\max } ^ {\star}}
$$
the chunkiness parameter of  $\Omega$  if  $\mathcal{R} \neq \emptyset$ .
Lemma 9 (Bramble-Hilbert Lemma [7, Lemma 4.3.8]). Let  $\Omega \in \mathbb{R}^d$  be open and bounded,  $\pmb{x}_0 \in \Omega$  and  $r > 0$  such that  $\Omega$  is the stared-shaped with respect to  $B := B_{r,\| \cdot \|}(\pmb{x}_0)$ , and  $r \geq \frac{1}{2} r_{\max}^{\star}$ . Moreover, let  $n \in \mathbb{N}_{+}$ ,  $1 \leq p \leq \infty$  and denote by  $\gamma$  by the chunkiness parameter of  $\Omega$ . Then there is a constant  $C(n,d,\gamma) > 0$  such that for all  $f \in W^{n,p}(\Omega)$
$$
\left| f - Q ^ {n} f \right| _ {W ^ {k, p} (\Omega)} \leq C (n, d, \gamma) h ^ {n - k} | f | _ {W ^ {n, p} (\Omega)} \quad f o r k = 0, 1, \dots , n
$$
where  $Q^n f$  denotes the Taylor polynomial of order  $n$  of  $f$  averaged over  $B$  and  $h = \mathrm{diam}(\Omega)$ .
Proof of Theorem 6. Without loss of generalization, we prove the case for  $m = (1,1,\dots ,1) = :m_{*}$
Denote  $E: W^{n,\infty}((0,1)^d) \to W^{n,\infty}(\mathbb{R}^d)$  be an extension operator [43] and set  $\tilde{f} \coloneqq Ef$  and  $C_E$  is the norm of the extension operator.
Define  $p_{f,i}$  as the average Taylor polynomial Definition 8 in  $B_{i,K} \coloneqq B_{\frac{1}{4K},|\cdot|}\left(\frac{8i + 3}{8K}\right)$  i.e.
$$
p _ {f, \boldsymbol {i}} := \int_ {B _ {\boldsymbol {i}, K}} T _ {\boldsymbol {y}} ^ {n} \tilde {f} (\boldsymbol {x}) b _ {\frac {1}{4 K}} (\boldsymbol {y}) d \boldsymbol {y}. \tag {56}
$$
Based on Lemma 8,  $p_{f,i}$  can be read as
$$
p _ {f, i} = \sum_ {| \boldsymbol {\alpha} | \leq n - 1} c _ {f, i, \boldsymbol {\alpha}} \boldsymbol {x} ^ {\boldsymbol {\alpha}} \tag {57}
$$
$$
\left| c _ {f, i, \alpha} \right| \leq C _ {2} (n, d). \tag {58}
$$
The reason to define average Taylor polynomial on  $B_{i,K}$  is to use the Bramble-Hilbert Lemma 9 on
$$
\Omega_ {\pmb {m} _ {*}, \pmb {i}} = B _ {\frac {3}{8 K}, \| \cdot \| _ {\ell_ {\infty}}} \left(\frac {8 \pmb {i} + 3}{8 K}\right) = \prod_ {j = 1} ^ {d} \left[ \frac {i _ {j}}{K}, \frac {3 + 4 i _ {j}}{4 K} \right].
$$
$$
\frac {1}{4 K} \geq \frac {1}{2} \cdot \frac {3}{8 K} = \frac {1}{2} r _ {\max } ^ {\star} \left(\Omega_ {\boldsymbol {m} _ {*}, i}\right), \gamma \left(\Omega_ {\boldsymbol {m} _ {*}, i}\right) = \frac {\operatorname {d i a m} \left(\Omega_ {\boldsymbol {m} _ {*} , i}\right)}{r _ {\max } ^ {\star} \left(\Omega_ {\boldsymbol {m} _ {*} , i}\right)} = 2 \sqrt {d}.
$$
Therefore we can apply the Bramble-Hilbert Lemma 9 and have
$$
\| \tilde {f} - p _ {f, i} \| _ {L ^ {\infty} \left(\Omega_ {m _ {*}, i}\right)} \leq C _ {B H} (n, d) K ^ {- n}
$$
$$
| \tilde {f} - p _ {f, i} | _ {W ^ {1, \infty} \left(\Omega_ {\boldsymbol {m} _ {*}, i}\right)} \leq C _ {B H} (n, d) K ^ {- (n - 1)} \tag {59}
$$
where  $C_{BH}(n,d) = |\{| \alpha | = n\} | \frac{1}{d \int_0^1 x^{d-1} e^{-(1-x^2)^{-1}} \, \mathrm{d}x} \left(2 + 4\sqrt{d}\right)^d C_E$  by following the proof of Lemma [7, Lemma 4.3.8]. Therefore,
$$
\| \tilde {f} - p _ {f, i} \| _ {W ^ {1, \infty} \left(\Omega_ {\boldsymbol {m} _ {*}, i}\right)} \leq C _ {1} (n, d) K ^ {- (n - 1)}
$$
where  $C_1(n,d) = 2C_{BH}(n,d)$ .
Now we construct a partition of unity that we use in this theorem. First of all, given any integer  $K$ , define  $\{h_i\}_{i=0}^K$  from  $\mathbb{R} \to \mathbb{R}$ :
$$
h _ {i} (x) := h \left(4 K \left(x - \frac {8 i + 3}{8 K}\right)\right), h (x) := \left\{ \begin{array}{l l} 1, & | x | <   \frac {3}{2} \\ 0, & | x | > 2 \\ 4 - 2 | x |, & \frac {3}{2} \leq | x | \leq 2. \end{array} \right. \tag {60}
$$
It is easy to check that  $\{h_i\}_{i=0}^K$  is a partition of unity of  $[0,1]$  and  $h_i(x) = 1$  for  $x \in \left[\frac{i}{K}, \frac{3 + 4i}{4K}\right]$ . Hence we can define  $h_i(\pmb{x})$  for  $\pmb{i} = (i_1, i_2, \dots, i_d) \in \{0,1,\dots,K\}^d$  and  $\pmb{x} = (x_1, x_2, \dots, x_d) \in \mathbb{R}^d$ :
$$
h _ {\boldsymbol {i}} (\boldsymbol {x}) = \prod_ {j = 1} ^ {d} h _ {i _ {j}} \left(x _ {j}\right), \tag {61}
$$
and  $\left\{h_{i}:\boldsymbol {i}\in \{0,1,\ldots ,K\}^{d}\right\}$  is a partition of unity of  $[0,1]^d$  and  $h_i(\pmb {x}) = 1$  for  $\pmb {x}\in$ $\prod_{j = 1}^{d}\left[\frac{i_j}{K},\frac{3 + 4i_j}{4K}\right] = \Omega_{\pmb {m}_*,i}$  and  $\pmb {i} = (i_1,i_2,\dots ,i_d)\in \{0,1,\dots ,K\} ^d.$
$$
\left\| h _ {i} \left(\tilde {f} - p _ {f, i}\right) \right\| _ {L ^ {\infty} \left(\Omega_ {m _ {*}, i}\right)} \leq \left\| \tilde {f} - p _ {f, i} \right\| _ {L ^ {\infty} \left(\Omega_ {m _ {*}, i}\right)} \leq C _ {B H} (n, d) K ^ {- n} \tag {62}
$$
$$
\left| h _ {i} \left(\tilde {f} - p _ {f, i}\right) \right| _ {W ^ {1, \infty} \left(\Omega_ {\boldsymbol {m} _ {*}, i}\right)} \leq \left| \tilde {f} - p _ {f, i} \right| _ {W ^ {1, \infty} \left(\Omega_ {\boldsymbol {m} _ {*}, i}\right)} \leq C _ {B H} (n, d) K ^ {- (n - 1)} \tag {63}
$$
which is due to  $h_i = 1$  on  $\Omega_{\pmb{m}_{*},i}$
$$
\| h _ {i} (\tilde {f} - p _ {f, i}) \| _ {W ^ {1, \infty} (\Omega_ {m _ {*}, i})} \leq C _ {1} (n, d) K ^ {- (n - 1)}.
$$
$$
\begin{array}{l} \left\| f - \sum_ {\boldsymbol {i} \in \{0, 1, \dots , K \} ^ {d}} h _ {\boldsymbol {i}} p _ {f, \boldsymbol {i}} \right\| _ {W ^ {1, \infty} \left(\Omega_ {m _ {*}}\right)} \leq \max  _ {\boldsymbol {i} \in \{0, 1, \dots , K \} ^ {d}} \| h _ {\boldsymbol {i}} (\tilde {f} - p _ {f, \boldsymbol {i}}) \| _ {W ^ {1, \infty} \left(\Omega_ {m _ {*} *}, \boldsymbol {i}\right)} \\ \leq C _ {1} (n, d) K ^ {- (n - 1)}, \tag {64} \\ \end{array}
$$
which is due to  $\cup_{i\in \{0,1,\dots ,K\}^d}\Omega_{m_*,i} = \Omega_{m_*}$  and supp  $h_i\cap \Omega_{m_*} = \Omega_{m_*},i$
$$
\left\| f - \sum_ {\boldsymbol {i} \in \{0, 1, \dots , K \} ^ {d}} h _ {\boldsymbol {i}} p _ {f, \boldsymbol {i}} \right\| _ {L ^ {\infty} \left(\Omega_ {1, d}\right)} \leq C _ {1} (n, d) K ^ {- n}. \tag {65}
$$
Last of all,
$$
\begin{array}{l} f _ {k, \boldsymbol {m} _ {*}} (\boldsymbol {x}) := \sum_ {\boldsymbol {i} \in \{0, 1, \dots , K \} ^ {d}} h _ {\boldsymbol {i}} p _ {f, \boldsymbol {i}} = \sum_ {\boldsymbol {i} \in \{0, 1, \dots , K \} ^ {d}} \sum_ {| \boldsymbol {\alpha} | \leq n - 1} h _ {\boldsymbol {i}} c _ {f, \boldsymbol {i}, \boldsymbol {\alpha}} \boldsymbol {x} ^ {\boldsymbol {\alpha}} \\ = \sum_ {| \boldsymbol {\alpha} | \leq n - 1} \sum_ {i \in \{0, 1, \dots , K \} ^ {d}} h _ {i} c _ {f, i, \boldsymbol {\alpha}} \boldsymbol {x} ^ {\boldsymbol {\alpha}} \\ =: \sum_ {| \boldsymbol {\alpha} | \leq n - 1} g _ {f, \boldsymbol {\alpha}, \boldsymbol {m} _ {*}} (\boldsymbol {x}) \boldsymbol {x} ^ {\boldsymbol {\alpha}} \tag {66} \\ \end{array}
$$
with  $|g_{f,\alpha,\mathbf{m}_*}(\mathbf{x})| \leq C_2(n,d)$  for  $x \in \Omega_{\mathbf{m}_*}$ . Note that  $g_{f,\alpha,\mathbf{m}_*}(\mathbf{x})$  is a step function from  $\Omega_{\mathbf{m}_*} \to \mathbb{R}$ :
$$
g _ {f, \boldsymbol {\alpha}, \boldsymbol {m} _ {*}} (\boldsymbol {x}) = c _ {f, i, \boldsymbol {\alpha}} \tag {67}
$$
for  $\pmb{x} \in \prod_{j=1}^{d}\left[\frac{i_j}{K}, \frac{3+4i_j}{4K}\right]$  and  $\pmb{i} = (i_1, i_2, \dots, i_d)$  since  $h_i(\pmb{x}) = 0$  for  $\pmb{x} \in \Omega_{\pmb{m}_*} \setminus \prod_{j=1}^{d}\left[\frac{i_j}{K}, \frac{3+4i_j}{4K}\right]$  and  $h_i(\pmb{x}) = 1$  for  $\pmb{x} \in \prod_{j=1}^{d}\left[\frac{i_j}{K}, \frac{3+4i_j}{4K}\right]$ .
7.2.3 Approximation of functions in  $W^{n,\infty}$  with  $W^{1,\infty}$  norm by ReLU neural networks in the whole space except a small set
Theorem 7. For any  $f \in W^{n,\infty}((0,1)^d)$  with  $\| f\|_{W^{n,\infty}((0,1)^d)} \leq 1$ , any  $N,L \in \mathbb{N}_{+}$ , and  $m = (m_1,m_2,\ldots ,m_d) \in \{1,2\}^d$ , there is a neural network  $\psi_{m}$  with the width  $25n^{d + 1}(N + 1)\log_2(8N)$  and depth  $27n^{2}(L + 2)\log_{2}(4L)$  such that
$$
\begin{array}{l} \| f (\boldsymbol {x}) - \psi_ {\boldsymbol {m}} (\boldsymbol {x}) \| _ {W ^ {1, \infty} \left(\Omega_ {\boldsymbol {m}}\right)} \leq C _ {6} (n, d) N ^ {- 2 (n - 1) / d} L ^ {- 2 (n - 1) / d} \\ \left\| f (\boldsymbol {x}) - \psi_ {\boldsymbol {m}} (\boldsymbol {x}) \right\| _ {L ^ {\infty} \left(\Omega_ {\boldsymbol {m}}\right)} \leq C _ {6} (n, d) N ^ {- 2 n / d} L ^ {- 2 n / d}, \tag {68} \\ \end{array}
$$
where  $C_6$  is the constant independent with  $N, L$ .
Proof. Without loss of the generalization, we consider the case for  $m_* = (1, 1, \ldots, 1)$ . Due to Theorem 6 and setting  $K = \lfloor N^{1/d} \rfloor^2 \lfloor L^{2/d} \rfloor$ , we have
$$
\left\| f - f _ {K, \boldsymbol {m} _ {*}} \right\| _ {W ^ {1, \infty} \left(\Omega_ {\boldsymbol {m} _ {*}}\right)} \leq C _ {1} (n, d) K ^ {- (n - 1)} \leq C _ {1} (n, d) N ^ {- 2 (n - 1) / d} L ^ {- 2 (n - 1) / d}
$$
$$
\left\| f - f _ {K, \boldsymbol {m} _ {*}} \right\| _ {L ^ {\infty} \left(\Omega_ {\boldsymbol {m} _ {*}}\right)} \leq C _ {1} (n, d) K ^ {- n} \leq C _ {1} (n, d) N ^ {- 2 n / d} L ^ {- 2 n / d}, \tag {69}
$$
where  $f_{K,\pmb{m}_{*}} = \sum_{|\alpha |\leq n - 1}g_{f,\pmb{\alpha},\pmb{m}_{*}}(\pmb {x})\pmb{x}^{\alpha}$  for  $x\in \Omega_{\pmb{m}_*}$ . Note that  $g_{f,\pmb {\alpha},\pmb{m}_{*}}(\pmb {x})$  is a constant function for  $\pmb {x}\in \prod_{j = 1}^{d}\left[\frac{i_j}{K},\frac{3 + 4i_j}{4K}\right]$  and  $\pmb {i} = (i_1,i_2,\dots ,i_d)\in \{0,1,\dots ,K - 1\} ^d$ . The remaining part is to approximate  $f_{K,\pmb{m}_{*}}$  by neural networks.
The way to approximate  $g_{f,\alpha,m_*(x)}(x)$  is similar with [23, Theorem 3.1]. First of all, due to Proposition 2, there is a neural network  $\phi_1(x)$  with the width  $4N + 5$  and depth  $4L + 4$  such that
$$
\phi (x) = k, x \in \left[ \frac {k}{K}, \frac {k + 1}{K} - \frac {1}{4 K} \right], k = 0, 1, \dots , K - 1. \tag {70}
$$
Note that we choose  $\delta = \frac{1}{4K} \leq \frac{1}{3K}$  in Proposition 2. Then define
$$
\boldsymbol {\phi} _ {2} (\boldsymbol {x}) = \left[ \frac {\phi_ {1} (x _ {1})}{K}, \frac {\phi_ {1} (x _ {2})}{K}, \dots , \frac {\phi_ {1} (x _ {d})}{K} \right] ^ {\intercal}.
$$
For each  $p = 0,1,\ldots ,K^d -1$  , there is a bijection
$$
\boldsymbol {\eta} (p) = \left[ \eta_ {1}, \eta_ {2}, \dots , \eta_ {d} \right] \in \{0, 1, \dots , K - 1 \} ^ {d}
$$
such that  $\sum_{j=1}^{d} \eta_j K^{j-1} = p$ . Then define
$$
\xi_ {\boldsymbol {\alpha}, p} = \frac {g _ {f , \boldsymbol {\alpha} , \boldsymbol {m} _ {*}} \left(\frac {\boldsymbol {\eta} (p)}{K}\right) + C _ {2} (n , d)}{2 C _ {2} (n , d)} \in [ 0, 1 ],
$$
where  $C_2(n,d)$  is the bounded of  $g_{f,\alpha,m_*}$  defined in Theorem 6. Therefore, based on Proposition 3, there is a neural network  $\tilde{\phi}_{\alpha}(x)$  with the width  $16n(N + 1)\log_2(8N)$  and depth  $(5L + 2)\log_2(4L)$  such that  $|\tilde{\phi}_{\alpha}(p) - \xi_{\alpha,p}| \leq N^{-2n}L^{-2n}$  for  $p = 0,1,\ldots,K^d - 1$ . Denote
$$
\phi_ {\boldsymbol {\alpha}} (\boldsymbol {x}) = 2 C _ {2} (n, d) \tilde {\phi} _ {\boldsymbol {\alpha}} \left(\sum_ {j = 1} ^ {d} x _ {j} K ^ {j}\right) - C _ {2} (n, d)
$$
and obtain that
$$
\left| \phi_ {\boldsymbol {\alpha}} \left(\frac {\boldsymbol {\eta} (p)}{K}\right) - g _ {f, \boldsymbol {\alpha}, \boldsymbol {m} _ {*}} \left(\frac {\boldsymbol {\eta} (p)}{K}\right) \right| = 2 C _ {2} (n, d) | \tilde {\phi} _ {\boldsymbol {\alpha}} (p) - \xi_ {\boldsymbol {\alpha}, p} | \leq 2 C _ {2} (n, d) N ^ {- 2 n} L ^ {- 2 n}.
$$
Then we obtain that