SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\left(\widetilde{T_{\mathbf{k}}}\circ \varphi_{n}^{-1}\right)(z) = \sum_{\substack{\boldsymbol{\ell}^{1},\boldsymbol{\ell}^{2}\in \mathbb{N}_{0}^{n}\\ \boldsymbol{\ell}^{1},\boldsymbol{\ell}^{2}\leq 2M}}b_{\boldsymbol{\ell}^{1},\boldsymbol{\ell}^{2}}^{\mathbf{k}}z^{\boldsymbol{\ell}^{1}}\overline{z}^{\boldsymbol{\ell}^{2}}
$$
with complex coefficients  $b_{\ell^1,\ell^2}^{\mathbf{k}} \in \mathbb{C}$ . Using Theorem 3.1, we choose  $\rho_1, \ldots, \rho_m \in \mathbb{C}^n$  and  $b \in \mathbb{C}$ , such that for any polynomial  $P \in \left\{\widetilde{T_{\mathbf{k}}} \circ \varphi_n^{-1} : \mathbf{k} \leq M\right\} \subseteq \mathcal{P}_{2M}^n$  there exist coefficients  $\sigma_1(P), \ldots, \sigma_m(P) \in \mathbb{C}$ , such that
$$
\left\| g _ {P} - P \right\| _ {L ^ {\infty} \left(\Omega_ {n}; \mathbb {C}\right)} \leq \left(\sum_ {\mathbf {k} \in \mathbb {N} _ {0} ^ {2 n}} \nu^ {- \mathbf {k}}\right) ^ {- 1} \cdot \nu^ {- (M + 1)}, \tag {D.2}
$$
Note that here we implicitly use the bound  $(4\cdot (2M) + 1)^{2n}\leq m$ . We are now going to show that the first layer  $\Phi \in \mathcal{F}_{n,m}^{\phi}$  defined using the  $\rho_{t}$  and  $b$  (i.e.,  $\Phi (z) = (\phi (\rho_t^T z + b))_{t = 1}^m)$  has the desired property.
To this end, take an arbitrary function  $f \in \mathcal{W}_n(\nu)$ . Proposition D.4 tells us that
$$
\| f - P _ {M} \circ \varphi_ {n} ^ {- 1} \| _ {L ^ {\infty} \left(\Omega_ {n}; \mathbb {C}\right)} = \| f \circ \varphi_ {n} - P _ {M} \| _ {L ^ {\infty} \left([ - 1, 1 ] ^ {2 n}; \mathbb {C}\right)} \leq c _ {1} \cdot \nu^ {- (M + 1)} \cdot \| f \circ \varphi_ {n} \| _ {\mathcal {V} _ {2 n} (\nu)}, \tag {D.3}
$$
$$
P_{M}\circ \varphi_{n}^{-1} = \sum_{\substack{\mathbf{k}\in \mathbb{N}_{0}^{2n}\\ \mathbf{k}\leq M}}\langle f\circ \varphi_{n},\widetilde{T_{\mathbf{k}}} \rangle_{\mu_{2n}}\cdot (\widetilde{T_{\mathbf{k}}}\circ \varphi_{n}^{-1}).
$$
We then define the approximating network
$$
g:= \sum_{\substack{\mathbf{k}\in \mathbb{N}_{0}^{2n}\\ \mathbf{k}\leq M}}\langle f\circ \varphi_{n},\widetilde{T_{\mathbf{k}}}\rangle_{\mu_{2n}}\cdot g_{\widetilde{T_{\mathbf{k}}}\circ \varphi_{n}^{-1}}.
$$
From the definition of the functions  $g_{\widetilde{T_{\mathbf{k}} \circ \varphi_{n}^{-1}}}$  it follows directly that  $g$  is a shallow CVNN with first layer  $\Phi \in \mathcal{F}_{n,m}^{\phi}$ . Furthermore, it holds
$$
\begin{array}{l} \| P_{M}\circ \varphi_{n}^{-1} - g\|_{L^{\infty}(\Omega_{n};\mathbb{C})}\leq \sum_{\substack{\mathbf{k}\in \mathbb{N}_{0}^{2n}\\ \mathbf{k}\leq M}}|\langle f\circ \varphi_{n},\widetilde{T_{\mathbf{k}}}\rangle_{\mu_{2n}}|\cdot \| \widetilde{T_{\mathbf{k}}}\circ \varphi_{n}^{-1} - g_{\widetilde{T_{\mathbf{k}}}\circ \varphi_{n}^{-1}}\|_{L^{\infty}(\Omega_{n};\mathbb{C})} \\ \stackrel {\text{(D.2),Lem. D.3}}{\leq} 2^{n}\cdot \| f\circ \varphi_{n}\|_{\mathcal{V}_{2n}(\boldsymbol {\nu})}\cdot \sum_{\substack{\mathbf{k}\in \mathbb{N}_{0}^{2n}\\ \mathbf{k}\leq M}}\boldsymbol{\nu}^{-\mathbf{k}}\cdot \left(\sum_{\mathbf{k}\in \mathbb{N}_{0}^{n}}\boldsymbol{\nu}^{-\mathbf{k}}\right)^{-1}\cdot \boldsymbol{\nu}^{-(M + 1)} \\ \leq 2 ^ {n} \cdot \| f \| _ {\mathcal {W} _ {n} (\nu)} \cdot \nu^ {- (M + 1)}. \\ \end{array}
$$
Combining this estimate with Equation (D.3) and applying the triangle inequality gives us
$$
\left\| f - g \right\| _ {L ^ {\infty} \left(\Omega_ {n}; \mathbb {C}\right)} \leq \left(c _ {1} + 2 ^ {n}\right) \cdot \nu^ {- (M + 1)} \cdot \left\| f \right\| _ {L ^ {\infty} \left(\mathcal {E} _ {\nu}; \mathbb {C}\right)}.
$$
Hence, the claim follows taking  $c \coloneqq c_1 + 2^n$  and using (D.1).
The continuity of the linear map  $f \mapsto \sigma(f)$  with respect to the  $L^{\infty}$ -norm follows directly from the fact that  $f \mapsto \langle f \circ \varphi_n, \widetilde{T_{\mathbf{k}}} \rangle_{\mu_{2n}}$  is trivially continuous with respect to the  $L_{\mu_{2n}}^{2}$ -norm and hence also continuous with respect to the  $L^{\infty}$ -norm since  $\mu_{2n}$  is a probability measure.
E Postponed proofs for the optimality results in the case of continuous weight selection
In this section we provide the proofs for the optimality results derived if a continuous weight selection is assumed. Specifically, we prove Theorems 4.1 and 5.1. The proofs of both results rely on a very general result about the approximation in normed spaces by subsets that are parametrizable with  $m$  parameters [19]. We decided to include a detailed proof for this approximation result in this paper since the nature of the continuity assumption in [19] is not completely clear.
Proposition E.1 ([19, Theorem 3.1]). Let  $(X,\| \cdot \|_{X})$  be a normed space,  $\varnothing \neq K\subseteq X$  a subset and  $V\subseteq X$  a linear, not necessarily closed subspace of  $X$  containing  $K$ . Let  $m\in \mathbb{N}$ , let  $\overline{a}:K\to \mathbb{R}^m$  be a map which is continuous with respect to some norm  $\| \cdot \| _V$  on  $V$  and  $M:\mathbb{R}^m\to X$  some arbitrary map. Let
$$
b _ {m} (K) _ {X} := \sup  _ {X _ {m + 1}} \sup  \left\{\varrho \geq 0: U _ {\varrho} \left(X _ {m + 1}\right) \subseteq K \right\}, \tag {E.1}
$$
where the first supremum is taken over all  $(m + 1)$ -dimensional linear subspaces  $X_{m + 1}$  of  $X$  and
$$
U _ {\varrho} \left(X _ {m + 1}\right) := \left\{y \in X _ {m + 1}: \| y \| _ {X} \leq \varrho \right\}.
$$
Further, we set  $b_{m}(K)_{X} \coloneqq 0$  if the supremum in (E.1) is not well-defined as a quantity in  $[0,\infty]$ . Then it holds
$$
\sup  _ {x \in K} \| x - M (\bar {a} (x)) \| _ {X} \geq b _ {m} (K) _ {X}.
$$
Proof. The claim is trivial if  $b_{m}(K)_{X} = 0$ . Thus, assume  $b_{m}(K)_{X} > 0$ . Let  $0 < \varrho \leq b_{m}(K)_{X}$  be any number such that there exists an  $(m + 1)$ -dimensional subspace  $X_{m + 1}$  of  $X$  with  $U_{\varrho}(X_{m + 1}) \subseteq K$ . It follows  $U_{\varrho}(X_{m + 1}) \subseteq V$ , hence  $X_{m + 1} \subseteq V$ , so  $\| \cdot \|_V$  defines a norm on  $X_{m + 1}$ . Thus, the
restriction of  $\overline{a}$  to  $\partial U_{\varrho}(X_{m + 1})$  is a continuous mapping to  $\mathbb{R}^m$  with respect to  $\| \cdot \| _V$ . Since all norms are equivalent on the finite-dimensional space  $X_{m + 1}$ , the Borsuk-Ulam-Theorem [18, Corollary 4.2] yields the existence of a point  $x_0\in \partial U_{\varrho}(X_{m + 1})$  with  $\bar{a} (x_0) = \bar{a} (-x_0)$ . We then see
$$
\begin{array}{l} 2 \varrho = 2 \| x _ {0} \| _ {X} \leq \| x _ {0} - M (\bar {a} (x _ {0})) \| _ {X} + \| x _ {0} + M (\bar {a} (- x _ {0})) \| _ {X} \\ = \| x _ {0} - M (\bar {a} (x _ {0})) \| _ {X} + \| - x _ {0} - M (\bar {a} (- x _ {0})) \| _ {X}, \\ \end{array}
$$
and hence at least one of the two summands on the right has to be larger than or equal to  $\varrho$ .
E.1 Proof of Theorem 4.1
Using Proposition E.1, we can deduce our lower bound in the context of  $C^k$ -spaces. The proof is in fact almost identical to what is done in [19, Theorem 4.2]. However, we decided to include a detailed proof in this paper, since [19] considers Sobolev functions and not  $C^k$ -functions.
Theorem E.2. Let  $s, k \in \mathbb{N}$ . Then there exists a constant  $c = c(s, k) > 0$  with the following property: For any  $m \in \mathbb{N}$  and any map  $\overline{a}: C^k([-1,1]^s; \mathbb{R}) \to \mathbb{R}^m$  that is continuous with respect to some norm on  $C^k([-1,1]^s; \mathbb{R})$  and any (possibly discontinuous) map  $M: \mathbb{R}^m \to C([-1,1]^s; \mathbb{R})$ , we have
$$
\begin{array}{l} \sup  \quad \| f - M (\bar {a} (f)) \| _ {L ^ {\infty} \left(\left[ - 1, 1 \right] ^ {s}; \mathbb {R}\right)} \geq c \cdot m ^ {- k / s}. \\ f \in C ^ {k} ([ - 1, 1 ] ^ {s}; \mathbb {R}) \\ \| f \| _ {C ^ {k} \left([ - 1, 1 ] ^ {s}; \mathbb {R}\right)} \leq 1 \\ \end{array}
$$
Proof. The idea is to apply Proposition E.1 to  $X \coloneqq C([-1,1]^s;\mathbb{R})$ ,  $V \coloneqq C^k ([-1,1]^s;\mathbb{R})$  and the set  $K \coloneqq \{f\in C^{k}([-1,1]^{s};\mathbb{R}): \| f\|_{C^{k}([-1,1]^{s};\mathbb{R})}\leq 1\}$ .
Assume in the beginning that  $m = n^s$  with an integer  $n > 1$ . Pick  $\phi \in C^\infty (\mathbb{R}^s)$  with  $\phi \equiv 1$  on  $[-3/4, 3/4]^s$  and  $\phi \equiv 0$  outside of  $[-1, 1]^s$ . Fix  $c_0 = c_0(s, k) > 0$  with
$$
1 \leq \| \phi \| _ {C ^ {k} ([ - 1, 1 ] ^ {s}; \mathbb {R})} \leq c _ {0}.
$$
Let  $Q_{1}, \ldots, Q_{m}$  be the partition (disjoint up to null-sets) of  $[-1, 1]^{s}$  into closed cubes of sidelength  $2/n$ . For every  $j \in \{1, \ldots, m\}$  we write  $Q_{j} = \times_{\ell=1}^{s}[a_{\ell}^{(j)} - 1/n, a_{\ell}^{(j)} + 1/n]$  with an appropriately chosen vector  $a = (a_{1}^{(j)}, \ldots, a_{s}^{(j)}) \in [-1, 1]^{s}$  and let
$$
\phi_ {j} (x) := \phi (n (x - a ^ {(j)})) \text {f o r} x \in \mathbb {R} ^ {s}.
$$
By choice of  $\phi$ , the maps  $\phi_j$  are supported on a proper subset of  $Q_j$  for every  $j \in \{1, \dots, m\}$  and an inductive argument shows
$$
\partial^ {\mathbf {k}} \phi_ {j} (x) = n ^ {| \mathbf {k} |} \cdot (\partial^ {\mathbf {k}} \phi) (n (x - a ^ {(j)})) \quad \text {f o r e v e r y} \mathbf {k} \in \mathbb {N} _ {0} ^ {s} \text {a n d} x \in \mathbb {R} ^ {s}
$$
and hence in particular
$$
\left\| \phi_ {j} \right\| _ {C ^ {k} ([ - 1, 1 ] ^ {s}; \mathbb {R})} \leq n ^ {| \mathbf {k} |} \cdot c _ {0}. \tag {E.2}
$$
Let  $X_{m} \coloneqq \operatorname{span}\{\phi_{1},\dots,\phi_{m}\}$  and  $S \in U_{1}(X_{m}) = \{f \in X_{m} : \| f \|_{L^{\infty}([-1,1]^{s};\mathbb{R})} \leq 1\}$ . Then we can write  $S$  in the form  $S = \sum_{j=1}^{m} c_{j} \phi_{j}$  with real numbers  $c_{1},\ldots,c_{m} \in \mathbb{R}$ . Suppose there exists  $j^{*} \in \{1,\dots,m\}$  with  $|c_{j^{*}}| > 1$ . Then we have
$$
\left\| S \right\| _ {L ^ {\infty} ([ - 1, 1 ] ^ {s}; \mathbb {R})} \geq | S (a ^ {(j ^ {*})}) | \geq | c _ {j ^ {*}} | > 1,
$$
since the functions  $\phi_j$  have disjoint support and  $\phi_j(a^{(j)}) = 1$ . This is a contradiction to  $S \in U_1(X_m)$  and we can thus infer that  $\max_j |c_j| \leq 1$ . Furthermore, we see again because the functions  $\phi_j$  have disjoint support that
$$
\| \partial^ {\mathbf {k}} S \| _ {L ^ {\infty} ([ - 1, 1 ] ^ {s}; \mathbb {R})} \leq \max _ {j} | c _ {j} | \cdot \| \partial^ {\mathbf {k}} \phi_ {j} \| _ {L ^ {\infty} ([ - 1, 1 ] ^ {s}; \mathbb {R})} \stackrel {{(\mathrm {E . 2})}} {\leq} n ^ {| \mathbf {k} |} \cdot c _ {0} \leq c _ {0} \cdot n ^ {k} = c _ {0} \cdot m ^ {k / s}
$$
for every  $\mathbf{k} \in \mathbb{N}_0^s$  with  $|\mathbf{k}| \leq k$  and hence
$$
\left\| S \right\| _ {C ^ {k} ([ - 1, 1 ] ^ {s}; \mathbb {R})} \leq c _ {0} \cdot m ^ {k / s}.
$$
Thus, letting  $\varrho := c_0^{-1} \cdot m^{-k / s}$  yields  $U_{\varrho}(X_m) \subseteq K$ , so we see by Proposition E.1 that
$$
\sup  _ {f \in K} \| f - M _ {m - 1} (\overline {{a}} (f)) \| _ {L ^ {\infty} ([ - 1, 1 ] ^ {s}; \mathbb {R})} \geq \varrho = c _ {1} \cdot m ^ {- k / s}
$$
with  $c_{1} = c_{0}^{-1}$  for every map  $\overline{a}: X \to \mathbb{R}^{m-1}$  which is continuous with respect to some norm on  $V$  and any map  $M_{m-1}: \mathbb{R}^{m-1} \to X$ . Using the inequality  $m \leq 2(m-1)$  (note  $m > 1$ ) we get
$$
\sup _ {f \in K} \| f - M _ {m - 1} (\overline {{a}} (f)) \| _ {L ^ {\infty} ([ - 1, 1 ] ^ {s}; \mathbb {R})} \geq c _ {1} \cdot m ^ {- k / s} \geq c _ {1} \cdot (2 (m - 1)) ^ {- k / s} \geq c _ {2} \cdot (m - 1) ^ {- k / s}
$$
with  $c_2 = c_1 \cdot 2^{-k / s}$ . Hence, the claim has been shown for all numbers  $m$  of the form  $n^s - 1$  with an integer  $n > 1$ .
In the end, let  $m \in \mathbb{N}$  be arbitrary and pick  $n \in \mathbb{N}$  with  $n^s \leq m < (n + 1)^s$ . For given maps  $\overline{a}: V \to \mathbb{R}^m$  and  $M: \mathbb{R}^m \to X$  with  $\overline{a}$  continuous with respect to some norm on  $V$ , let
$$
\tilde {a}: \quad V \to \mathbb {R} ^ {(n + 1) ^ {s} - 1}, \quad f \mapsto (\overline {{a}} (f), 0) \quad \text {a n d}
$$
$$
M _ {(n + 1) ^ {s} - 1}: \mathbb {R} ^ {(n + 1) ^ {s} - 1} \to X, (x, y) \mapsto M (x),
$$
where  $x\in \mathbb{R}^m$ $y\in \mathbb{R}^{(n + 1)^s -1 - m}$  . Then we get
$$
\begin{array}{l} \sup  _ {f \in K} \| f - M (\bar {a} (f)) \| _ {L ^ {\infty} ([ - 1, 1 ] ^ {s}; \mathbb {R})} = \sup  _ {f \in K} \| f - M _ {(n + 1) ^ {s} - 1} (\tilde {a} (f)) \| _ {L ^ {\infty} ([ - 1, 1 ] ^ {s}; \mathbb {R})} \\ \geq c _ {2} \cdot ((n + 1) ^ {s} - 1) ^ {- k / s} \geq c _ {2} \cdot (2 ^ {s} n ^ {s}) ^ {- k / s} \geq c _ {3} \cdot m ^ {- k / s} \\ \end{array}
$$
with  $c_{3} = c_{2} \cdot 2^{-k}$ . Here we used the bound  $(n + 1)^{s} - 1 \leq (2n)^{s}$ . This proves the full claim.
Using this theorem, we can now prove Theorem 4.1.
Proof of Theorem 4.1. Let  $\overline{a}: C^k(\Omega_n; \mathbb{C}) \to \mathbb{C}^m$  be any map that is continuous with respect to some norm  $\| \cdot \|_V$  on  $C^k(\Omega_n; \mathbb{C})$ , and let  $M: \mathbb{C}^m \to C(\Omega_n; \mathbb{C})$  be arbitrary. With  $\varphi_n, \varphi_m$  defined as in Equation (A.1), let
$$
\tilde {a}: \quad C ^ {k} ([ - 1, 1 ] ^ {2 n}; \mathbb {R}) \to \mathbb {R} ^ {2 m}, \quad \tilde {f} \mapsto \varphi_ {m} ^ {- 1} \left(\bar {a} \left(\tilde {f} \circ \varphi_ {n} ^ {- 1} \big | _ {\Omega_ {n}}\right)\right).
$$
Clearly,  $\tilde{a}$  is continuous on  $C^k ([-1,1]^{2n};\mathbb{R})$  with respect to the norm  $\| \cdot \|_{\tilde{V}}$  on  $C^k ([-1,1]^{2n};\mathbb{R})$  defined as
$$
\| \tilde {f} \| _ {\tilde {V}} := \left\| \tilde {f} \circ \varphi_ {n} ^ {- 1} \big | _ {\Omega_ {n}} \right\| _ {V} \quad \text {f o r} \tilde {f} \in C ^ {k} ([ - 1, 1 ] ^ {2 n}; \mathbb {R}).
$$
$$
\widetilde {M}: \mathbb {R} ^ {2 m} \to C ([ - 1, 1 ] ^ {2 n}; \mathbb {R}), \quad \widetilde {M} (x) := \operatorname {R e} (M (\varphi_ {m} (x))) \circ \varphi_ {n} \big | _ {[ - 1, 1 ] ^ {2 n}}.
$$
$$
\begin{array}{l} \sup  _ {f \in C ^ {k} (\Omega_ {n}; \mathbb {C})} \| f - M (\overline {{a}} (f)) \| _ {L ^ {\infty} (\Omega_ {n}; \mathbb {C})} \\ \| f \| _ {C ^ {k} (\Omega_ {n}; \mathbb {C})} \leq 1 \\ \geq \sup  _ {f \in C ^ {k} \left(\Omega_ {n}; \mathbb {R}\right)} \| f - \operatorname {R e} (M (\bar {a} (f))) \| _ {L ^ {\infty} \left(\Omega_ {n}; \mathbb {R}\right)} \\ \| f \| _ {C ^ {k} (\Omega_ {n}; \mathbb {R})} \leq 1 \\ = \sup  _ {\tilde {f} \in C ^ {k} ([ - 1, 1 ] ^ {2 n}; \mathbb {R})} \left\| \tilde {f} \circ \varphi_ {n} ^ {- 1} - \operatorname {R e} \left(M \left(\bar {a} \left(\tilde {f} \circ \varphi_ {n} ^ {- 1} \mid_ {\Omega_ {n}}\right)\right)\right) \right\| _ {L ^ {\infty} (\Omega_ {n}; \mathbb {R})} \\ \| \tilde {f} \| _ {C ^ {k} ([ - 1, 1 ] ^ {2 n}; \mathbb {R})} \leq 1 \\ = \sup  _ {\tilde {f} \in C ^ {k} ([ - 1, 1 ] ^ {2 n}; \mathbb {R})} \left\| \tilde {f} - \operatorname {R e} \left(M \left(\varphi_ {m} \left(\varphi_ {m} ^ {- 1} \left(\bar {a} \left(\tilde {f} \circ \varphi_ {n} ^ {- 1} \mid_ {\Omega_ {n}}\right)\right)\right)\right)\right) \circ \varphi_ {n} \right\| _ {L ^ {\infty} ([ - 1, 1 ] ^ {2 n}; \mathbb {R})} \\ \| \tilde {f} \| _ {C ^ {k} ([ - 1, 1 ] ^ {2 n}; \mathbb {R})} \leq 1 \\ = \sup  _ {\tilde {f} \in C ^ {k} ([ - 1, 1 ] ^ {2 n}; \mathbb {R})} \left\| \tilde {f} - \widetilde {M} (\tilde {a} (\tilde {f})) \right\| _ {L ^ {\infty} ([ - 1, 1 ] ^ {2 n}; \mathbb {R})} \geq \tilde {c} \cdot (2 m) ^ {- k / (2 n)}, \\ \| \tilde {f} \| _ {C ^ {k} ([ - 1, 1 ] ^ {2 n}; \mathbb {R})} \leq 1 \\ \end{array}
$$
with a constant  $\tilde{c} = \tilde{c}(n,k)$  provided by Theorem E.2. Hence, the claim follows by choosing  $c = c(n,k) \coloneqq 2^{-k/(2n)} \cdot \tilde{c}$ .
As a corollary, we formulate a special case of Theorem 4.1 for the case of shallow complex-valued neural networks.
Corollary E.3. Let  $n, k \in \mathbb{N}$ . Then there exists a constant  $c = c(n, k) > 0$  with the following property: For any  $m \in \mathbb{N}$ ,  $\phi \in C(\mathbb{C}; \mathbb{C})$  and any map
$$
\eta : \quad C ^ {k} (\Omega_ {n}; \mathbb {C}) \to (\mathbb {C} ^ {n}) ^ {m} \times \mathbb {C} ^ {m} \times \mathbb {C} ^ {m}, \quad g \mapsto (\eta_ {1} (g), \eta_ {2} (g), \eta_ {3} (g))
$$
which is continuous with respect to some norm on  $C^k(\Omega_n; \mathbb{C})$ , there exists  $f \in C^k(\Omega_n; \mathbb{C})$  satisfying  $\| f \|_{C^k(\Omega_n; \mathbb{C})} \leq 1$  and
$$
\left\| f - \Psi (f) \right\| _ {L ^ {\infty} (\Omega_ {n}; \mathbb {C})} \geq c \cdot m ^ {- k / (2 n)},
$$
where  $\Psi (f)\in C(\Omega_n;\mathbb{C})$  is given by
$$
\Psi (f) (z) := \sum_ {j = 1} ^ {m} (\eta_ {3} (f)) _ {j} \phi \left(\left[ \eta_ {1} (f) \right] _ {j} ^ {T} z + (\eta_ {2} (f)) _ {j}\right).
$$
Proof. Using Theorem 4.1, we deduce that there exists  $f \in C^k(\Omega_n; \mathbb{C})$  satisfying  $\| f \|_{C^k(\Omega_n; \mathbb{C})} \leq 1$  and
$$
\| f - \Psi (f) \| _ {L ^ {\infty} \left(\Omega_ {n}; \mathbb {C}\right)} \geq c ^ {\prime} \cdot (m (n + 2)) ^ {- k / (2 n)}
$$
for a constant  $c' = c'(n, k) > 0$ . Hence, the claim follows by letting  $c \coloneqq c' \cdot (n + 2)^{-k / (2n)}$ .
We can use Proposition E.1 not only to show that the rate of convergence established in this paper is optimal (which is done in Appendix E.1) but also to show that the problem of approximating  $C^k$ -functions using a set of functions that can be parametrized with finitely many parameters is intractable in the sense that it suffers from the curse of dimensionality, provided that the map which assigns to each  $C^k$ -function the parameters of the approximating function is continuous. This is the subject of this section.
In [35] a certain space of polynomials was used to show the intractability in the case of linear approximation methods. We are also going to use this class of polynomials, but combine it with Proposition E.1 to infer intractability in the case of continuous approximation methods. We start with a lemma discussing an important property of this space of polynomials. This property is stated as part of a proof in [35], but no complete proof is provided.
Lemma E.4. Let  $s \in \mathbb{N}$  and consider a function  $f \in C^{\infty}([-1,1]^{s};\mathbb{R})$  which is given via
$$
f (x) = \sum_ {k \in \{0, 1 \} ^ {s}} a _ {k} x ^ {k} \tag {E.3}
$$
with coefficients  $a_{\pmb{k}} \in \mathbb{R}$  for every  $\pmb{k} \in \{0,1\}^s$ . Then it holds
$$
\left\| f \right\| _ {C ^ {k} ([ - 1, 1 ] ^ {s}; \mathbb {R})} = \left\| f \right\| _ {L ^ {\infty} ([ - 1, 1 ] ^ {s}; \mathbb {R})}
$$
for every  $k\in \mathbb{N}$
Proof. The proof is by induction over  $s$ . We start with the case  $s = 1$  and note that we can write  $f(x) = ax + b$  with  $a, b \in \mathbb{R}$  in that case. Switching to  $-f$  if necessary, we can assume  $a \geq 0$ . Clearly,  $\| f \|_{L^{\infty}([-1,1];\mathbb{R})} \leq |a| + |b|$ . Conversely, if  $b \geq 0$  then  $|f(1)| = |a + b| = |a| + |b|$ . If otherwise  $b < 0$  then  $|f(-1)| = |b - a| = |a - b| = |a| + |b|$ . Thus,  $\| f \|_{L^{\infty}([-1,1];\mathbb{R})} = |a| + |b|$ . For the derivatives, we have  $\| f' \|_{L^{\infty}([-1,1];\mathbb{R})} = |a|$  and  $\| f^{(k)} \|_{L^{\infty}([-1,1];\mathbb{R})} = 0$  for  $k \geq 2$ . This proves the claim in the case  $s = 1$ .
We now assume that the claim holds for some arbitrary but fixed  $s \in \mathbb{N}$ . We further let  $\alpha \in \mathbb{N}_0^{s+1}$  and fix a point  $(x_1, \ldots, x_{s+1}) \in [-1, 1]^{s+1}$ . We decompose  $\alpha = (\alpha', \alpha_{s+1})$  with  $\alpha' \in \mathbb{N}_0^s$ . Let
$$
\widetilde {f}: \quad [ - 1, 1 ] \rightarrow \mathbb {R}, \quad y _ {s + 1} \mapsto \partial^ {(\alpha^ {\prime}, 0)} f (x _ {1},..., x _ {s}, y _ {s + 1})
$$
$$
\partial^ {\alpha} f (x _ {1}, \dots , x _ {s + 1}) = \widetilde {f} ^ {(\alpha_ {s + 1})} (x _ {s + 1}).
$$
Note that  $f$  is affine-linear with respect to each variable (with all other variables hold fixed). Hence,  $\widetilde{f}$  is an affine function and we can thus apply the case  $s = 1$  to  $\widetilde{f}$  and get
$$
\left\| \widetilde {f} ^ {\left(\alpha_ {s + 1}\right)} \right\| _ {L ^ {\infty} ([ - 1, 1 ]; \mathbb {R})} \leq \left\| \widetilde {f} \right\| _ {L ^ {\infty} ([ - 1, 1 ]; \mathbb {R})}.
$$
Putting this together, we infer
$$
\left| \partial^ {\alpha} f \left(x _ {1}, \dots , x _ {s + 1}\right) \right| \leq \| \widetilde {f} \| _ {L ^ {\infty} ([ - 1, 1 ]; \mathbb {R})} = \sup  _ {y _ {s + 1} \in [ - 1, 1 ]} \left| \partial^ {\left(\alpha^ {\prime}, 0\right)} f \left(x _ {1}, \dots , x _ {s}, y _ {s + 1}\right) \right|. \tag {E.4}
$$
We now fix an arbitrary point  $y_{s + 1}\in [-1,1]$  and consider
$$
\widehat {f}: \quad [ - 1, 1 ] ^ {s} \to \mathbb {R}, \quad (y _ {1}, \dots , y _ {s}) \mapsto f (y _ {1}, \dots , y _ {s}, y _ {s + 1}).
$$
$$
\partial^ {(\alpha^ {\prime}, 0)} f (x _ {1}, \dots , x _ {s}, y _ {s + 1}) = \partial^ {\alpha^ {\prime}} \widehat {f} (x _ {1}, \dots , x _ {s}).
$$
Applying the induction hypothesis to  $\widehat{f}$  (which is easily seen to be of the form (E.3)) we get
$$
\left| \partial^ {\alpha^ {\prime}} \widehat {f} \left(x _ {1}, \dots , x _ {s}\right) \right| \leq \| \widehat {f} \| _ {L ^ {\infty} \left([ - 1, 1 ] ^ {s}; \mathbb {R}\right)} \leq \| f \| _ {L ^ {\infty} \left([ - 1, 1 ] ^ {s + 1}; \mathbb {R}\right)}. \tag {E.5}
$$
Combining (E.4) and (E.5) yields
$$
\left| \partial^ {\alpha} f \left(x _ {1},..., x _ {s + 1}\right) \right| \leq \| f \| _ {L ^ {\infty} ([ - 1, 1 ] ^ {s + 1}; \mathbb {R})}.
$$
Since  $\alpha \in \mathbb{N}_0^{s + 1}$  was arbitrary, we get the claim by noting that
$$
\| f \| _ {C ^ {k} ([ - 1, 1 ] ^ {s}; \mathbb {R})} \geq \| f \| _ {L ^ {\infty} ([ - 1, 1 ] ^ {s}; \mathbb {R})}
$$
holds trivially for every  $k\in \mathbb{N}$
Using the above lemma, we can now deduce that the approximation of smooth functions using continuous approximation methods is intractable in terms of the input dimension.
Proof of Theorem 5.1. We apply Proposition E.1 to  $X \coloneqq C([-1,1]^s;\mathbb{R})$ ,  $V \coloneqq C^{\infty ,*,s}$  and to the set  $K \coloneqq \{f\in C^{\infty ,*,s}: \| f\|_{C^{\infty}([-1,1]^{s};\mathbb{R})}\leq 1\}$  and  $m \coloneqq 2^s -1$ . The space
$$
X _ {m + 1} := \left\{\left[ - 1, 1 \right] ^ {s} \ni x \mapsto \sum_ {\mathbf {k} \in \{0, 1 \} ^ {s}} a _ {\mathbf {k}} x ^ {\mathbf {k}}: a _ {\mathbf {k}} \in \mathbb {R} \right\}
$$
consisting of all functions considered in the previous Lemma E.4 is an  $(m + 1)$ -dimensional subspace of  $C([-1,1]^s;\mathbb{R})$ . For every  $f\in X_{m + 1}$  with  $\| f\|_{L^{\infty}([-1,1]^{s};\mathbb{R})}\leq 1$ , Lemma E.4 tells us  $\| f\|_{C^{\infty}([-1,1]^{s};\mathbb{R})}\leq 1$ . Hence,  $U_{1}(X_{m + 1})\subseteq K$  and Proposition E.1 then yields the claim.
Remark E.5. The statement of Theorem 5.1 also holds if the functions satisfy  $\overline{a}: C^{\infty, *, s} \to \mathbb{R}^m$  and  $M: \mathbb{R}^m \to C([-1, 1]^s; \mathbb{R})$  with  $m \leq 2^s - 1$ . This can be seen by defining
$$
\tilde {a}: \quad C ^ {\infty , *, s} \to \mathbb {R} ^ {2 ^ {s} - 1}, \quad f \mapsto (\bar {a} (f), 0, \dots , 0)
$$
$$
\widetilde {M}: \mathbb {R} ^ {2 ^ {s} - 1} \to C ([ - 1, 1 ] ^ {s}; \mathbb {R}), (a, b) \mapsto M (a)
$$
with  $a\in \mathbb{R}^m$  and  $b\in \mathbb{R}^{2^s -1 - m}$
The following Corollary E.6 transfers Theorem 5.1 to the complex-valued setting.
Corollary E.6. Let  $n \in \mathbb{N}$ . For any function  $f \in C^{\infty}(\Omega_n; \mathbb{C})$  we write
$$
\| f \| _ {C ^ {\infty} \left(\Omega_ {n}; \mathbb {C}\right)} := \sup  _ {k \in \mathbb {N}} \| f \| _ {C ^ {k} \left(\Omega_ {n}; \mathbb {C}\right)}
$$
and let  $C_{\mathbb{C}}^{\infty,*,n}$  denote the space consisting of all functions for which this expression is finite. Let  $\overline{a}: C_{\mathbb{C}}^{\infty,*,n} \to \mathbb{C}^{2^{2n-1}-1}$  be continuous with respect to some norm on  $C_{\mathbb{C}}^{\infty,*,n}$  and moreover, let  $M: \mathbb{C}^{2^{2n-1}-1} \to C(\Omega_n; \mathbb{C})$  be an arbitrary map. Then it holds
$$
\sup_{\substack{f\in C^{\infty ,*,n}_{\mathbb{C}}\\ \| f\|_{C^{\infty}(\Omega_{n};\mathbb{C})}\leq 1}}\| f - M(\overline{a} (f))\|_{L^{\infty}(\Omega_{n};\mathbb{C})}\geq 1.
$$