SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Paul Geuchen
MIDS,
KU Eichstatt-Ingolstadt,
Auf der Schanz 49,
85049 Ingolstadt, Germany
paul.geuchen@ku.de
Felix Voigtlaender
felix.voigtlaender@ku.de
Complex-valued neural networks (CVNNs) have recently shown promising empirical success, for instance for increasing the stability of recurrent neural networks and for improving the performance in tasks with complex-valued inputs, such as in MRI fingerprinting. While the overwhelming success of Deep Learning in the real-valued case is supported by a growing mathematical foundation, such a foundation is still largely lacking in the complex-valued case. We thus analyze the expressivity of CVNNs by studying their approximation properties. Our results yield the first quantitative approximation bounds for CVNNs that apply to a wide class of activation functions including the popular modReLU and complex cardioid activation functions. Precisely, our results apply to any activation function that is smooth but not polyharmonic on some non-empty open set; this is the natural generalization of the class of smooth and non-polynomial activation functions to the complex setting. Our main result shows that the error for the approximation of  $C^k$ -functions scales as  $m^{-k / (2n)}$  for  $m\to \infty$  where  $m$  is the number of neurons,  $k$  the smoothness of the target function and  $n$  is the (complex) input dimension. Under a natural continuity assumption, we show that this rate is optimal; we further discuss the optimality when dropping this assumption. Moreover, we prove that the problem of approximating  $C^k$ -functions using continuous approximation methods unavoidably suffers from the curse of dimensionality.
Deep Learning currently predominantly relies on real-valued neural networks, which have led to breakthroughs in fields like image classification or speech recognition [23, 27, 43]. However, recent work has uncovered several application areas in which the use of complex-valued neural networks (CVNNs) leads to better results than the use of their real-valued counterparts. These application areas mainly include tasks where complex numbers inherently occur as inputs of a machine learning model such as Magnetic Resonance Imaging (MRI) [16, 38, 44] and Polarimetric Synthetic Aperture Radar (PolSAR) Imaging [8, 9, 48]. Moreover, CVNNs have been used to improve the stability of recurrent neural networks [5] and have been successfully applied in various other fields [32, 37]. The mathematical theory of these complex-valued neural networks, however, is still in its infancy. There is therefore a great interest in studying CVNNs and in particular in uncovering the differences and commonalities between CVNNs and their real-valued counterparts. A prominent example highlighting the unexpected differences between both network classes is the universal approximation theorem for neural networks, whose most general real-valued version was proven in 1993 [28] (with a more restricted version appearing earlier [17]) and which was recently generalized to the case of CVNNs [45]. The two theorems describe necessary and sufficient conditions on
Condition on activation function	Continuity of weight selection	Approximation Error
Theorem 3.2	smooth & non-polyharmonic	possible	\(\mathcal{O}(m^{-k/(2n)})\)
Consequence of Theorem 4.1	continuous	assumed	\(\Omega(m^{-k/(2n)})\)
Theorem 4.2	very special activation function	not assumed	\(\mathcal{O}(m^{-k/(2n-1)})\)
Theorem 4.3	\(\frac{1}{1 + e^{-\mathrm{Re}(z)}}\)	not assumed	\(\widetilde{\Omega}(m^{-k/(2n)})\)
Table 1: Overview of the proven approximation bounds.  $k$  is the regularity of the approximated functions (which are assumed to be  $C^k$ ),  $n$  the (complex) input dimension and  $m$  the number of neurons in the hidden layer of the network. The notation  $\widetilde{\Omega}$  is similar to  $\Omega$ , but ignoring log factors.
an activation function which guarantee that arbitrarily wide neural networks of a fixed depth can approximate any continuous function on any compact set arbitrarily well (with respect to the uniform norm). Already here it was shown that complex-valued networks behave significantly different from real-valued networks: While real-valued networks are universal if and only if the activation function is non-polynomial, complex-valued networks with a single hidden layer are universal if and only if the activation function is non-polyharmonic (see below for a definition). Furthermore, there exist continuous activation functions for which deep CVNNs are universal but shallow CVNNs are not, whereas the same cannot happen for real-valued neural networks. This example shows that it is highly relevant to study the properties of CVNNs and to examine which of the fundamental properties of real-valued networks extend to complex-valued networks.
Essentially the only existing quantitative result regarding the approximation-theoretical properties of CVNNs is [14], which provides results for approximating  $C^k$ -functions by deep CVNNs using the modReLU activation function. However, for real-valued NNs it is known that already shallow NNs can approximate  $C^k$ -functions at the optimal rate. Precisely, Mhaskar showed in [33] that one can approximate  $C^k$ -functions on  $[-1, 1]^n$  with an error of order  $m^{-k/n}$  as  $m \to \infty$ , where  $m$  is the number of neurons in the hidden layer. Here he assumed that the activation function is smooth on an open interval and that at some point of this interval no derivative vanishes. This is equivalent to the activation function being smooth and non-polynomial on that interval, cf. [20, p. 53].
The present paper shows that a comparable result holds in the setting of complex-valued networks, by proving that one can approximate every function in  $C^k(\Omega_n; \mathbb{C})$  (where differentiability is understood in the sense of real variables) with an error of the order  $m^{-k/(2n)}$  (as  $m \to \infty$ ) using shallow complex-valued neural networks with  $m$  neurons in the hidden layer. Here we define the cube  $\Omega_n := [-1, 1]^n + i[-1, 1]^n$ . The result holds whenever the activation function  $\phi: \mathbb{C} \to \mathbb{C}$  is smooth and non-polyharmonic on some non-empty open set. This is a very natural condition, since for polyharmonic activation functions there exist  $C^k$ -functions that cannot be approximated at all below some error threshold using shallow neural networks with this activation function [45].
Furthermore, the present paper studies in how far the approximation order of  $m^{-(k / (2n))}$  is optimal, meaning that an order of  $m^{-(k / 2n) - \alpha}$  (where  $\alpha > 0$ ) cannot be achieved. Here it turns out that the derived order of approximation is indeed optimal (even in the class of CVNNs with possibly more than one hidden layer) in the setting that the weight selection is continuous, meaning that the map that assigns to a function  $f \in C^k(\Omega_n; \mathbb{C})$  the weights of the approximating network is continuous with respect to some norm on  $C^k(\Omega_n; \mathbb{C})$ . This continuity assumption is natural since typical learning algorithms such as (stochastic) gradient descent use samples  $f(x_j)$  of the target function and then apply continuous operations to them to update the network weights.
We investigate this optimality result further by dropping the continuity assumption and constructing two special smooth and non-polyharmonic activation functions with the first one having the property that the order of approximation can indeed be strictly improved via a discontinuous selection of
the related weights. For the second activation function we show that the order of  $m^{-k / (2n)}$  cannot be improved, even if one allows a discontinuous weight selection. This in particular shows that in the given generality of arbitrary smooth, non-polyharmonic activation functions, the upper bound  $\mathcal{O}\left(m^{-k / (2n)}\right)$  cannot be improved, even for a possibly discontinuous choice of the weights. An overview of the approximation bounds proven in this paper can be found in Table 1.
Moreover, we analyze the tractability (in terms of the input dimension  $n$ ) of the considered problem of approximating  $C^k$ -functions using neural networks. Theorem 5.1 shows that one necessarily needs a number of parameters exponential in  $n$  to obtain a non-trivial approximation error. To the best of our knowledge, Theorem 5.1 is the first result showing that the problem of approximating  $C^k$ -functions using continuous approximation methods is intractable (in terms of the input dimension  $n$ ).
Real-valued neural networks. By now, the approximation properties of real-valued neural networks are quite well-studied (cf. [10, 28, 33, 40, 41, 46, 47] and the references therein). We here only discuss a few papers that are most closely related to the present work.
In [33], Mhaskar analyzes the rate of approximation of shallow real-valued neural networks for target functions of regularity  $C^k$ . Our results can be seen as the generalization of [33] to the complex setting. Our proofs rely on several techniques from [33]; however, significant modifications are required to make the proofs work for general smooth non-polyharmonic functions.
One of the first papers to observe that neural networks with general (smooth) activation function can be surprisingly expressive is [31] where it was shown that a neural network of constant size can be universal. One of the activation functions in Section 4 is based on a similar idea.
The importance of distinguishing between continuous and discontinuous weight selection (which in our setting is discussed in Section 4) was observed for ReLU-networks in [47].
The paper [25] shows that neural network approximation is not continuous in the following sense: The best approximating neural network  $\Phi (f)$  of a given size does not depend continuously on  $f\in C^k$  This result, however, is not in conflict with our results: We want to assign to any  $C^k$  -function  $f$  a network  $\widetilde{\Phi} (f)$  that approximates  $f$  below the error threshold  $m^{-k / (2n)}$  . The network  $\widetilde{\Phi} (f)$  , however, does not have to coincide with the best approximating network  $\Phi (f)$
Complex-valued neural networks. When it comes to general literature about mathematical properties of complex-valued neural networks, surprisingly little work can be found. The Universal Approximation Theorem for Complex-Valued Neural Networks [45] has already been mentioned above. In particular, it has been shown that shallow CVNNs are universal if and only if the activation function  $\phi$  is not polyharmonic. Thus, the condition assumed in the present paper (that  $\phi$  should be smooth, but not polyharmonic) is quite natural.
Regarding quantitative approximation results for CVNNs, the only existing work of which we are aware is [14], analyzing the approximation capabilities of deep CVNNs where the modReLU is used as activation function. Since the modReLU satisfies our condition regarding the activation function, the present work can be seen as an improvement to [14]. Precisely, (i) we consider general activation functions, including but not limited to the modReLU, (ii) we improve the order of approximation by a log factor, and (iii) we show that this order of approximation can be achieved using shallow networks instead of the deep networks used in [14]. We stress that our proof techniques differ significantly from the ones applied in [14]: The arguments in [14] take their main ideas from [46] making heavy use of the specific definition of the modReLU. In contrast, since we consider quite general activation functions, we necessarily follow a much more general approach following the ideas from [33].
Shallow complex-valued neural networks. In this paper we mainly consider so-called shallow complex-valued neural networks, meaning complex-valued neural networks with a single hidden layer. Precisely, we consider functions of the form
$$
\mathbb {C} ^ {n} \ni z \mapsto \sum_ {j = 1} ^ {m} \sigma_ {j} \phi (\rho_ {j} ^ {T} \cdot z + \eta_ {j}) \in \mathbb {C},
$$
Figure 1: Graphical representation of a shallow neural network. Input and output neurons are depicted as dots, hidden neurons are depicted as circles. The term first layer describes the transformation from the input to the hidden neurons, including the application of the activation function.
with  $\rho_1, \ldots, \rho_m \in \mathbb{C}^n$ ,  $\sigma_1, \ldots, \sigma_m, \eta_1, \ldots, \eta_m \in \mathbb{C}$  and an activation function  $\phi: \mathbb{C} \to \mathbb{C}$ . Here,  $m \in \mathbb{N}$  denotes the number of neurons of the network and we write  $v^T$  for the transpose of a vector  $v$ .
To simplify the formulation of the results, we introduce the following notation: We write  $\mathcal{F}_{n,m}^{\phi}$  for the set of first layers of shallow complex-valued neural networks with activation function  $\phi$ , with  $n$  input neurons and  $m$  hidden neurons, meaning
$$
\mathcal {F} _ {n, m} ^ {\phi} := \left\{z \mapsto \left(\phi \left(\rho_ {j} ^ {T} \cdot z + \eta_ {j}\right)\right) _ {j = 1} ^ {m}: \rho_ {j} \in \mathbb {C} ^ {n}, \eta_ {j} \in \mathbb {C} \right\} \subseteq \left\{F: \mathbb {C} ^ {n} \to \mathbb {C} ^ {m} \right\}.
$$
Hence, each shallow CVNN can be written as  $\sigma^T\Phi$  with  $\sigma \in \mathbb{C}^m$  and  $\Phi \in \mathcal{F}_{n,m}^{\phi}$ ; see Figure 1 for a graphical representation of a shallow CVNN.
Approximation. The paper aims to analyze the approximation of  $C^k$ -functions on the complex cube
$$
\Omega_ {n} := [ - 1, 1 ] ^ {n} + i [ - 1, 1 ] ^ {n}
$$
using shallow CVNNs. Here, we say that a function  $f: \Omega_n \to \mathbb{C}$  is in  $C^k(\Omega_n; \mathbb{C})$  if and only if  $f$  is  $k$  times continuously differentiable on  $\Omega_n$ , where the derivative is to be understood in the sense of real variables, i.e., in the sense of interpreting  $f$  as a function  $[-1, 1]^{2n} \to \mathbb{R}^2$  and taking usual real derivatives. We further define the  $C^k$ -norm of a function  $f \in C^k(\Omega_n; \mathbb{C})$  as
$$
\| f \| _ {C ^ {k} \left(\Omega_ {n}; \mathbb {C}\right)} := \sup  _ {\substack {\mathbf {k} \in \mathbb {N} _ {0} ^ {2 n} \\ | \mathbf {k} | \leq k}} \| \partial^ {\mathbf {k}} f \| _ {L ^ {\infty} \left(\Omega_ {n}; \mathbb {C}\right)}, \quad \text {w h e r e} \quad \| g \| _ {L ^ {\infty} \left(\Omega_ {n}; \mathbb {C}\right)} := \sup  _ {z \in \Omega_ {n}} | g (z) | \tag{2.1}
$$
for any function  $g: \Omega_n \to \mathbb{C}$ . Note that we write  $\mathbb{N} = \{1, 2, 3, \ldots\}$  and  $\mathbb{N}_0 = \{0\} \cup \mathbb{N}$ . Using the previously introduced notation, we thus seek to bound the worst-case approximation error, i.e.,
$$
\sup_{\substack{f\in C^{k}(\Omega_{n};\mathbb{C})\\ \| f\|_{C^{k}(\Omega_{n};\mathbb{C})}\leq 1}}\inf_{\substack{\Phi \in \mathcal{F}_{n,m}^{\phi}\\ \sigma \in \mathbb{C}^{m}}}\| f - \sigma^{T}\Phi \|_{L^{\infty}(\Omega_{n};\mathbb{C})}.
$$
Wirtinger calculus and polyharmonic functions. For a function  $f: \mathbb{C} \to \mathbb{C}$  which is differentiable in the sense of real variables at a point  $z_0 \in \mathbb{C}$  we define its Wirtinger derivatives at  $z_0$  as
$$
\partial_ {\mathrm {w i r t}} f (z _ {0}) := \frac {1}{2} \left(\frac {\partial f}{\partial x} (z _ {0}) - i \cdot \frac {\partial f}{\partial y} (z _ {0})\right) \quad \text {a n d} \quad \overline {{\partial}} _ {\mathrm {w i r t}} f (z _ {0}) := \frac {1}{2} \left(\frac {\partial f}{\partial x} (z _ {0}) + i \cdot \frac {\partial f}{\partial y} (z _ {0})\right).
$$
Here,  $\frac{\partial}{\partial x}$  and  $\frac{\partial}{\partial y}$  denote the usual partial derivatives in the sense of real variables. We extend this definition to multivariate functions defined on open subsets of  $\mathbb{C}^n$  by considering coordinatewise Wirtinger derivatives.
A function  $f: U \to \mathbb{C}$ , where  $U \subseteq \mathbb{C}$  is an open set, is called smooth if it is differentiable arbitrarily many times (in the sense of real variables). We write  $f \in C^{\infty}(U; \mathbb{C})$  in that case. Moreover,  $f$  is called polyharmonic (on  $U$ ) if it is smooth and if there exists  $m \in \mathbb{N}_0$  satisfying
$$
\Delta^ {m} f \equiv 0 \quad \text {o n} U.
$$
Figure 2: Absolute value of the activation functions  $\sigma_{\mathrm{modReLU}, -1}$  (left) and card (right).
Here,  $\Delta := \frac{\partial^2}{\partial x^2} + \frac{\partial^2}{\partial y^2} = 4\partial_{\mathrm{wirt}}\overline{\partial}_{\mathrm{wirt}}$  denotes the usual Laplace-Operator.
The following Proposition 2.1 describes a property of non-polyharmonic functions which is crucial for proving the approximation results of this paper.
Proposition 2.1. Let  $\varnothing \neq U\subseteq \mathbb{C}$  be an open set and let  $\phi \in C^{\infty}(U;\mathbb{C})$  be non-polyharmonic. Then for every  $M\in \mathbb{N}_0$  there exists a point  $z_{M}\in U$  satisfying
$$
\partial_ {\mathrm {w i r t}} ^ {m} \overline {{\partial}} _ {\mathrm {w i r t}} ^ {\ell} \phi (z _ {M}) \neq 0 \quad f o r a l l m, \ell \in \mathbb {N} _ {0} w i t h m, \ell \leq M.
$$
The proof of Proposition 2.1 is an application of the Baire category theorem; see Appendix B.2.
Important complex activation functions. We briefly discuss in how far two commonly used complex activation functions satisfy our assumptions: The modReLU proposed in [5] and the complex cardioid used in [44] for MRI fingerprinting where the performance could be significantly improved using complex-valued neural networks. The modReLU is defined as
$$
\sigma_ {\mathrm {m o d R e L U}, b}: \quad \mathbb {C} \to \mathbb {C}, \quad \sigma_ {\mathrm {m o d R e L U}, b} (z) := \left\{ \begin{array}{l l} (| z | + b) \frac {z}{| z |}, & \text {i f} | z | + b \geq 0, \\ 0, & \text {o t h e r w i s e}, \end{array} \right.
$$
where  $b < 0$  is a fixed parameter. The complex cardioid is defined as
$$
\operatorname {c a r d}: \quad \mathbb {C} \to \mathbb {C}, \quad \operatorname {c a r d} (z) := \frac {1}{2} (1 + \cos (\triangleleft z)) z.
$$
Here,  $\triangleleft z = \theta \in \mathbb{R}$  denotes the polar angle of a complex number  $z = r e^{i\theta}$ , where we define  $\triangleleft 0 := 0$ ; see Figure 2 for plots of the absolute value of the two functions.
Both functions are smooth and non-polyharmonic on a non-empty open subset of  $\mathbb{C}$ , which is proven in Appendix A.2. Furthermore, they are both continuous on  $\mathbb{C}$ . Therefore, our approximation bounds established in Theorems 3.1 and 3.2 in particular apply to those two functions.
In this section we state the main results of this paper and provide proof sketches for them. Detailed proofs of the two statements can be found in Appendices B.3 and C.3.
First we show in Theorem 3.1 that it is possible to approximate any complex polynomial in  $z$  and  $\overline{z}$  arbitrarily well using shallow CVNNs with the size of the networks only depending on the degree of the polynomial (not on the desired approximation accuracy). Using this result we can prove the main approximation bound, Theorem 3.2, by first approximating a given  $C^k$ -function using a polynomial in  $z$  and  $\overline{z}$  and then approximating this polynomial using Theorem 3.1.
For  $m,n\in \mathbb{N}$  let
$$
\mathcal {P} _ {m} ^ {n} := \left\{\mathbb {C} ^ {n} \to \mathbb {C}, z \mapsto \sum_ {\mathbf {m} \leq m} \sum_ {\ell \leq m} a _ {\mathbf {m}, \ell} z ^ {\mathbf {m}} \bar {z} ^ {\ell}: a _ {\mathbf {m}, \ell} \in \mathbb {C} \right\}
$$
denote the space of all complex polynomials on  $\mathbb{C}^n$  in  $z$  and  $\overline{z}$  of componentwise degree at most  $m$ . Here, we are summing over all multi-indices  $\mathbf{m}, \boldsymbol{\ell} \in \mathbb{N}_0^n$  with  $\mathbf{m}_j, \boldsymbol{\ell}_j \leq m$  for every  $j \in \{1, \dots, n\}$  and use the notation
$$
z ^ {\mathbf {m}} := \prod_ {j = 1} ^ {n} z _ {j} ^ {\mathbf {m} _ {j}} \quad \text {a n d} \quad \overline {{z}} ^ {\ell} := \prod_ {j = 1} ^ {n} \overline {{z _ {j}}} ^ {\ell_ {j}}.
$$
The space  $\mathcal{P}_m^n$  is finite-dimensional; hence, it makes sense to talk about bounded subsets of  $\mathcal{P}_m^n$  without specifying a norm.
Theorem 3.1. Let  $m, n \in \mathbb{N}, \varepsilon > 0$  and  $\phi: \mathbb{C} \to \mathbb{C}$  be smooth and non-polyharmonic on an open set  $\varnothing \neq U \subseteq \mathbb{C}$ . Let  $\mathcal{P}' \subseteq \mathcal{P}_m^n$  be bounded and set  $N := (4m + 1)^{2n}$ . Then there exists a first layer  $\Phi \in \mathcal{F}_{n,N}^{\phi}$  with the following property: For each polynomial  $p \in \mathcal{P}'$  there exists  $\sigma \in \mathbb{C}^N$ , such that
$$
\left\| p - \sigma^ {T} \Phi \right\| _ {L ^ {\infty} \left(\Omega_ {n}; \mathbb {C}\right)} \leq \varepsilon .
$$
Sketch of proof. For any multi-indices  $\mathbf{m},\ell \in \mathbb{N}_0^n$  an inductive argument shows for every fixed  $z\in \Omega_{n}$  and  $b\in \mathbb{C}$  that
$$
\partial_ {\mathrm {w i r t}} ^ {\mathbf {m}} \overline {{\partial}} _ {\mathrm {w i r t}} ^ {\ell} [ w \mapsto \phi (w ^ {T} z + b) ] = z ^ {\mathbf {m}} \bar {z} ^ {\ell} \cdot \left(\partial_ {\mathrm {w i r t}} ^ {| \mathbf {m} |} \overline {{\partial}} _ {\mathrm {w i r t}} ^ {| \ell |} \phi\right) (w ^ {T} z + b).
$$
Here,  $\partial_{\mathrm{wirt}}^{\mathbf{m}}$  and  $\overline{\partial}_{\mathrm{wirt}}^{\ell}$  denote the multivariate Wirtinger derivatives with respect to  $w$  according to the multi-indices  $\mathbf{m}$  and  $\ell$ , respectively. Evaluating this at  $w = 0$  and taking  $b \in \mathbb{C}$  such that none of the mixed Wirtinger derivatives of  $\phi$  at  $b$  up to a sufficiently high order vanish (where such a  $b$  exists by Proposition 2.1) shows that we can rewrite
$$
z ^ {\mathbf {m}} \bar {z} ^ {\boldsymbol {\ell}} = \left. \left(\partial_ {\mathrm {w i r t}} ^ {\mathbf {m}} \bar {\partial} _ {\mathrm {w i r t}} ^ {\boldsymbol {\ell}} [ w \mapsto \phi (w ^ {T} z + b) ]\right) \right| _ {w = 0} \cdot \left(\left(\partial_ {\mathrm {w i r t}} ^ {| \mathbf {m} |} \bar {\partial} _ {\mathrm {w i r t}} ^ {| \boldsymbol {\ell} |} \phi\right) (b)\right) ^ {- 1}. \tag {3.1}
$$
The mixed Wirtinger derivatives can by definition be expressed as linear combinations of usual partial derivatives. Those partial derivatives can be approximated using a generalized version of divided differences: If  $g \in C^k((-r, r)^s; \mathbb{R})$  and  $\mathbf{p} \in \mathbb{N}_0^s$  with  $|\mathbf{p}| \leq k$  we have
$$
\partial^ {\mathbf {p}} g (0) \approx (2 h) ^ {- | \mathbf {p} |} \sum_ {0 \leq \mathbf {r} \leq \mathbf {p}} (- 1) ^ {| \mathbf {p} | - | \mathbf {r} |} \binom {\mathbf {p}} {\mathbf {r}} g (h (2 \mathbf {r} - \mathbf {p})) \quad \text {f o r} h \searrow 0. \tag {3.2}
$$
See Appendix B.1 for a proof of this approximation. Note that when one takes  $g(w) = \phi (w^T z + b)$ , the right-hand side of (3.2) is a shallow neural network, as a function of  $z$ .
Combining (3.1) and (3.2) yields the desired result; see Appendix B.3 for the details.
It is crucial that the size of the networks considered in Theorem 3.1 is independent of the approximation accuracy  $\varepsilon$ . Moreover, the first layer  $\Phi$  can be chosen independently of the particular polynomial  $p$ . Only the weights  $\sigma$  connecting hidden layer and output neuron have to be adapted to  $p$ .
The final approximation result is as follows. Its full proof can be found in Appendix C.3.
Theorem 3.2. Let  $n, k \in \mathbb{N}$ . Then there exists a constant  $c = c(n, k) > 0$  with the following property: For any activation function  $\phi : \mathbb{C} \to \mathbb{C}$  that is smooth and non-polyharmonic on an open set  $\varnothing \neq U \subseteq \mathbb{C}$  and for any  $m \in \mathbb{N}$  there exists a first layer  $\Phi \in \mathcal{F}_{n,m}^{\phi}$  with the following property: For any  $f \in C^{k}(\Omega_{n};\mathbb{C})$  there exist coefficients  $\sigma = \sigma(f) \in \mathbb{C}^{m}$ , such that
$$
\left\| f - \sigma^ {T} \Phi \right\| _ {L ^ {\infty} \left(\Omega_ {n}; \mathbb {C}\right)} \leq c \cdot m ^ {- k / (2 n)} \cdot \| f \| _ {\mathcal {C} ^ {k} \left(\Omega_ {n}; \mathbb {C}\right)}.
$$