SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In particular, the values  $z_{k,\Delta x}$  that are distinct on the length scale  $\Delta x$  may collide when mapped to the length scale  $l\Delta x$ . However, in our setting we can bound the number of collisions thanks to the finiteness of the set of frequencies.
Specifically, consider length scales of the form  $2^{-t}\Delta x$ , with  $t = 0,1,\ldots$ . The total number of collisions over all  $t$  is bounded, since there are at most  $K = N$  distinct values  $z_{k}$ . It follows that at sufficiently large  $t$  there are no collisions, so that the number  $K = K_{t}$  of distinct values  $z_{k,2^{-t}\Delta x}$  is the same for all  $t$ , and we can enumerate these values so that for  $t_2 > t_1$
$$
z _ {k, 2 ^ {- t _ {1}} \Delta x} = z _ {k, 2 ^ {- t _ {2}} \Delta x} ^ {2 ^ {t _ {2} - t _ {1}}}. \tag {85}
$$
Moreover, observe that for sufficiently large  $t$  there will be no  $\omega_{-}$ -component (corresponding to  $z_{-} = -1$ ) in the (79)-(81). Indeed, since the square roots of  $-1$  are not real, the presence of the  $\omega_{-}$ -component for some  $t$  would mean that  $K_{t + 1} > K_t$ . But such an increase is possible only for a finite number of  $t$ 's.
Similarly, the  $\omega_{+}$ -component of the expansion (79)-(81) must have the form
$$
f _ {*, 0} (s \Delta x) = \sum_ {m = 0} ^ {2 M _ {+} - 1} c _ {+, m} (s \Delta x) ^ {m} \tag {86}
$$
with the same  $c_{+,m}$  for all sufficiently large  $t$ , since any change of this component with  $t$  is accompanied by an increase in  $K_{t}$ . The function  $f_{*,0}$  uniquely extends to a continuous function on the whole interval [0, 1],
$$
f _ {*, 0} (x) = \sum_ {m = 0} ^ {2 M _ {+} - 1} c _ {+, m} x ^ {m}, \tag {87}
$$
which agrees with the first component in the expansion (6).
This discussion shows that without loss of generality (by subtracting  $f_{*,0}$  from  $f_*$  if necessary), we may assume that expansion (79)-(81) contains only the last component, standing in line (81).
Assuming that transition from the length scale  $\Delta x$  to  $\Delta x' = l\Delta x$  occurs without collisions in  $z_{k}$ , the downscaling invariance condition can be written as
$$
c _ {k m, \Delta x} = c _ {k m, \Delta x ^ {\prime}}, \tag {88}
$$
$$
W _ {k, \Delta x} \subset W _ {k, \Delta x ^ {\prime}}, \tag {89}
$$
where the sets  $W_{k,\Delta x'}$  are defined by Eq. (82). By Lemma 4, the coefficients  $c_{km,\Delta x}$  are uniquely determined as soon as  $\Delta x$  and  $\Delta x'$  are small enough.
Considering again the length scales of the form  $2^{-t}\Delta x$ , we get a nested sequence
$$
\dots \subset W _ {k, 2 ^ {- 2} \Delta x} \subset W _ {k, 2 ^ {- 1} \Delta x} \subset W _ {k, \Delta x}. \tag {90}
$$
All the inclusions here are strict. The intersection
$$
W _ {k, \lim } = \cap_ {t = 0} ^ {\infty} W _ {k, 2 ^ {- t} \Delta x} \tag {91}
$$
is either empty or consists of a single frequency  $\{\omega_k\}$ . In the latter case we can again (as previously with  $f_{*,0}$ ) identify the respective component of the function  $f_*$ :
$$
f _ {*, k} (x) = \sum_ {m = 0} ^ {M _ {k} - 1} x ^ {m} \left(c _ {k m} e ^ {i \omega_ {k} x} + \overline {{c _ {k m}}} e ^ {- i \omega_ {k} x}\right). \tag {92}
$$
Our goal is to prove that the intersections  $W_{k,\lim}$  are indeed nonempty for all  $k$ . We will obtain this as a consequence of the continuity of the function  $f_{*}$ , which in turn follows from the fact that  $f_{*}$  is assumed to be a uniform limit of continuous functions  $f_{r}$ .
To this end, observe that when upscaling from  $2^{-t}\Delta x$  to  $2^{-t - 1}\Delta x$ , the value  $z_{k,2^{-t - 1}\Delta x}$  is one of the two square roots of  $z_{k,2^{-t}\Delta x}$ . The case  $W_{k,\lim} \neq \emptyset$  corresponds to the case when for all sufficiently large  $t$  we take the square root with positive real part, so that
$$
z _ {k, 2 ^ {- t} \Delta x} \xrightarrow {t \rightarrow \infty} 1. \tag {93}
$$
In contrast, in the case  $W_{k,\lim} = \emptyset$ , for any  $t_0$  there is  $t > t_0$  such that  $z_{k,2^{-t - 1}\Delta x}$  has a negative real part, so that
$$
z _ {k, 2 ^ {- t} \Delta x} \xrightarrow [ / \rightarrow ]{t \rightarrow \infty} 1. \tag {94}
$$
Now consider the vector  $\mathbf{u}^*$  obtained by sampling the function  $f_{*}$  at the points  $s\Delta x$  as in Eq. (67). Consider the sequence  $\mathbf{u}^{(*,t)}$  of vectors of the same dimension, obtained by sampling the function  $f_{*}$  with additional shifts  $2^{-t}\Delta x$ :
$$
u _ {s} ^ {(*, t)} = f _ {*} \left(\left(s + 2 ^ {- t}\right) \Delta x\right), \quad s = 0, 1, \dots , \dim \mathbf {u} ^ {*} - 1. \tag {95}
$$
Since  $f_{*}$  is continuous,
$$
\mathbf {u} ^ {(*, t)} \stackrel {t \rightarrow \infty} {\longrightarrow} \mathbf {u} ^ {*}. \tag {96}
$$
Recall that the vector  $\mathbf{u}^*$  admits an expansion (74) over the vectors  $\mathbf{v}_{\pm,m}, \mathbf{v}_{km}, \overline{\mathbf{v}_{km}}$ . As mentioned earlier, we can assume without loss of generality that there are no  $\mathbf{v}_{\pm,m}$  terms:
$$
\mathbf {u} ^ {*} = \sum_ {k = 1} ^ {K} \sum_ {m = 0} ^ {M _ {k} - 1} \left(b _ {k m} \mathbf {v} _ {k m} + \overline {{b _ {k m}}} \overline {{\mathbf {v} _ {k m}}}\right). \tag {97}
$$
Observe now that an analogous expansion can also be written for the shifted vectors:
$$
\mathbf {u} ^ {(*, t)} = \sum_ {k = 1} ^ {K} \sum_ {m = 0} ^ {M _ {k} - 1} \left(b _ {k m} \mathbf {v} _ {k m} ^ {(t)} + \overline {{b _ {k m}}} \overline {{\mathbf {v} _ {k m} ^ {(t)}}}\right). \tag {98}
$$
Moreover, the vectors  $\mathbf{v}_{km}^{(t)}$  are either exact or approximate multiples of the vectors  $\mathbf{v}_{km}$ :
$$
\mathbf {v} _ {k m} ^ {(t)} = \left\{\begin{array}{l l}z _ {k, 2 ^ {- t} \Delta x} \mathbf {v} _ {k m},&m = 0,\\z _ {k, 2 ^ {- t} \Delta x} \mathbf {v} _ {k m} + o (1)&(t \rightarrow \infty), \quad m > 0.\end{array}\right. \tag {99}
$$
By Lemma 4, the vectors  $\mathbf{v}_{km}$  and  $\overline{\mathbf{v}_{km}}$  are linearly independent for sufficiently small  $\Delta x$ , so convergence (96) requires all those coefficients  $z_{k,2^{-t}\Delta x}$  for which  $b_{km} \neq 0$  at least for one  $m$  to converge to 1 (as in Eq. (93)). This proves that all  $W_{k,\lim}$  are non-empty for such  $k$ . If for some  $k$  we have  $b_{km} = 0$  for all  $m$ , then this  $k$ -component is effectively missing and can be ignored. This completes the proof of the theorem.
B Proof of Theorem 5
Our proof will consist of three steps. In the first step we show that the polynomial constraint claimed in the theorem holds for functions of the form  $e^{P(x)}$ . In the second step we show that if polynomial constraints hold for the arguments of an algebraic function, then its values also admit a polynomial constraint. In the final third step we combine these results to complete the proof of the theorem.
We remark that our arguments are close to the arguments of the theory of algebraic-transcendent functions [16, 17], but we work in a discretized setting of uniformly sampled data without considering any derivatives.
Step 1: Polynomial constraints for exponentials of polynomials. Consider the function
$$
g (x) = e ^ {P (x)}, \tag {100}
$$
where  $P$  is a (possibly complex-valued) polynomial of degree  $\leq d$ . Consider operators  $D_h^s$  of forward discrete derivatives of order  $s$  with step  $h$ :
$$
D _ {h} ^ {s} g (x) = \frac {1}{h} \left(D _ {h} ^ {s - 1} g (x + h) - D _ {h} ^ {s - 1} g (x)\right) = \dots = h ^ {- s} \sum_ {k = 0} ^ {s} (- 1) ^ {s - k} \binom {s} {k} g (x + k h). \tag {101}
$$
We have  $D_h^{d+1}P = 0$  for any  $h \in \mathbb{C}$  and any polynomial  $P$  of degree  $\leq d$ :
$$
\sum_ {k = 0} ^ {d + 1} (- 1) ^ {d + 1 - k} \binom {d + 1} {k} P (x + k h) = 0. \tag {102}
$$
Separating the positive and negative coefficients, we can write this, equivalently, as
$$
\sum_ {k = 0} ^ {\lfloor (d + 1) / 2 \rfloor} \binom {d + 1} {2 k} P (x + 2 k h) = \sum_ {m = 0} ^ {\lfloor d / 2 \rfloor} \binom {d + 1} {2 m + 1} P (x + (2 m + 1) h). \tag {103}
$$
Exponentiating, it follows for  $g(x)$  given by Eq. (100) that
$$
\prod_ {k = 0} ^ {\lfloor (d + 1) / 2 \rfloor} (g (x + 2 k h)) ^ {\binom {d + 1} {2 k}} - \prod_ {m = 0} ^ {\lfloor d / 2 \rfloor} (g (x + (2 m + 1) h)) ^ {\binom {d + 1} {2 m + 1}} = 0. \tag {104}
$$
We can view this as a polynomial constraint of the form
$$
R (g (x), g (x + h), \dots , g (x + (d + 1) h)) = 0 \tag {105}
$$
with some fixed polynomial  $R$  of  $d + 2$  variables.
Step 2: Extension of polynomial constraints to algebraic functions.
1. Suppose that  $Q(y_0, y_1, \ldots, y_N)$  is an algebraic function, and  $g_0(x), \ldots, g_N(x)$  are functions satisfying polynomial relations
$$
R _ {n} \left(g _ {n} (x), g _ {n} (x + h), \dots , g _ {n} (x + s h)\right) = 0, \quad \forall x, h. \tag {106}
$$
Let  $g(x) = Q(g_0(x), \ldots, g_N(x))$  and  $m \geq s(N + 1)$ . Then there exists a nontrivial polynomial  $R$  of  $m + 1$  variables such that
$$
R (g (x), g (x + h), \dots , g (x + m h)) = 0, \quad \forall x, h. \tag {107}
$$
2. Moreover, suppose that the function  $Q$  is not fixed, and is only constrained by its degree  $\deg Q \leq q$  and the degrees  $\deg \phi_k \leq r$  of the polynomials appearing in its definition. Let  $m \geq (q + 1)\binom{N + r + 1}{N + 1} + s(N + 1)$ . Then there exists a nontrivial polynomial  $R$  of  $m + 1$  variables such that relation (107) holds for all such  $Q$ .
Proof. 1. Recall the defining relation (7) of the algebraic function  $Q$ :
$$
\sum_ {k = 0} ^ {q} \phi_ {k} \left(z _ {0}, \dots , z _ {N}\right) Q ^ {k} \left(z _ {0}, \dots , z _ {N}\right) = 0. \tag {108}
$$
We can use this relation to express  $Q^q$  in terms of lower powers  $Q^0, \ldots, Q^{q-1}$ :
$$
Q ^ {q} \left(z _ {0}, \dots , z _ {N}\right) = - \phi_ {q} ^ {- 1} \left(z _ {0}, \dots , z _ {N}\right) \sum_ {k = 0} ^ {q - 1} \phi_ {k} \left(z _ {0}, \dots , z _ {N}\right) Q ^ {k} \left(z _ {0}, \dots , z _ {N}\right). \tag {109}
$$
By iterating this relation, for any power  $t \geq q$  we can express  $Q^t$  in terms of the powers  $Q^0, \ldots, Q^{q-1}$ :
$$
Q ^ {t} \left(z _ {0}, \dots , z _ {N}\right) = \phi_ {q} ^ {- t} \left(z _ {0}, \dots , z _ {N}\right) \sum_ {k = 0} ^ {q - 1} \phi_ {t, k} \left(z _ {0}, \dots , z _ {N}\right) Q ^ {k} \left(z _ {0}, \dots , z _ {N}\right), \tag {110}
$$
where  $\phi_{t,k}$  are suitable polynomials of degree not greater than  $t\max_{k = 0,\dots ,q}\deg \phi_k$ . Substituting
$$
z _ {n} = g _ {n} (x),
$$
$$
Q ^ {t} (\mathbf {g} (x)) = \phi_ {q} ^ {- t} (\mathbf {g} (x)) \sum_ {k = 0} ^ {q - 1} \phi_ {t, k} (\mathbf {g} (x)) Q ^ {k} (\mathbf {g} (x)), \tag {111}
$$
where for brevity we have denoted  $\mathbf{g}(x) = (g_0(x),\dots ,g_N(x))$
Formula (111) remains valid if we replace  $x$  by  $x + lh$  with any  $l$ . Consider various monomials in the variables  $Q(\mathbf{g}(x + lh))$  for  $l = 0, \ldots, m$ :
$$
\begin{array}{l} F _ {\mathbf {t}} = \prod_ {l = 0} ^ {m} Q ^ {t _ {l}} (\mathbf {g} (x + l h)) (112) \\ = \left(\prod_ {l = 0} ^ {m} \phi_ {q} ^ {- t _ {l}} (\mathbf {g} (x + l h))\right) \sum_ {k _ {0} = 0} ^ {q - 1} \dots \sum_ {k _ {m} = 0} ^ {q - 1} \widetilde {\Phi} _ {\mathbf {t}, \mathbf {k}} (\mathbf {g} (x), \dots , \mathbf {g} (x + m h)) \mathbf {Q} ^ {\mathbf {k}}, (113) \\ \end{array}
$$
where  $\mathbf{t} = (t_0,\dots ,t_m),\mathbf{k} = (k_0,\dots ,k_m)$ $\widetilde{\Phi}_{\mathbf{t},\mathbf{k}}$  are some polynomials of degree not greater than  $(m + 1)(\max_l t_l)\max_k\deg \phi_k$  , and
$$
\mathbf {Q} ^ {\mathbf {k}} = \prod_ {l = 0} ^ {m} Q ^ {k _ {l}} (\mathbf {g} (x + l h)). \tag {114}
$$
Our goal is to prove that for sufficiently large  $m$ , the monomials  $F_{\mathbf{t}}$  are linearly dependent (with some coefficients independent of  $x$  and  $h$ ), which is exactly equivalent to the existence of a polynomial  $R$  satisfying relation (107). This is done by showing a linear dependence of the expressions (113) with different  $\mathbf{t}$ .
Specifically, observe that, regardless of  $\mathbf{t}$ , expressions (113) contain only finitely many (namely,  $q^{m + 1}$ ) different components  $\mathbf{Q}^{\mathbf{k}}$ . Since there are infinitely many monomials  $F_{\mathbf{t}}$ , it follows already from this observation that there exist nontrivial vanishing linear combinations of the monomials  $F_{\mathbf{t}}$  if the coefficients in these linear combinations are allowed to belong to the field of all rational functions of  $\mathbf{g}(x), \ldots, \mathbf{g}(x + mh)$ . However, we need to show that nontrivial vanishing linear combinations can be found even with constant coefficients.
To this end, we will further reduce the combinatorial complexity of expressions (113) by removing high powers of the variables  $g_{n}(x + lh)$  at the expense of introducing new rational functions of the variables  $g_{n}(x + l'h)$  with  $l' > l$ . Recall that each  $g_{n}$  is subject to polynomial relation (106). We can use these relations to transform polynomial functions of  $\mathbf{g}(x)$  into rational functions of  $\mathbf{g}(x), \mathbf{g}(x + h), \ldots, \mathbf{g}(x + sh)$  that contain only bounded powers of  $\mathbf{g}(x)$ . Specifically, isolate the variables  $g_{n}(x)$  in the polynomials  $R_{n}$  by writing equations (106) in the form
$$
\sum_ {j = 0} ^ {d _ {n}} \psi_ {n, j} \left(g _ {n} (x + h), \dots , g _ {n} (x + s h)\right)) g _ {n} ^ {j} (x) = 0 \tag {115}
$$
with suitable polynomials  $\psi_{n,j}$  of the remaining variables  $g_{n}(x + h),\ldots ,g_{n}(x + sh)$ . The value  $d_{n}$  is the degree of  $g_{n}(x)$  in this representation. Then, similarly to how we did this for  $Q$ , we can express
$$
g _ {n} ^ {d _ {n}} (x) = - \psi_ {n, d _ {n}} ^ {- 1} \left(g _ {n} (x + h), \dots , g _ {n} (x + s h)\right)) \sum_ {j = 0} ^ {d _ {n} - 1} \psi_ {n, j} \left(g _ {n} (x + h), \dots , g _ {n} (x + s h)\right)) g _ {n} ^ {j} (x). \tag {116}
$$
By iterating this relation, for any  $t \geq d_n$  we get
$$
g _ {n} ^ {t} (x) = \psi_ {n, d _ {n}} ^ {- t} \left(g _ {n} (x + h), \dots , g _ {n} (x + s h)\right)) \sum_ {j = 0} ^ {d _ {n} - 1} \psi_ {n, j, t} \left(g _ {n} (x + h), \dots , g _ {n} (x + s h)\right)) g _ {n} ^ {j} (x), \tag {117}
$$
with some polynomials  $\psi_{n,j,t}$  of degree not greater than  $t\max_j\deg \psi_{n,j}$ . This relation remains valid if we replace  $x$  by  $x + lh$  with any  $l$ :
$$
\begin{array}{l} g _ {n} ^ {t} (x + l h) = \psi_ {n, d _ {n}} ^ {- t} \left(g _ {n} (x + (l + 1) h), \dots , g _ {n} (x + (l + s) h)\right)) (118) \\ \times \sum_ {j = 0} ^ {d _ {n} - 1} \psi_ {n, j, t} \left(g _ {n} (x + (l + 1) h), \dots , g _ {n} (x + (l + s) h))\right) g _ {n} ^ {j} (x + l h). (119) \\ \end{array}
$$
We can now transform expression (113) by first replacing there all powers  $t \geq d_n$  of  $g_{n}(x)$  with powers  $t \leq d_{n}$  and rational functions of  $g_{n}(x + h),\ldots ,g_{n}(x + sh)$ , then replacing powers  $t \geq d_{n}$  of  $g_{n}(x + h)$  with powers  $t \leq d_{n}$  and rational functions of  $g_{n}(x + 2h),\ldots ,g_{n}(x + (s + 1)h)$ , and so on. We continue this process while the involved variables  $g(x + lh)$  obey the condition  $l \leq m$ . As a result, we obtain a rational function of the variables  $\mathbf{g}(x),\dots ,\mathbf{g}(x + mh)$  that contains only powers of the variables  $g_{n}(x),\ldots ,g_{n}(x + (m - s)h)$  not exceeding  $d_{n} - 1$ .
To establish the desired linear dependence, we want the rational functions appearing in these transformations of the monomials  $F_{\mathbf{t}}$  to have the same denominator for all  $\mathbf{t}$ . It is convenient to ensure this in the following way. Choose some large  $T$  and assume that all the powers  $t_k$  in the monomials  $F_{\mathbf{t}}$  vary between 0 and  $T$ , so that in total we have  $(T + 1)^{m + 1}$  monomials. In expression (113), instead of the denominator  $\prod_{l = 0}^{m}\phi_q^t (\mathbf{g}(x + lh))$  use the common denominator
$$
Z _ {0} = \prod_ {l = 0} ^ {m} \phi_ {q} ^ {T} (\mathbf {g} (x + l h)). \tag {120}
$$
This changes the polynomials  $\tilde{\Phi}_{\mathbf{t},\mathbf{k}}$  ..
$$
F _ {\mathbf {t}} = Z _ {0} ^ {- 1} \sum_ {\mathbf {k}} \Phi_ {\mathbf {t}, \mathbf {k}} (\mathbf {g} (x), \dots , \mathbf {g} (x + m h)) \mathbf {Q} ^ {\mathbf {k}}; \tag {121}
$$
the new polynomials  $\Phi_{\mathbf{t},\mathbf{k}}$  now have degrees not exceeding  $(m + 1)T\max_k\deg \phi_k$ . Now we perform the iterative substitutions of the powers  $g_n^t (x + km)$  in the polynomials  $\Phi_{\mathbf{t},\mathbf{k}}$  as described above, while keeping the same denominator for all  $\mathbf{t}$ ; this requires multiplying the common denominator by appropriate powers of  $\psi_{n,d_n}(g_n(x + (l + 1)h),\ldots ,g_n(x + (l + s)h)))$ . As a result of this process we obtain expressions
$$
F _ {\mathbf {t}} = Z ^ {- 1} \sum_ {\mathbf {k}} \mathbf {Q} ^ {\mathbf {k}} \sum_ {J} \left(\prod_ {\substack {n = 0, \dots , N \\ l = 0, \dots , m - s}} g _ {n} ^ {J _ {n l}} (x + l h)\right) \Psi_ {\mathbf {t}, \mathbf {k}, J} (\mathbf {g} (x + (m - s + 1) h), \dots , \mathbf {g} (x + m h)). \tag{122}
$$
Here  $J = (J_{nl})$  is a matrix with elements  $0 \leq J_{nl} \leq d_n - 1$ , so summation over  $J$  is finite. The polynomials  $\Psi_{\mathbf{t},\mathbf{k},J}$  only depend on the remaining variables  $\mathbf{g}(x + (m - s + 1)h), \ldots, \mathbf{g}(x + mh)$ .
Importantly, by taking into account previously observed bounds on the degrees of polynomials appearing in Eqs. (119) and (121), we observe that  $\deg \Psi_{\mathbf{t},\mathbf{k},J} \leq CT$  with some constant  $C$  depending on  $s, m, N$  and the degrees of the polynomials  $\phi_k$  and  $\psi_{n,j}$  appearing in the assumed expansions (108) and (115).
Now the desired linear dependence of the monomials  $F_{\mathbf{t}}$  results, with a suitable choice of  $m$  and  $T$ , by comparing the number  $(T + 1)^{m + 1}$  of different monomials  $F_{\mathbf{t}}$  with the number of different monomials that may appear in the expansions of the polynomials  $\Psi_{\mathbf{t},\mathbf{k},J}$ . Since these polynomials depend on the  $s(N + 1)$  variables  $\mathbf{g}(x + (m - s + 1)h),\ldots ,\mathbf{g}(x + mh)$  and have degrees not exceeding  $CT$ , the number of different monomials in them does not exceed  $C_1T^{s(N + 1)}$  with some constant  $C_1$  depending on the same parameters as  $C$ . Taking into account that the summations over  $\mathbf{k}$  and  $J$  in expansion (122) involve, respectively,  $q^{m + 1}$  and  $\prod_{n = 0}^{N}d_n^{m - s + 1}$  terms, we see that the  $(T + 1)^{m + 1}$  monomials  $F_{\mathbf{t}}$  belong to a  $C_1q^{m + 1}(\prod_{n = 0}^N d_n^{m - s + 1})T^{s(N + 1)}$ -dimensional linear functional space. Accordingly, if  $s(N + 1) < m + 1$ , then some of the monomials  $F_{\mathbf{t}}$  will be linearly dependent if  $T$  is large enough. This completes the proof of part 1 of the lemma.
2. The argument above produces nontrivial linear combinations of monomials  $F_{\mathbf{t}}$  with coefficients generally dependent on  $Q$ . We show now that, assuming appropriate degree bounds for  $Q$  and choosing a larger  $m$ , one can find nontrivial linear combinations with  $Q$ -independent coefficients. To this end, note that the main obstacle to choosing  $Q$ -independent coefficients is that the polynomials  $\Psi_{\mathbf{t},\mathbf{k},J}$  in expansion (122) are  $Q$ -dependent. To overcome this obstacle, we express these polynomials both in the variables  $\mathbf{g}(x + (m - s + 1)h), \ldots, \mathbf{g}(x + mh)$  and the coefficients  $\mathbf{b}$  of the polynomials  $\phi_k$ :
$$
\Psi_ {\mathbf {t}, \mathbf {k}, J} (\mathbf {g} (x + (m - s + 1) h), \dots , \mathbf {g} (x + m h)) = \widetilde {\Psi} _ {\mathbf {t}, \mathbf {k}, J} (\mathbf {b}, \mathbf {g} (x + (m - s + 1) h), \dots , \mathbf {g} (x + m h)). \tag {123}
$$
Here, the vector  $\mathbf{b}$  includes all the coefficients of all  $q + 1$  polynomials  $\phi_{k}$  appearing in the defining identity (108). By retracing the steps to construct the polynomials  $\Psi_{\mathbf{t},\mathbf{k},J}$ , we see that their dependence on the coefficients  $\mathbf{b}$  is indeed polynomial. Moreover,  $\deg \widetilde{\Psi}_{\mathbf{t},\mathbf{k},J}$  is again bounded by  $CT$ , where  $C$  is some constant possibly dependent on  $N$ ,  $s$ ,  $m$ ,  $q$ ,  $\max \deg \phi_{k}$ ,  $\max_{j} \deg \psi_{n,j}$ , but not on  $T$ . Since we assume  $\deg \phi_{k} \leq r$ , we have  $\dim \mathbf{b} \leq (q + 1)\binom{N + r + 1}{N + 1}$ . The total number of variables in  $\widetilde{\Psi}_{\mathbf{t},\mathbf{k},J}$  is thus not greater than
$$
p = (q + 1) \binom {N + r + 1} {N + 1} + s (N + 1). \tag {124}
$$
We repeat now the dimensionality-based linear dependence argument. The total dimensionality of polynomials  $\widetilde{\Psi}_{\mathbf{t},\mathbf{k},J}$  considered at all  $\mathbf{k}, J$  is  $O(T^{p})$ . On the other hand, the number of monomials  $F_{\mathbf{t}}$  is  $(T + 1)^{m + 1}$ . If we set  $m \geq p$ , then for sufficiently large  $T$  we can find a nontrivial vanishing linear combination of  $F_{\mathbf{t}}$  with coefficients independent of  $Q$ .
Step 3: Proof of Theorem 5. We apply now Lemma 5 to our setting in which  $g_0(x) = x$  and  $g_k(x) = e^{P_k(x)}$ ,  $k = 1, \ldots, N$ . We know from Step 1 that for  $k = 1, \ldots, N$  conditions (106) hold with  $s = \deg P_k + 1$ . For  $g_0$ , condition (106) holds with  $s = 2$ :
$$
g _ {0} (x) - 2 g _ {0} (x + h) + g _ {0} (x + 2 h) = 0. \tag {125}
$$
Since we assumed that  $\deg P_k \leq d$ , we can set  $s = \max(d, 1) + 1$  as a common value with which condition (106) holds for all  $n = 0, \dots, N$ . The conclusion of Theorem 5 then follows from statement 2 of Lemma 5.
C Proof of Theorem 6
Since the network has not more than  $2^{N}$  hidden neurons and each of our activation functions has at most two branches, the output  $g(x)$  of the network at any  $x$  belongs to one of at most  $2^{N}$  branches represented by analytic expressions. Consider each of these branches as defined for all  $x \in [0,1]$  and denote the family of all branches resulting from all  $g \in H_N^{(4)}$  by  $\widetilde{H}_N^{(4)}$ . Observe that  $\widetilde{H}_N^{(4)} \subset H_{N',q,r,d}^{(3)}$  if  $N', q,r,d$  are large enough. Indeed, the input of any algebraic-exponential neuron is a polynomial of degree not greater than  $d = 2^{N}$  (if the squared ReLU is present; otherwise, if only the step function and ReLU/leaky ReLU are present, then one can take  $d = 1$ ). All the considered algebraic-exponential activations are rational functions of  $x, e^x$  or  $e^{\pm ix}$ , and the subsequent neurons are polynomial, so any branch of the full network can be written in the form (8) with a rational  $Q$  and  $2N$  as the number of its exponential arguments. Rational  $Q$ 's have algebraic degree  $q = 1$ . The maximum degree of the
polynomials  $\phi_k$  appearing in the defining relation (7) does not exceed  $r = 2^N$  (again, if the squared ReLU is present; otherwise one can take  $r = 1$ ). Summarizing,
$$
\widetilde {H} _ {N} ^ {(4)} \subset H _ {2 N, 1, 2 ^ {N}, 2 ^ {N}} ^ {(3)}. \tag {126}
$$
Theorem 5 implies now that there exists some  $\widetilde{m}$  and a nonzero polynomial  $\widetilde{R}$  of  $\widetilde{m} + 1$  variables such that for any  $\widetilde{g} \in \widetilde{H}_N^{(4)}$  and any size-  $(\widetilde{m} + 1)$  arithmetic progression  $\widetilde{a}, \widetilde{a} + \widetilde{h}, \ldots, a + \widetilde{m}\widetilde{h} \in [0,1]$
$$
\widetilde {R} (\widetilde {g} (\widetilde {a}), \widetilde {g} (\widetilde {a} + \widetilde {h}), \widetilde {g} (\widetilde {a} + 2 \widetilde {h}), \dots , \widetilde {g} (\widetilde {a} + \widetilde {m} \widetilde {h})) = 0. \tag {127}
$$