SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Comparison with TSSOS on the Chebychev basis. We compare GloptiNets with TSSOS on random Chebychev polynomials in table 2, similarly to the comparison with trigonometric polynomials in table 1. Minimizing polynomials defined on the canonical basis is easier: contrary to trigonometric polynomials, there is no need to account for the imaginary part of the variable. If  $d$  is the dimension, complex polynomials are encoded in a variable of dimension  $2d$  in TSSOS, following the definition of Hermitian Sum-of-Squares introduced in Josz and Molzahn [2018]. Hence, the random polynomials we consider are characterized by the dimension  $d$  and their number of coefficients  $n$ ; instead of bounding the degree, we use all the basis elements  $H_{\omega}(\mathbf{x}) = \prod_{\ell=1}^{d} H_{\omega_{\ell}}(\mathbf{x}_{\ell})$  for which  $\| \omega \|_{\infty} \leq p$ . The maximum degree is then  $dp$ . The RKHS norm of  $f$  is fixed to 1. As with the comparison on Trigonometric polynomial table 1, we see that GloptiNets provides similar certificates no matter the number of coefficients in  $f$ . Even though it lags behind TSSOS for small polynomials, it handles large polynomials which are intractable to TSSOS. The "small" and "big" models have the same structure as for the trigonometric polynomials experiments.
Sampling from the Bessel distribution. The function  $\omega \mapsto e^{-s}I_{\omega}(s)$  decays rapidly. In fact, with  $s = 2$ , which is the value used to generate the random polynomials, it falls under machine precision as soon as  $\omega > 14$ . Thus, we approximate the distribution with a discrete one with weights  $I_{\omega}(s)$  for  $\omega$  s.t. the result is above the machine precision. We then extend it to multiple dimension with a tensor product. Finally, we use a hash table to store the already sampled frequency, to make the evaluation of million of frequencies much faster. For instance in dimension 5, sampling  $10^{6}$  frequencies from the Bessel distribution of parameter  $s = 2$  on  $\mathbb{N}^5$  yields only  $\approx 10^4$  unique frequencies. This allows for tighter certificates, as it makes the r.h.s of eq. (9), in  $1/N$ , much smaller. Note that the time to generate this hash table is not reported in tables 1 and 2, and of the order of a few seconds.
Optimizing a kernel mixture. As it is the case with polynomials, when optimizing a function of the form  $h(x) = \sum_{i=1}^{n} \alpha_i K(x_i, x)$  the certificate provided by GloptiNets only depends on the function norm  $\| h \|_{\mathcal{H}}^2$  and not on  $e.g.$  the number of coefficients  $n$ . This is illustrated in fig. 4.
D Fourier coefficients in linear time
Lemma 4 (Fourier coefficient of the Bessel kernel in linear time). Let  $g$  be an extended  $k$ -SoS model as in definition 1. Then, its Fourier transform can be evaluated in linear time in  $m$  with
$$
\widehat {g} _ {\omega} = \sum_ {k = 1} ^ {r} \sum_ {n \in \mathbb {Z} ^ {d}} \left(\sum_ {i = 1} ^ {m} R _ {k i} \prod_ {\ell = 1} ^ {d} \phi_ {\ell , -} (\mathbf {z} _ {i \ell}) _ {n _ {\ell}}\right) \cdot \left(\sum_ {i = 1} ^ {m} R _ {k i} \prod_ {\ell = 1} ^ {d} \phi_ {\ell , +} (\mathbf {z} _ {i \ell})\right) \tag {32}
$$
$$
\forall n \in \mathbb {Z}, z \in \mathbb {T}, \ell \in [ d ], \phi_ {\ell , \pm} (z) _ {n} = \sqrt {q _ {\ell , n}} e ^ {\pi \mathrm {i} (n \pm \omega_ {\ell}) z}
$$
and  $q_{\cdot ,\cdot}(s)$  is defined with lemma 6.
Lemma 4 provides a formula for computing  $\widehat{g}_{\omega}$  which is linear in  $m$ , but which still requires numerical approximation to compute the sum on  $n \in \mathbb{Z}^d$ . For instance, restraining the sum to the hyperbolic
Figure 4: Certificate vs. number of parameters in  $g$  when certifying mixture of Bessel functions, characterized by their RKHS norm (1 in blue, 2 in red) and their number of coefficients (10 in circles, 100 in rectangles). As with polynomials, this shows that GloptiNets is only sensible to the former, and not to the way the function is represented. We are not aware of other algorithms able to certify this class of functions.
cross Dung et al. [2017]
$$
\operatorname {H C} (d, n) = \left\{\omega \in \mathbb {Z} ^ {d}; \prod_ {\ell = 1} ^ {d} \max  \left\{1, | \omega_ {\ell} | \right\} \leq n \right\}
$$
would result in a complexity of  $O(n(\log d)^n mr)$  and should produce reasonably accurate estimate of  $\widehat{g}_{\omega}$  for low  $n$ .
Furthermore, since  $q$  is real-even w.r.t  $n$ , the inner-product in eq. (36) can be simplified by computing only half of the terms.
Proof. From lemma 1, we have that
$$
\widehat {g} _ {\omega} = \sum_ {i, j = 1} ^ {m} R _ {i} ^ {\top} R _ {j} \prod_ {\ell = 1} ^ {d} e ^ {- 2 \mathbf {s} _ {\ell}} I _ {| \omega_ {\ell} |} \left(2 \mathbf {s} _ {\ell} \cos \pi \left(\mathbf {z} _ {i \ell} - \mathbf {z} _ {j \ell}\right)\right) e ^ {- \mathrm {i} \pi \omega_ {\ell} \left(\mathbf {z} _ {i \ell} + \mathbf {z} _ {j \ell}\right)}. \tag {33}
$$
$$
f _ {\ell} (x, y) = e ^ {- 2 \mathbf {s} _ {\ell}} I _ {| \omega_ {\ell} |} (2 \mathbf {s} _ {\ell} \cos \pi (x - y)) e ^ {- \mathrm {i} \pi \omega_ {\ell} (x + y)}, \tag {34}
$$
eq. (33) simplifies to
$$
\widehat {g} _ {\omega} = \sum_ {i, j = 1} ^ {m} R _ {i} ^ {\top} R _ {j} \prod_ {\ell = 1} ^ {d} f _ {\ell} \left(\mathbf {z} _ {i \ell}, \mathbf {z} _ {j \ell}\right). \tag {35}
$$
Using lemma 6, for any  $x,y\in \mathbb{T}$
$$
e ^ {- 2 \mathbf {s} _ {\ell}} I _ {| \omega_ {\ell} |} (2 \mathbf {s} _ {\ell} \cos \pi (x - y)) = \sum_ {n \in \mathbb {Z}} q _ {\ell , n} e ^ {\pi \mathrm {i} n (x - y)}
$$
$(q_{\ell ,n}$  depends on  $\omega_{\ell})$  so that,  $f_{\ell}$  defined in eq. (34) now writes
$$
\begin{array}{l} f _ {\ell} (x, y) = \sum_ {n \in \mathbb {Z}} q _ {\ell , n} e ^ {\pi \mathrm {i} n (x - y)} e ^ {- \pi \mathrm {i} \omega_ {\ell} (x + y)} \\ = \sum_ {n \in \mathbb {Z}} q _ {\ell , n} e ^ {\pi \mathrm {i} (n - \omega_ {\ell}) x} e ^ {- \pi \mathrm {i} (n + \omega_ {\ell}) y} \\ = \phi_ {\ell , -} (x) \cdot \phi_ {\ell , +} (y) \tag {36} \\ \end{array}
$$
where, for any  $\ell \in \{1,\dots ,d\}$  and  $z\in \mathbb{T}$ , we defined
$$
\phi_ {\ell , \pm} (z) = \left(\sqrt {q _ {\ell , n}} e ^ {\pi \mathrm {i} (n \pm \omega_ {\ell}) z}\right) _ {n \in \mathbb {Z}}. \tag {37}
$$
We then define the embedding  $\phi_{\pm}:\mathbb{T}\to (\mathbb{Z}^d\to \mathbb{C})$  be the tensor product of the  $\phi_{\ell ,\pm}$ . Then, eq. (36), enables to write  $\widehat{g}_{\omega}$  in eq. (35) as
$$
\begin{array}{l} \widehat {g} _ {\omega} = \sum_ {i, j = 1} ^ {m} \sum_ {k = 1} ^ {r} R _ {k i} R _ {k j} \phi_ {-} (\mathbf {z} _ {i}) \cdot \phi_ {+} (\mathbf {z} _ {j}) \\ = \sum_ {k = 1} ^ {r} \left[ \sum_ {i = 1} ^ {m} R _ {k i} \phi_ {-} (\mathbf {z} _ {i}) \right] \cdot \left[ \sum_ {i = 1} ^ {m} R _ {k i} \phi_ {+} (\mathbf {z} _ {i}) \right] \\ = \sum_ {k = 1} ^ {r} \left[ \sum_ {i = 1} ^ {m} R _ {k i} \phi_ {1, -} (\mathbf {z} _ {i 1}) \otimes \dots \otimes \phi_ {d, -} (\mathbf {z} _ {i d}) \right] \cdot \left[ \sum_ {i = 1} ^ {m} R _ {k i} \phi_ {1, +} (\mathbf {z} _ {i 1}) \otimes \dots \otimes \phi_ {d, +} (\mathbf {z} _ {i d}) \right] \\ = \sum_ {k = 1} ^ {r} \sum_ {n \in \mathbb {Z} ^ {d}} \left(\sum_ {i = 1} ^ {m} R _ {k i} \prod_ {\ell = 1} ^ {d} \phi_ {\ell , -} (\mathbf {z} _ {i \ell}) _ {n _ {\ell}}\right) \cdot \left(\sum_ {i = 1} ^ {m} R _ {k i} \prod_ {\ell = 1} ^ {d} \phi_ {\ell , +} (\mathbf {z} _ {i \ell})\right) \\ \end{array}
$$
E Other computation
Lemma 5. Let  $f$  be the function defined on  $(-1,1)$  with
$$
\forall u \in (0, 1 / 2), f (\cos 2 \pi u) = e ^ {s \cos 2 \pi (u - v)}. \tag {38}
$$
Then, its Chebychev coefficient are given with
$$
f _ {\omega} = \left(1 + \mathbf {1} _ {\omega \neq 0}\right) \cos (2 \pi \omega v) I _ {\omega} (s). \tag {39}
$$
Proof. The  $\omega \in \mathbb{N}_{*}$ . The component  $\omega$  of a function  $f$  on the Chebychev basis is given with
$$
f _ {\omega} = \frac {2}{\pi} \int_ {- 1} ^ {1} f (x) T _ {\omega} (x) \frac {\mathrm {d} x}{\sqrt {1 - x ^ {2}}},
$$
which we conveniently rewrite, with the classical change of variable  $x = \cos 2\pi u$
$$
f _ {\omega} = 2 \int_ {I _ {1}} f (\cos 2 \pi u) \cos (2 \pi \omega u) \mathrm {d} u \tag {40}
$$
which is valid for any interval  $I_{1}\subset \mathbb{R}$  of length 1.
Now, for  $s > 0$ , consider the function  $f$  defined on  $(-1,1)$  with  $x \mapsto e^{s\cos(\arccos(x) - 2\pi v)}$ , or equivalently
$$
\forall u \in (0, 1 / 2), f (\cos 2 \pi u) = e ^ {s \cos 2 \pi (u - v)}. \tag {41}
$$
Putting eq. (41) into eq. (40), we obtain
$$
\begin{array}{l} f _ {\omega} = 2 \int_ {I _ {1}} e ^ {s \cos 2 \pi (u - v)} \cos (2 \pi \omega u) d u \\ = 2 \int_ {I _ {1}} e ^ {s \cos 2 \pi u} \cos (2 \pi \omega (u + v)) \mathrm {d} u \\ = 2 \int_ {I _ {1}} e ^ {s \cos 2 \pi u} \cos (2 \pi \omega u) \cos (2 \pi \omega v) \mathrm {d} u - 2 \int_ {I _ {1}} e ^ {s \cos 2 \pi u} \sin (2 \pi \omega u) \sin (2 \pi \omega v) \mathrm {d} u. \\ \end{array}
$$
The last term is odd, hence integrate to 0 on an interval centered around 0. Hence,
$$
f _ {\omega} = 2 \cos (2 \pi \omega v) \int_ {I _ {1}} e ^ {s \cos 2 \pi u} \cos (2 \pi \omega u) d u. \tag {42}
$$
We recognize the definition of the modified Bessel function of the first kind, defined in eq. (14). Plugging this into eq. (42), we obtain
$$
f _ {\omega} = 2 \cos (2 \pi \omega v) I _ {\omega} (s) = 2 I _ {\omega} (s) H _ {\omega} (\cos (2 \pi v)). \tag {43}
$$
If  $\omega = 0$ , we add a factor  $1/2$  into the definition in eq. (40), which yields
$$
f _ {\omega} = I _ {0} (s). \tag {44}
$$
Lemma 6 (Fourier decomposition of Bessel composed with cosine). Let  $s > 0$ ,  $\omega \in \mathbb{N}$  and  $z \in \mathbb{T}$ . Then,
$$
e ^ {- 2 s} I _ {\omega} (2 s \cos 2 \pi z) = \sum_ {n \in \mathbb {Z}} q _ {\omega , n} e ^ {2 \pi \mathrm {i} n z},
$$
$$
w h e r e \forall n \geq 0, q _ {\omega , n} = \left\{ \begin{array}{l l} e ^ {- 2 s} \sum_ {p \geq \left(\frac {n - \omega}{2}\right) +} \frac {(s / 2) ^ {2 p + \omega}}{p ! (p + \omega) !} \binom {2 p + \omega} {p - \frac {n - \omega}{2}} & i f n \equiv \omega , \\ 0 & o t h e r w i s e. \end{array} \right. \tag {45}
$$
and  $q_{\omega, -n} = q_{\omega, n}$  by evenness of the coefficients.
Proof. From the definition of the modified Bessel function of the first kind [Watson, 1922, p.77, Eq. 2], we have
$$
I _ {\omega} (z) = \sum_ {p \geq 0} \frac {(z / 2) ^ {2 p + \omega}}{p ! (p + \omega) !},
$$
$$
\begin{array}{l} I _ {\omega} (2 s \cos 2 \pi z) = \sum_ {p \geq 0} \frac {s ^ {2 p + \omega}}{p ! (p + \omega) !} \cos (2 \pi z) ^ {2 p + \omega} \\ = \sum_ {p \geq 0} \frac {(s / 2) ^ {2 p + \omega}}{p ! (p + \omega) !} \left(e ^ {2 \pi \mathrm {i} z} + e ^ {- 2 \pi \mathrm {i} z}\right) ^ {2 p + \omega} \\ = \sum_ {p \geq 0} \frac {(s / 2) ^ {2 p + \omega}}{p ! (p + \omega) !} \sum_ {k = 0} ^ {2 p + \omega} \binom {2 p + \omega} {k} e ^ {2 \pi \mathrm {i} (2 (p - k) + \omega) z}. \tag {46} \\ \end{array}
$$
Using the change of variable  $n = 2(p - k) + \omega$  into eq. (46), we see that  $n$  has the same parity as  $\omega$  and
$$
I _ {\omega} (2 s \cos 2 \pi z) = \sum_ {p \geq 0} \frac {(s / 2) ^ {2 p + \omega}}{p ! (p + \omega) !} \sum_ {n = - \left(\frac {2 p - \omega}{n \equiv \omega}\right)} ^ {2 p + \omega} \binom {2 p + \omega} {p - \frac {n - \omega}{2}} e ^ {2 \pi \mathrm {i} n z}. \tag {47}
$$
Equation (47) can be rewritten
$$
I_{\omega}(2s\cos 2\pi z) = \sum_{\substack{n\in \mathbb{Z}\\ n\equiv \omega}}e^{2\pi \mathrm{i}nz}\sum_{p\geq 0}\frac{(s / 2)^{2p + \omega}}{p!(p + \omega)!}\binom {2p + \omega}{p - \frac{n - \omega}{2}}\mathbf{1}_{-(2p + \omega)\leq n\leq 2p + \omega},
$$
for which eq. (45) is a concise rewriting.