SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $\mathrm{fft}(\cdot)$  and  $\mathrm{ifft}(\cdot)$  denote the FFT and inverse FFT operation. Then, we can achieve  $x_{q}^{*}$  by the index  $i^{*}$  of the minimum element in vector  $\pmb{c}^{q} = \pmb{K}_{\Delta}^{q^{\top}}\left(\widehat{\pmb{k}}_{\Delta}^{q} \odot \pmb{w}\right)$ , and set  $x_{q}^{*} = \frac{i^{*}z_{q} \bmod n}{n}$ .
We present the algorithm of the fast coordinate search in Algorithm 1. The Algorithm 1 returns a targeted sample with a small prediction value in a fast manner. We can use the targeted sample to accelerate the stochastic optimization. Finally, we present our overall stochastic optimization algorithm in the Algorithm 2. We choose INGO [Lyu and Tsang, 2021] as our backbone algorithm because of its simple implementation and fewer hyperparameters. One can plug our RLTS into other stochastic optimization methods to improve query efficiency.
The first element of  $k_{\Delta}$  is set to  $k_{\Delta}(\mathbf{0}) + \sigma^2$
Algorithm 2 Rank-1 Lattice Targeted Sampling
Input: Number of batch samples  $n$ , step-size  $\beta$  and  $\eta$ , number of internal iterations  $T$  for Fast Coordinate Search, and initial variance  $\sigma^2$ .  
Initialization: Initialize  $\mu_0 = 0$  and  $\Sigma_0 = \sigma^2 I$ .  
while Termination condition not satisfied do  
    Sample a shift vector  $\Delta$  uniformly from  $[0,1]^d$ .  
    Construct shifted rank-1 lattice  $\bar{X} = [\bar{x}_1,\dots ,\bar{x}_n]$  by Eq.(9).  
    Construct QMC Gaussian Samples  $\epsilon_1,\dots ,\epsilon_n$  by Eq.(10).  
    Set  $\xi_{i} = \mu_{t} + \Sigma_{t}^{\frac{1}{2}}\epsilon_{i}$  for  $i\in \{1,\dots n\}$ .  
    Query the batch observations  $\{f(\pmb {\xi}_1),\dots ,f(\pmb {\xi}_n)\}$   
    Compute  $\widehat{\sigma} = \mathrm{std}(f(\pmb {\xi}_1),\dots,f(\pmb {\xi}_n))$ .  
    Compute  $\widehat{\mu} = \frac{1}{n}\sum_{i = 1}^{n}f(\pmb {\xi}_i)$ .  
    Set  $y_{i} = \frac{f(\pmb{\xi}_{i}) - \widehat{\mu}}{\widehat{\sigma}}$  for  $i\in \{1,\dots n\}$ .  
    Perform fast exact GP training with rank-1 lattice  $\bar{X}$  and  $\pmb{y}$  by Eq.(17) and Eq.(18).  
    Get targeted grid sample  $\bar{x}^*$  by Algorithm 1 with  $T$  steps.  
    Get targeted Gaussian sample  $\pmb{\xi}^{*} = \Phi^{-1}(\bar{\pmb{x}}^{*} + \Delta$  mod 1)  
    Query the observation  $f(\pmb{\xi}^{*})$ .  
    Set  $\Sigma_{t + 1}^{-1} = \Sigma_{t}^{-1} + \frac{\beta}{n}\sum_{i = 1}^{n}y_{i}\Sigma_{t}^{-\frac{1}{2}}\pmb{\epsilon}_{i}\pmb{\epsilon}_{i}^{\top}\Sigma_{t}^{-\frac{1}{2}}$ .  
    Set  $\mu_{t + 1} = \mu_t - \frac{\beta}{n}\sum_{i = 1}^{n}y_i\Sigma_t^{\frac{1}{2}}\pmb{\epsilon}_i$   
    if  $f(\pmb{\xi}^{*}) < \min_{i\in \{1,\dots ,n\}}f(\pmb{\xi}_i)$  then  
        Set  $\mu_{t + 1} = (1 - \eta)\mu_{t + 1} + \eta \pmb{\xi}^{*}$   
    end if  
end while
3.4 Closed-form Rank-1 Lattice Construction
This subsection will show how to construct our closed-form rank-1 lattice for fast sampling. For  $\forall \pmb{x},\pmb{y}\in [0,1]^d$  and  $\alpha >1$ , define a reproducing kernel as Eq. (28)
$$
K (\boldsymbol {x}, \boldsymbol {y}) = \sum_ {\boldsymbol {k} \in \mathbb {Z} ^ {d}} \gamma_ {\alpha} (\boldsymbol {k}) \exp \left(2 \pi \mathbf {i} \boldsymbol {k} ^ {\top} (\boldsymbol {x} - \boldsymbol {y})\right), \tag {28}
$$
where  $\mathbf{i}^2 = -1$  and  $\gamma_{\alpha}(\pmb{k}) = \prod_{j=1}^{d} \gamma_{\alpha}(k_j)$  with  $\gamma_{\alpha}(k)$  is given as follows:
$$
\gamma_ {\alpha} (k) = \left\{ \begin{array}{l l} 1 & \text {i f} k = 0 \\ | k | ^ {- \alpha} & \text {i f} k \neq 0. \end{array} \right. \tag {29}
$$
A Korobov space is a reproducing kernel Hilbert space (RKHS) associated with the kernel in Eq.(28), denoted as  $\mathcal{H}_k$ .
Our closed form of the generating vector is given as Eq.(30):
$$
\boldsymbol {z} = \left[ g ^ {0}, g ^ {\frac {n - 1}{2 d - 1}}, g ^ {\frac {2 (n - 1)}{2 d - 1}}, \dots , g ^ {\frac {(d - 1) (n - 1)}{2 d - 1}} \right] \bmod n, \tag {30}
$$
where  $g$  denotes the primitive root modulo the prime number  $n$ , and  $(2d - 1)(n - 1)$ . Then, our close-form rank-1 lattice can be achieved by Eq. (6)
Given a point set  $\mathcal{P} = \{\pmb{x}_1, \dots, \pmb{x}_n\}$ , the square worst case integral approximation error for  $f \in \mathcal{H}_k$  is defined as Eq.(31):
$$
e ^ {2} \left(\mathcal {H} _ {k}; \mathcal {P}\right) = \sup  _ {f \in \mathcal {H} _ {k}, \| f \| _ {\mathcal {H} _ {k}} \leq 1} \left| \int_ {[ 0, 1 ] ^ {d}} f (\boldsymbol {x}) d \boldsymbol {x} - \frac {1}{n} \sum_ {j = 0} ^ {n - 1} f \left(\boldsymbol {x} _ {j}\right) \right| ^ {2}. \tag {31}
$$
We further show that our rank-1 lattice constructed by Eq. (30) has a regular worst-case error pattern in Theorem 1. The proof is given in the Appendix.
Theorem 1. Let  $n$  be a prime number such that  $(2d - 1)|(n - 1)$ . Suppose the integrand function  $f \in \mathcal{H}_k, \| f\|_{\mathcal{H}_k} \leq 1$ , the square worst-case integral approximation error of rank-1 lattice  $\mathcal{P}$  constructed by Eq.(30) is given as Eq.(32):
$$
e ^ {2} \left(\mathcal {H} _ {k}; \mathcal {P}\right) = \frac {1}{2 n} \mathbf {1} ^ {\top} \left(\boldsymbol {h} ^ {0} \odot \dots \odot \boldsymbol {h} ^ {2 d - 2} - \mathbf {1} - \left(\boldsymbol {h} ^ {1} \odot \dots \odot \boldsymbol {h} ^ {d - 1} - \mathbf {1}\right) \odot \boldsymbol {h} ^ {0} \odot \left(\boldsymbol {h} ^ {- 1} \odot \dots \odot \boldsymbol {h} ^ {- (d - 1)} - \mathbf {1}\right)\right) + \frac {1}{n ^ {\alpha}} \zeta (\alpha , 1), \tag {32}
$$
(a) Rosenbrock-dim-50
(b) Rastrigin10-dim-50
(c) Nesterov-dim-50
(d) Rosenbrock-dim-500
(e) Rastrigin10-dim-500
(f) Nesterov-dim-500
Figure 2: Cumulative min objective value v.s. the number of query evaluations on 50-dimensional and 500-dimensional benchmark test functions.
where  $\odot$  denotes the element-wise product, symbol  $\mathbf{1}$  denotes the vector with elements all ones, and  $\pmb{h}^{i} = \pmb{F}^{i}\pmb{\gamma}$  with  $\pmb{F}$  as the discrete Fourier matrix, i.e.,  $\pmb{F}_{jk} = \exp (2\pi \mathbf{i}\frac{jk}{n})$ , and  $\pmb{F}^{i}$  denotes the matrix after permutation of the rows of  $\pmb{F}$  such that the  $j^{th}$  row of  $\pmb{F}^{i}$  equals to the  $\widetilde{j}^{th}$  row of  $\pmb{F}$ , where  $\widetilde{j} = jg^{\frac{i(n - 1)}{2d - 1}} \bmod n$ . And  $\pmb{\gamma} = [\gamma_{1},\dots ,\gamma_{n}]^{\top}$  with  $\gamma_{k} = \frac{1}{n^{\alpha}}\left(\zeta (\alpha ,\frac{k_{i}}{n}) + \zeta (\alpha ,\frac{n - k_{i}}{n})\right)$  for  $k\in \{1,\dots ,n - 1\}$  and  $\gamma_{n} = 1 + \frac{2}{n^{\alpha}}\zeta (\alpha ,1)$ , where  $\zeta (\cdot ,\cdot)$  denotes the Hurwitz zeta function.
Remarks: The term  $H = \pmb{h}^{0} \odot \dots \odot \pmb{h}^{2d - 2} - \mathbf{1} - (\pmb{h}^{1} \odot \dots \odot \pmb{h}^{d - 1} - \mathbf{1}) \odot \pmb{h}^{0} \odot (\pmb{h}^{-1} \odot \dots \odot \pmb{h}^{-(d - 1)} - \mathbf{1})$  has a regular pattern because of  $\{g^0, g^{\frac{n - 1}{2d - 1}}, g^{\frac{2(n - 1)}{2d - 1}}, \dots, g^{\frac{(d - 1)(n - 1)}{2d - 1}}, \dots, g^{\frac{(2d - 2)(n - 1)}{2d - 1}}\}$  mod  $n$  forms a subgroup of  $\{1, \dots, n - 1\}$  mod  $n$ . According to the Lagrange's theorem in group theory [Dummit and Foote, 2004], the vector  $\pmb{h}^{0} \odot \dots \odot \pmb{h}^{2d - 2}$  has  $\frac{n - 1}{2d - 1}$  different elements.
We replace the i.i.d. Gaussian sampling of the INGO [Lyu and Tsang, 2021] with our RLTS. We evaluate our RLTS by comparing it with the standard INGO and the CMAES [Hansen, 2006]. In all the experiments, we keep the number of batch samples and the initialization the same for RLTS, INGO and CMAES. For all the methods, we initialize the  $\mu = 0$ . For INGO and RLTS, we set the step-size parameter  $\beta = 0.2$  in all experiments. For RLTS, we set the parameter  $\eta = 1$  in all experiments.
4.1 Evaluation on Benchmark Functions
We first evaluate our RLTS on challenging benchmark test functions: Rosenbrock, Rastrigin, and Nesterov. Rastrigin and Rosenbrock are smooth multi-mode functions, and Nesterov is a non-smooth function. These functions are very challenging benchmarks for black-box optimization. We offset the optimum by setting  $x = x - 5$  of the test functions. This increases the distance between the optimum and the initial point  $\mu = 0$ , which makes the test problems more challenging. We implement INGO by ourselves. For CMAES, we use the publicly available code<sup>2</sup>. We initialized  $\Sigma = I$  for all the methods.
$^{2}$ https://pypi.org/project/cma/
(a) DBpedia
(b) ss2
(c) SNLI
(d) AG's News
(e) MRPC
(f) RTE
Figure 3: Hinge loss v.s. the number of query evaluations on different black-box fine-tuning models.
We evaluate RLTS on 50 and 500-dimensional problems. The batchsize of all the methods are set to 200 and 2000 for 50 and 500-dimensional problems, respectively. All the experiments are performed in ten independent runs. The experimental results are shown in Figure 2. From Figure 2, we can observe that RLTS consistently converge faster than INGO on all the test functions on both 50-dimensional and 500-dimensional cases. It shows that our RLTS significantly improves the query efficiency of INGO, which verifies the effectiveness of RLTS. Moreover, we can see that RLTS outperforms CMAES on all the test functions on both 50-dimensional and 500-dimensional cases. In addition, we see that CMAES converge slowly on the 500-dimensional benchmark problems, while RLTS converges faster.
4.2 Evaluation on Black-box Prompt Fine-tuning Tasks
Prompt fine-tuning of large language models is a promising direction to achieve expertise models efficiently for downstream tasks. We evaluate our RLTS on black-box prompt fine-tuning tasks.
We employ the deep model in [Sun et al., 2022a] with publicly available code  $^{3}$  as the backbone model for black-box prompt fine-tuning. It has 24 layers. For each layer, we set the dimension of the continuous prompt to 500. Thus, the total dimension is  $24 \times 500$ . We employ the hinge loss of training data as the black-box objective. Six benchmark datasets for different language tasks are employed for evaluation: DBpedia, SS2, SNLI, AG's News, MRPC and RTE. The SST2 [Socher et al., 2013] dataset is a dataset for the sentiment analysis task. AG's News and DBPedia datasets [Zhang et al., 2015] are used for topic classification tasks. SNLI [Bowman et al., 2015] and RTE [Wang et al., 2019] are employed for natural language inference. MRPC dataset [Dolan and Brockett, 2005] is used for the paraphrasing task.
In all the experiments, we keep the number of batch samples and the initialization the same for RLTS, INGO and CMAES. We set the number of batch samples to 2000. Specifically, our RLTS employs 1999 rank-1 lattice QMC Gaussian samples and one sample from targeted sampling. INGO employs 1999 rank-1 lattice QMC Gaussian samples and one Gaussian sample. CMAES employs 2000 Gaussian samples. We initialize the  $\mu = 0$  and  $\Sigma = 0.2I$  for all the methods. For INGO and RLTS, we set the step-size parameter  $\beta = 0.2$  in all experiments. For RLTS, we set the parameter  $\eta = 1$  in all experiments. All the experiments are performed in five independent runs with seeds in  $\{1,2,3,4,5\}$ . The layer-wise coordinate descent update approach in [Sun et al., 2022a] is employed for all the methods.
<sup>3</sup>https://github.com/txsun1997/Black-Box-Tuning
(a) Rosenbrock-dim-500
(b) Rastrigin10-dim-500
Figure 4: Cumulative min objective value v.s. the number of query evaluations on 500-dimensional benchmark test functions.
(c) Nesterov-dim-500
The experimental results of mean objective  $\pm$  std v.s. the number of queries are shown in Figure 3. From Figure 3, we can observe that our RLTS decreases the objective significantly faster than INGO and CMAES on all six fine-tuning tasks, which shows the superior query efficiency of our RLTS.
4.3 Additional Comparison with High-dimensional Bayesian Optimization
We further compare our RLTS with the high-dimensional BO method TuRBO [Eriksson et al., 2019]. We evaluate RLTS on the three benchmark functions: the Rosenbrock function, the Rastrigin10 function, and the Nesterov function. We offset the optimum by setting  $x = x - 5$  of the test functions. The dimension is set to 500. The number of initial points of TuRBO is set to 2000. The batch size of both RLTS and TuRBO is set to 2000. The maximum number of queries is set to 50,000. We employ the default box boundary for TuRBO, i.e.,  $[-5,10]^d$ . The initial parameter  $\mu$  of RLTS is set to  $\mu = 0$ , and  $\Sigma$  is set to  $\Sigma = I$ . For TuRBO, we employ the official code provided in the paper [Eriksson et al., 2019]. All the methods are performed in three independent runs.
The convergence performance regarding the number of query evaluations is shown in Figure 4. We can observe that RLTS converges faster than TuRBO on the benchmark test problems, demonstrating that RLTS improves query efficiency.
We further report the running time of RLTS and TuRBO on the same machine for evaluation. The results are shown in Table 1. We can observe that RLTS performs significantly faster than TuRBO, achieving around 300 times speedup regarding running time. The computation time of Bayesian Optimization usually grows cubically fast as the number of queries increases. In contrast, our RLTS reduces the expensive  $O(n^{3})$  operation to  $O(n\log n)$  time complexity, which enables a fast plug-in of the ES-type algorithms.
Table 1: Running time on benchmark test functions. Symbol (s) denotes seconds.
Rosenbrock	Rastrigin10	Nesterov
RLTS	83.62(s)	84.04(s)	83.46(s)
TuRBO	25927.39(s)	25941.66(s)	25697.87(s)
We proposed a novel Rank-1 Lattice Targeted Sampling technique in this paper. Our RLTS has a  $O(n \log n)$  time complexity w.r.t.  $n$  batch samples, which is fast for plugging into stochastic optimization methods to improve query efficiency while scaling up to high-dimensional problems. Empirically, we plugged our RLTS into the sampling phase of INGO, significantly improving the query efficiency on benchmark test functions and black-box prompt fine-tuning tasks. Moreover, we proposed a closed-form rank-1 lattice by analyzing the integral approximation error of functions in Korobov space. Our closed-form rank-1 lattice provides an efficient way for QMC Gaussian sampling, with properties enabling fast exact GP training and inference with a  $O(n \log n)$  time complexity, which is critical for our RLTS to be a fast internal step for stochastic optimization. In addition, our closed-form rank-1 lattice is a fundamental tool that may have potential applications beyond the black-box optimization task.
We thank the anonymous reviewers for their valuable comments and helpful suggestions.
Thomas Back, Frank Hoffmeister, and Hans-Paul Schwefel. A survey of evolution strategies. In Proceedings of the fourth international conference on genetic algorithms, volume 2. Morgan Kaufmann Publishers San Mateo, CA, 1991.
Juan Cruz Barsce, Jorge A Palombarini, and Ernesto C Martínez. Towards autonomous reinforcement learning: Automatic setting of hyper-parameters using bayesian optimization. In Computer Conference (CLEI), 2017 XLIII Latin American, pages 1-9. IEEE, 2017.
Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. A large annotated corpus for learning natural language inference. Proceedings of the 2015 conference on empirical methods in natural language processing, 2015.
Krzysztof Choromanski, Aldo Pacchiano, Jack Parker-Holder, and Yunhao Tang. From complexity to simplicity: Adaptive es-active subspaces for blackbox optimization. arXiv:1903.04268, 2019.
Josef Dick, Frances Y Kuo, and Ian H Sloan. High-dimensional integration: the quasi-monte carlo way. Acta Numerica, 22:133-288, 2013.
Josef Dick, Peter Kritzer, and Friedrich Pillichshammer. Lattice Rules. Springer, 2022.
Bill Dolan and Chris Brockett. Automatically constructing a corpus of sentential paraphrases. In Third International Workshop on Paraphrasing (IWP2005), 2005.
David Steven Dummit and Richard M Foote. Abstract algebra, volume 3. Wiley Hoboken, 2004.
David Eriksson, Michael Pearce, Jacob Gardner, Ryan D Turner, and Matthias Poloczek. Scalable global optimization via local bayesian optimization. Advances in neural information processing systems, 32, 2019.
Jacob Gardner, Chuan Guo, Kilian Weinberger, Roman Garnett, and Roger Grosse. Discovering and exploiting additive structure for bayesian optimization. In Artificial Intelligence and Statistics, pages 1311-1319. PMLR, 2017.
Robert M Gray et al. Toeplitz and circulant matrices: A review. Foundations and Trends® in Communications and Information Theory, 2(3):155-239, 2006.
Nikolaus Hansen. The cma evolution strategy: a comparing review. In Towards a new evolutionary computation, pages 75-102. Springer, 2006.
Jose Miguel Hernandez-Lobato, James Requeima, Edward O Pyzer-Knapp, and Alan Aspuru-Guzik. Parallel and distributed thompson sampling for large-scale accelerated exploration of chemical space. In International conference on machine learning, pages 1470-1479. PMLR, 2017.
Nikolai Mikhailovich Korobov. Properties and calculation of optimal coefficients. In Doklady Akademii Nauk, volume 132, pages 1009-1012. Russian Academy of Sciences, 1960.
Daniel J Lizotte, Tao Wang, Michael H Bowling, and Dale Schuurmans. Automatic gait optimization with gaussian process regression. In *IJCAI*, volume 7, pages 944-949, 2007.
Yueming Lyu and Ivor W Tsang. Black-box optimizer with stochastic implicit natural gradient. In Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13-17, 2021, Proceedings, Part III 21, pages 217-232. Springer, 2021.
Yueming Lyu, Yuan Yuan, and Ivor Tsang. Subgroup-based rank-1 lattice quasi-monte carlo. Advances in Neural Information Processing Systems, 33:6269-6280, 2020.
Daan Wierstra, Tom Schaul, Tobias Glasmachers, Yi Sun, Jan Peters, and Jürgen Schmidhuber. Natural evolution strategies. The Journal of Machine Learning Research (JMLR), 15(1):949-980, 2014a.
Daan Wierstra, Tom Schaul, Tobias Glasmachers, Yi Sun, Jan Peters, and Jürgen Schmidhuber. Natural evolution strategies. The Journal of Machine Learning Research, 15(1):949-980, 2014b.
Christopher KI Williams and Carl Edward Rasmussen. Gaussian processes for machine learning, volume 2. MIT press Cambridge, MA, 2006.
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28, 2015.
Theorem 1. Let  $n$  be a prime number such that  $(2d - 1)(n - 1)$ . Suppose the integrand function  $f \in \mathcal{H}_k, \|f\|_{\mathcal{H}_k} \leq 1$ , the square worst-case integral approximation error of rank-1 lattice  $\mathcal{P}$  constructed by Eq.(30) is given as Eq.(33):
$$
e ^ {2} \left(\mathcal {H} _ {k}; \mathcal {P}\right) = \frac {1}{2 n} \mathbf {1} ^ {\top} \left(\boldsymbol {h} ^ {0} \odot \dots \odot \boldsymbol {h} ^ {2 d - 2} - \mathbf {1} - \left(\boldsymbol {h} ^ {1} \odot \dots \odot \boldsymbol {h} ^ {d - 1} - \mathbf {1}\right) \odot \boldsymbol {h} ^ {0} \odot \left(\boldsymbol {h} ^ {- 1} \odot \dots \odot \boldsymbol {h} ^ {- (d - 1)} - \mathbf {1}\right)\right) + \frac {1}{n ^ {\alpha}} \zeta (\alpha , 1), \tag {33}
$$
where  $\odot$  denotes the element-wise product, symbol  $\mathbf{1}$  denotes the vector with elements all ones, and  $h^i = F^i\gamma$  with  $\pmb{F}$  as the discrete Fourier matrix, i.e.,  $\pmb{F}_{jk} = \exp (2\pi \mathbf{i}\frac{jk}{n})$ , and  $\pmb{F}^{i}$  denotes the matrix after permutation of the rows of  $\pmb{F}$  such that the  $j^{th}$  row of  $\pmb{F}^{i}$  equals to the  $\widetilde{j}^{th}$  row of  $\pmb{F}$ , where  $\widetilde{j} = jg^{\frac{i(n - 1)}{2d - 1}} \bmod n$ . And  $\gamma = [\gamma_{1},\dots ,\gamma_{n}]^{\top}$  with  $\gamma_{k} = \frac{1}{n^{\alpha}}\bigl (\zeta (\alpha ,\frac{k_{i}}{n}) + \zeta (\alpha ,\frac{n - k_{i}}{n})\bigr)$  for  $k\in \{1,\dots ,n - 1\}$  and  $\gamma_{n} = 1 + \frac{2}{n^{\alpha}}\zeta (\alpha ,1)$ , where  $\zeta (\cdot ,\cdot)$  denotes the Hurwitz zeta function.
To prove our main Theorem 1, we begin with several Lemma.
Lemma 1. For  $\forall x, y \in [0,1]^d$  and  $\alpha > 1$ , define a reproducing kernel as Eq.(34)
$$
K (\boldsymbol {x}, \boldsymbol {y}) = \sum_ {\boldsymbol {k} \in \mathbb {Z} ^ {d}} \gamma_ {\alpha} (\boldsymbol {k}) \exp \left(2 \pi \mathbf {i} \boldsymbol {k} ^ {\top} (\boldsymbol {x} - \boldsymbol {y})\right), \tag {34}
$$
where  $\gamma_{\alpha}(\pmb {k}) = \prod_{j = 1}^{d}\gamma_{\alpha}(k_{j})$  with  $\gamma_{\alpha}(k)$  given in Eq.(35)
$$
\gamma_ {\alpha} (k) = \left\{ \begin{array}{l l} 1 & \text {i f} k = 0 \\ | k | ^ {- \alpha} & \text {i f} k \neq 0. \end{array} \right. \tag {35}
$$
Let  $\mathcal{P} = [\pmb{x}_1, \dots, \pmb{x}_n]$  be a rank-1 lattice constructed by the generating vector  $\pmb{z}$  with a prime number  $n$ . Then, for  $\forall f \in \mathcal{H}_k$ ,  $\|f\|_{\mathcal{H}_k} \leq 1$  associated with the reproducing kernel Eq.(34), we have the square worst-case integral approximation error of  $\mathcal{P}$  as Eq.(36).
$$
e ^ {2} \left(\mathcal {H} _ {k}; \mathcal {P}\right) = \sup  _ {f \in \mathcal {H} _ {k}, \| f \| _ {\mathcal {H} _ {k}} \leq 1} \left| \int_ {[ 0, 1 ] ^ {d}} f (\boldsymbol {x}) d \boldsymbol {x} - \frac {1}{n} \sum_ {j = 0} ^ {n - 1} f \left(\boldsymbol {x} _ {j}\right) \right| ^ {2} = \sum_ {\boldsymbol {k} \in L ^ {\perp} \backslash \{\boldsymbol {0} \}} \gamma_ {\alpha} (\boldsymbol {k}) \tag {36}
$$
where  $L^{\perp}$  denote the dual lattice defined in Eq.(37).
$$
L ^ {\perp} := \left\{\boldsymbol {k} \mid \boldsymbol {k} ^ {\top} \boldsymbol {z} \equiv 0 (\text {m o d} n), \boldsymbol {k} \in \mathbb {Z} ^ {d} \right\}. \tag {37}
$$
Proof. Given a point set  $\mathcal{P} = \{\pmb{x}_1, \dots, \pmb{x}_n\}$ , the worst case approximation error for  $\forall f \in \mathcal{H}_k, \|f\|_{\mathcal{H}_k} \leq 1$  is
$$
\begin{array}{l} e ^ {2} \left(\mathcal {H} _ {k}; \mathcal {P}\right) = \sup  _ {f \in \mathcal {H} _ {k}, \| f \| _ {\mathcal {H} _ {k}} \leq 1} \left| \int_ {[ 0, 1 ] ^ {d}} f (\boldsymbol {x}) \mathrm {d} \boldsymbol {x} - \frac {1}{n} \sum_ {j = 0} ^ {n - 1} f \left(\boldsymbol {x} _ {j}\right) \right| ^ {2} (38) \\ = \sup  _ {f \in \mathcal {H} _ {k}, \| f \| _ {\mathcal {H} _ {k}} \leq 1} \left| \left\langle f, \int_ {[ 0, 1 ] ^ {d}} K (\boldsymbol {x}, \cdot) d \boldsymbol {x} - \frac {1}{n} \sum_ {j = 0} ^ {n - 1} K \left(\boldsymbol {x} _ {j}, \cdot\right) \right\rangle_ {\mathcal {H} _ {k}} \right| ^ {2} (39) \\ = \sup  _ {f \in \mathcal {H} _ {k}, \| f \| _ {\mathcal {H} _ {k}} \leq 1} \| f \| _ {\mathcal {H} _ {k}} \| \int_ {[ 0, 1 ] ^ {d}} K (\boldsymbol {x}, \cdot) d \boldsymbol {x} - \frac {1}{n} \sum_ {j = 0} ^ {n - 1} K \left(\boldsymbol {x} _ {j}, \cdot\right) \| _ {\mathcal {H} _ {k}} (40) \\ = \int_ {[ 0, 1 ] ^ {d}} \int_ {[ 0, 1 ] ^ {d}} K (\boldsymbol {x}, \boldsymbol {y}) d \boldsymbol {x} d \boldsymbol {y} - \frac {2}{n} \sum_ {j = 1} ^ {n} \int_ {[ 0, 1 ] ^ {d}} K (\boldsymbol {x}, \boldsymbol {x} _ {j}) d \boldsymbol {x} + \frac {1}{n ^ {2}} \sum_ {i, j = 1} ^ {n} K (\boldsymbol {x} _ {i}, \boldsymbol {x} _ {j}) (41) \\ \end{array}
$$
Then, from the definition of the reproducing kernel  $K(\pmb{x}, \pmb{y})$  in Eq.(34), we know that
$$
\begin{array}{l} \int_ {[ 0, 1 ] ^ {d}} \int_ {[ 0, 1 ] ^ {d}} K (\boldsymbol {x}, \boldsymbol {y}) \mathrm {d} \boldsymbol {x} \mathrm {d} \boldsymbol {y} = \int_ {[ 0, 1 ] ^ {d}} \int_ {[ 0, 1 ] ^ {d}} \sum_ {\boldsymbol {k} \in \mathbb {Z} ^ {d}} \gamma_ {\alpha} (\boldsymbol {k}) \exp \left(2 \pi \mathbf {i} \boldsymbol {k} ^ {\top} (\boldsymbol {x} - \boldsymbol {y})\right) \mathrm {d} \boldsymbol {x} \mathrm {d} \boldsymbol {y} (42) \\ = 1 + \sum_ {\boldsymbol {k} \in \mathbb {Z} ^ {d}, \boldsymbol {k} \neq \boldsymbol {0}} \gamma_ {\alpha} (\boldsymbol {k}) \int_ {[ 0, 1 ] ^ {d}} \int_ {[ 0, 1 ] ^ {d}} \exp \left(2 \pi \mathbf {i} \boldsymbol {k} ^ {\top} (\boldsymbol {x} - \boldsymbol {y})\right) d \boldsymbol {x} d \boldsymbol {y} (43) \\ = 1 + \sum_ {\boldsymbol {k} \in \mathbb {Z} ^ {d}, \boldsymbol {k} \neq \boldsymbol {0}} \gamma_ {\alpha} (\boldsymbol {k}) \cdot 0 = 1 (44) \\ \end{array}
$$
In addition, the second term in Eq.(41) as follows