Title: Fast Rank-1 Lattice Targeted Sampling for Black-box Optimization
Venue: NeurIPS 2023
Accepted: yes
Keywords: Black-box Optimization

Derivate-free Optimization

Kernel methods
Abstract: Black-box optimization has gained great attention for its success in recent applications.    However, scaling up to high-dimensional problems with good query efficiency remains challenging. This paper proposes a novel Rank-1 Lattice Targeted Sampling (RLTS) technique to address this issue. Our RLTS benefits from random rank-1 lattice Quasi-Monte Carlo, which enables us to perform fast local exact Gaussian processes (GP)  training and inference with $O(n \log n)$ complexity w.r.t. $n$ batch samples. Furthermore, we developed a fast coordinate searching method with $O(n \log n)$ time complexity for fast targeted sampling. The fast computation enables us to plug our RLTS into the sampling phase of stochastic optimization methods. This improves the query efficiency while  scaling up to higher dimensional problems than Bayesian optimization. Moreover,  to construct rank-1 lattices efficiently, we proposed a closed-form construction.  Extensive experiments on challenging benchmark test functions and black-box prompt fine-tuning for large language models demonstrate the query efficiency of our RLTS technique.

Fast Rank-1 Lattice Targeted Sampling for Black-box Optimization

Abstract

Black-box optimization has gained great attention for its success in recent applications. However, scaling up to high-dimensional problems with good query efficiency remains challenging. This paper proposes a novel Rank-1 Lattice Targeted Sampling (RLTS) technique to address this issue. Our RLTS benefits from random rank-1 lattice Quasi-Monte Carlo, which enables us to perform fast local exact Gaussian processes (GP) training and inference with O ( n log n ) complexity w.r.t. n batch samples. Furthermore, we developed a fast coordinate searching method with O ( n log n ) time complexity for fast targeted sampling. The fast computation enables us to plug our RLTS into the sampling phase of stochastic optimization methods. This improves the query efficiency while scaling up to higher dimensional problems than Bayesian optimization. Moreover, to construct rank-1 lattices efficiently, we proposed a closed-form construction. Extensive experiments on challenging benchmark test functions and black-box prompt fine-tuning for large language models demonstrate the query efficiency of our RLTS technique.

1 Introduction

Black-box optimization has gained great attention for its success in many recent applications, such as prompt fine-tuning for large language models [Sun et al., 2022b,a], policy search for robot control and reinforcement learning [Choromanski et al., 2019, Lizotte et al., 2007, Barsce et al., 2017, Salimans et al., 2017], automatic hyper-parameters tuning in machine learning problems [Snoek et al., 2012], black-box architecture search in engineering design [Wang and Shan, 2007], drug discovery [Negoescu et al., 2011] and accelerated simulation for scientific discovery [Maddox et al., 2021, Hernández-Lobato et al., 2017], etc. Many efforts have been made for black-box optimization in the literature, including Bayesian optimization (BO) methods [Srinivas et al., 2010, Gardner et al., 2017, Nayebi et al., 2019], stochastic optimization methods like evolution strategies (ES) [Back et al., 1991, Hansen, 2006, Wierstra et al., 2014b, Lyu and Tsang, 2021] and genetic algorithms [Srinivas and Patnaik, 1994, Mirjalili and Mirjalili, 2019].

Bayesian optimization usually builds a global (GP) model as a surrogate and provides queries by optimizing some acquisition functions [Snoek et al., 2012]. Although BO achieves good query efficiency for low-dimensional problems, it often fails to handle high-dimensional problems with large sample budgets [Eriksson et al., 2019]. The computation of GP with a large number of samples itself is expensive, and the internal optimization of the acquisition functions is challenging. Recently, Müller et al. [2021], Nguyen et al. [2022] builds a GP model for both the function value and the gradient and performs local Bayesian optimization. Although these methods improve the scalability of global BO, they usually cannot scale up to five hundred dimensional complex problems. This may

be because the learned gradient heavily depends on the accuracy of the GP model. However, achieving an accurate GP model is challenging for high-dimensional problems. A slightly misspecified GP model may lead to a wrong estimated gradient due to the highly nonlinear acquisition functions.

On the other line, stochastic optimization methods, e.g., ES [Rechenberg and Eigen, 1973, Nesterov and Spokoiny, 2017], natural evolution strategies (NES) [Wierstra et al., 2014b], CMAES [Hansen, 2006], and implicit natural gradient optimizer (INGO) [Lyu and Tsang, 2021], typically sampling form Gaussian distribution and approximate the (natural) gradient for the update of the Gaussian distribution parameters for continuous optimization. These methods can scale up to higher dimensional problems compared with BO. However, the gradient approximation may have a large variance, especially for high-dimensional problems. Thus, the update direction may not be toward the descent direction, leading to inferior query efficiency.

To address high-dimensional black-box problems with good query efficiency, we propose a novel Rank-1 Lattice Targeted Sampling (RLTS) technique. Our RLTS has a O ( n log n ) time complexity, which is fast for plugging into the sampling phase of stochastic optimization methods. In this way, our methods can improve the query efficiency of stochastic optimization methods while addressing higher-dimensional problems than BO. Our contributions are summarized as follows:

We propose a novel Rank-1 Lattice Targeted Sampling (RLTS) technique. Our RLTS builds a local GP with a random rank-1 lattice, which enables fast exact GP training and inference with O ( n log n ) time complexity w.r.t. n batch samples. Furthermore, we develop a fast coordinate search that enables target sampling with O ( n log n ) time complexity.

We propose a closed-form subgroup rank-1 lattice by considering the dual lattice regarding the integral approximation error of functions in Korobov space. Our rank-1 lattice has a more regular pattern of approximation error terms. Moreover, our subgroup rank-1 lattice capitalizes on constructing a circulant kernel Gram matrix benefit from its group property. This enables efficient O ( n log n ) computations in GP training/inference and fast candidate searching. In contrast, low-discrepancy QMC sequences, such as Sobol sequences or Halton sequences, lack these capabilities. In addition, our new closed-form rank-1 lattice may have potential applications in downstream tasks beyond black-box optimization.

We plug our RLTS into the sampling phase at each step of stochastic optimization methods to improve query efficiency. In this way, during the optimization procedure, our RLTS sampling from an updated promising region instead of a fixed one at each step. This approach can scale up to address high-dimensional problems.

Empirically, extensive experiments on high-dimensional challenging benchmark test functions and practical black-box prompt fine-tuning for large language models demonstrate the effectiveness of our RLTS technique.

2 Background

2.1 Black-box Optimization

Given a proper function f ( x ) : R d → R such that f ( x ) > -∞ , black-box optimization is to minimize f ( x ) by using function queries only. Black-box stochastic optimization methods typically employ a sampling distribution p ( x ; θ ) and optimizes the parameter of the distribution regarding the relaxed problem: J ( θ ) := E p ( x ; θ ) [ f ( x )] .

Evolution Strategies (ES) [Rechenberg and Eigen, 1973, Nesterov and Spokoiny, 2017] employ a Gaussian distribution N ( µ , σ 2 I ) for sampling. The approximate gradient descent update is given as

where ϵ i ∼ N ( 0 , I ) and β denotes the step-size. The ES method performs the approximate first-order gradient descent update. As a result, the convergence of ES may be slow. Several second-order gradient descent methods have been proposed to improve convergence. Wierstra et al. [2014a] proposed the natural evolution strategies (NES), which perform the approximate natural gradient update. When a Gaussian distribution N ( µ , Σ ) is employed for sampling. The update rule of NES is

given in Eq.(2) and Eq.(3):

where ϵ i ∼ N ( 0 , I ) and Σ 1 2 = Σ 1 2 ⊤ and Σ 1 2 Σ 1 2 = Σ . The NES takes advantage of second-order gradient information, which improves the convergence of ES.

Lyu and Tsang [2021] proposed an implicit natural gradient optimizer (INGO) for black-box optimization, which provides an alternative way to compute the natural gradient update. The update rule of INGO is given as in Eq.(4) and Eq.(5):

where x i ∼ N ( µ t , Σ t ) , ̂ µ = ∑ n i =1 f ( x i ) n and ̂ σ denotes the standard deviation of f ( x i ) . The normalization f ( x i ) -̂ µ σ is employed to reduce the variance.

̂

CMAES [Hansen, 2006] provides a more sophisticated update rule and performs well on a wide range of black-box optimization problems. All the above stochastic optimization methods rely on sampling. Thus, the sampling phase is vitally important. And a better sampling technique is promising to achieve further improvement.

2.2 Rank-1 Lattice

A rank-1 lattice is a particular case of the general lattice with a simple operation for point-set construction. It can be used as Quasi-Monte Carlo for integral approximation [Sloan, 2000, Dick et al., 2013]. A rank-1 lattice point set P = { x 1 , · · · , x n } can be constructed as Eq.(6):

where z ∈ Z d is the so-called generating vector, and mod denotes the modulo operation.

Korobov [1960] proposes a rank-1 lattice with the generating vector having a particular form as Eq.(7)

where k is searching over { 1 , · · · , n -1 } to reduce approximation error.

Sloan and Reztsov [2002] further proposed a component-by-component searching method for the generating vector without assuming the Korobov form in Eq. (7). Recently, Lyu et al. [2020] proposed a simple closed-form subgroup-based rank-1 lattice by considering the Toroidal distance in the primal lattice space. The generating vector is given as Eq.(8)

where g denotes the primitive root modulo the prime number n . More details of the lattice rules for numerical integration can be found in the book [Dick et al., 2022].

In this paper, we proposed a closed-form subgroup rank-1 lattice by ensuring the approximation error terms of the dual lattice have a more regular pattern. In contrast, Lyu et al. [2020] construct the rank-1 lattice evenly spaced in the primal lattice space.

3 Fast Rank-1 Lattice Targeted Sampling

3.1 Random Rank-1 Lattice Quasi-Monte Carlo Gaussian Sampling

We first show how to construct random rank-1 lattice Quasi-Monte Carlo Gassuain samples. These samples enable us to perform the black-box stochastic optimization listed in section 2.1. More importantly, the nice property of the structure of these samples facilitates a fast targeted sampling.

3

2

1

0

1

2

3

X1

3

2

1

0

1

2

3

X2

0.02

0.04

0.06

0.08

0.10

0.12

0.14

(a)

Our Closed-form Rank-1 Lattice Sampling

3

2

1

0

1

2

3

X1

3

2

1

0

1

2

3

X2

0.02

0.04

0.06

0.08

0.10

0.12

0.14

(b)

3

2

1

0

1

2

3

X1

3

2

1

0

1

2

3

X2

0.03

0.06

0.09

0.12

0.15

0.18

0.21

(c)

Our Closed-form Rank-1 Lattice Sampling

3

2

1

0

1

2

3

X1

3

2

1

0

1

2

3

X2

0.03

0.06

0.09

0.12

0.15

0.18

0.21

(d)

Given a rank-1 lattice point set P = { x 1 , · · · , x n } , we first construct a random shifted rank-1 lattice [Dick et al., 2013] as Eq. (9),

where ∆ ∼ Uniform [0 , 1] d , and the mod 1 operation denotes a modulo operation that takes the non-negative fractional part of the input number element-wise. Then, we can construct random QMC Gaussian samples as Eq. (10)

where Φ -1 ( · ) computes the inverse cumulative density function of the standard Gaussian distribution w.r.t. the input element-wise. Then, the samples for Gaussian N ( µ , Σ ) can be constructed as follows:

An illustration of the random QMC Gaussian samples constructed by our closed-form rank-1 lattice is shown in Figure 1. We can see that our rank-1 lattice QMC Gassuan samples are spaced more evenly w.r.t. the density.

3.2 Fast Exact GP Training and Inference with Rank-1 Lattice

This subsection will show how to perform fast exact GP training and inference using our rank-1 lattice samples with a O ( n log n ) time complexity w.r.t n samples.

Let K θ denotes the kernel Gram matrix, i.e., K θ = [ k θ ( x i , x j )] 1 ≤ i,j ≤ n , the marginal log-likelihood of a GP model [Williams and Rasmussen, 2006] can be formulated as Eq. (12)

The standard GP model needs a O ( n 3 ) time complexity to compute the marginal log-likelihood, which is prohibitive for fast training as an inner step for stochastic optimization.

In this paper, we construct the random QMC samples based on rank-1 lattice, which enables us to perform fast GP training. Specifically, we build the GP model with the rank-1 lattice as the training data instead of the Gaussian samples. Define modulo kernel as Eq. (13):

where k ∆ ( · ) is a shift-invariant kernel, and the function ϕ ( x i -x j ) is given as Eq. (14)

where operation min( · , · ) outputs the minimum among its two inputs element-wise, and mod 1 output the positive fractional parts of its inputs element-wise. The nonnegative fractional part of a real number x is x -⌊ x ⌋ , where ⌊·⌋ denotes the floor function.

For a GP model with a modulo kernel defined in Eq.(13), the kernel Gram matrix is a circulant matrix thanks to the properties of rank-1 lattice. To be concrete, for rank-1 lattice data, we have Eq.(15)

Then the marginal log-likelihood L ( p ( y | X )) can be computed with a O ( n log n ) time complexity by Fast Fourier Transform (FFT).

Specifically, note that the kernel Gram matrix K θ + σ 2 I is a symmetric circulant matrix generated by vector k ∆ 1 , where k ∆ is a vector with its i th element given as Eq. (16) .

We know that K θ + σ 2 I can be diagonalized as K θ + σ 2 I = 1 n F ∗ Λ F , where the j th row and k th column element of F is F jk = e -2 πjk i /n . And the matrix Λ is the diagonal eigenvalue matrix that can be computed as Λ = diag ( F k ∆ ) . The matrix-vector product F k ∆ can be computed via FFT with O ( n log n ) time complexity. And matrix-vector product 1 n F ∗ v for a vector v can be computed via inverse FFT. More details about the properties of circulant matrices and fast computation via FFT can be found in [Gray et al., 2006].

Then, we achieve the fast computation of the terms in log-likelihood as Eq.(17) and Eq.(18):

where ifft ( · ) , fft ( · ) denotes the inverse FFT and FFT operation, respectively, the operator / in Eq.(17) performs divide element-wise. And the log( · ) is an element-wise operation. And λ i in Eq.(18) denotes the eigenvalue of kernel Gram matrix K θ .

For inference, GP model has closed-form posterior mean and variance [Williams and Rasmussen, 2006] given as Eq.(19) and Eq.(20) :

where k θ ( x ) = [ k θ ( x , x 1 ) , ..., k θ ( x , x n )] ⊤ .

With rank-1 lattice input data, we can perform fast inference by Eq.(21) and Eq.(22):

Both the exact GP training and inference benefit from the structure of rank-1 lattice and FFT acceleration, which can be performed with a O ( n log n ) time complexity. A deep learning toolbox, e.g., Pytorch, can be used to train the parameters of the kernel.

3.3 Fast Coordinate Search for Targeted Sampling

This subsection shows how to perform a fast coordinate search for targeted sampling. A rank-1 lattice with n points is contained in a grid { 0 , 1 n , · · · , n -1 n } d . We thus perform a coordinate descent search from the index set { 0 , 1 , · · · , n -1 } d to minimize the GP posterior mean in Eq.(19).

Let k ( · , · ) = k ∆ ( · ) be a shift-invariant kernel with a decomposition structure as Eq. (23):

where x ∗ q , x q denotes the q th element in x ∗ , x , respectively. We can perform a coordinate search by fixing all the components except the q th one as the current working component for index searching. Formally, let w = ( K θ + σ 2 I ) -1 y . Then, we have the GP posterior mean function given as Eq. (24):

where ⊙ denotes the element-wise product, and k q ∆ ( x ∗ q ) denotes a vector with i th element given as k q ∆ i = k ∆ ( ϕ ( x ∗ q -X qi )) , and X qi denotes the element in q th -row and i th -column of the rank-1 lattice matrix X = [ x 1 , · · · , x n ] . The vector ̂ k q ∆ denotes the remainder vector with its i th -element given as Eq. (25):

To optimize the q th component x ∗ q of x ∗ , we fix the other components of x ∗ and the corresponding vector ̂ k q ∆ . We find x ∗ q by solving the subproblem given in Eq. (26)

Directly enumerate computation of the problem (26) needs a O ( n 2 ) time complexity. In our paper, we can perform a fast computation with O ( n log n ) time complexity thanks to the rank-1 lattice X . Specially, when X is a rank-1 lattice with the generating vector z = [ z 1 , · · · , z d ] , then the matrix K q ∆ = [ k q ∆ (0) , k q ∆ ( 1 z q mod n n ) , · · · , k q ∆ ( ( n -1) z q mod n n )] forms a circulant matrix, and the problem (26) can be accelerated via FFT by Eq. (27)

where fft ( · ) and ifft ( · ) denote the FFT and inverse FFT operation. Then, we can achieve x ∗ q by the index i ∗ of the minimum element in vector c q = K q ⊤ ∆ ( ̂ k q ∆ ⊙ w ) , and set x ∗ q = i ∗ z q mod n n .

We present the algorithm of the fast coordinate search in Algorithm 1. The Algorithm 1 return a targeted sample with a small prediction value in a fast manner. We can use the targeted sample to accelerate the stochastic optimization. Finally, we present our overall stochastic optimization algorithm in the Algorithm 2. We choose INGO [Lyu and Tsang, 2021] as our backbone algorithm because of its simple implementation and fewer hyperparameters. One can plug our RLTS into other stochastic optimization methods to improve query efficiency.

Algorithm 2 Rank-1 Lattice Targeted Sampling

Input: Number of batch samples n , step-size β and η , number of internal iterations T for Fast Coordinate Search, and initial variance σ 2 .

Initialization:

Initialize µ 0 = 0 and Σ 0 = σ 2 I .

while Termination condition not satisfied do

Sample a shift vector ∆ uniformly from [0 , 1] d .

Construct shifted rank-1 lattice ¯ X = [¯ x 1 , · · · , ¯ x n ] by Eq.(9).

Construct QMC Gaussian Samples ϵ 1 , · · · , ϵ n by Eq.(10).

Query the batch observations { f ( ξ 1 ) , ..., f ( ξ n ) }

̂ Perform fast exact GP training with rank-1 lattice ¯ X and y by Eq.(17) and Eq.(18).

Get targeted grid sample ¯ x ∗ by Algorithm 1 with T

Query the observation f ( ξ ∗ ) .

Get targeted Gaussian sample ξ ∗ = Φ -1 (¯ x ∗ +∆ mod 1)

end if end while

3.4 Closed-form Rank-1 Lattice Construction

This subsection will show how to construct our closed-form rank-1 lattice for fast sampling. For ∀ x , y ∈ [0 , 1] d and α > 1 , define a reproducing kernel as Eq. (28)

where i 2 = -1 and γ α ( k ) = ∏ d j =1 γ α ( k j ) with γ α ( k ) is given as follows:

̸

A Korobov space is a reproducing kernel Hilbert space (RKHS) associated with the kernel in Eq.(28), denoted as H k .

Our closed form of the generating vector is given as Eq.(30):

where g denotes the primitive root modulo the prime number n , and (2 d -1) | ( n -1) . Then, our close-form rank-1 lattice can be achieved by Eq. (6)

Given a point set P = { x 1 , · · · , x n } , the square worst case integral approximation error for f ∈ H k is defined as Eq.(31):

We further show that our rank-1 lattice constructed by Eq. (30) has a regular worst-case error pattern in Theorem 1. The proof is given in the Appendix.

Theorem 1. Let n be a prime number such that (2 d -1) | ( n -1) . Suppose the integrand function f ∈ H k , ∥ f ∥ H k ≤ 1 , the square worst-case integral approximation error of rank-1 lattice P constructed by Eq.(30) is given as Eq.(32):

steps.

1e6

0

5000

10000

15000

20000

25000

30000

Number of Evaluations

0

1

2

3

4

Objective Value

RLTS

INGO

CMAES

(a)

Rosenbrock-dim-50

0

5000

10000

15000

20000

25000

30000

Number of Evaluations

0

5000

10000

15000

20000

25000

Objective Value

RLTS

INGO

CMAES

(b)

Rastrigin10-dim-50

0

5000

10000

15000

20000

25000

30000

Number of Evaluations

0

100

200

300

400

500

600

700

Objective Value

RLTS

INGO

CMAES

(c)

Nesterov-dim-50

0

50000

100000

150000

200000

250000

300000

Number of Evaluations

0

1

2

3

4

Objective Value

1e7

RLTS

INGO

CMAES

(d)

0

50000

100000

150000

200000

250000

300000

Number of Evaluations

0

50000

100000

150000

200000

250000

Objective Value

RLTS

INGO

CMAES

(e)

0

50000

100000

150000

200000

250000

300000

Number of Evaluations

0

1000

2000

3000

4000

5000

6000

7000

Objective Value

RLTS

INGO

CMAES

(f)

where ⊙ denotes the element-wise product, symbol 1 denotes the vector with elements all ones, and h i = F i γ with F as the discrete Fourier matrix, i.e., F jk = exp(2 π i jk n ) , and F i denotes the matrix after permutation of the rows of F such that the j th row of F i equals to the ˜ j th row of F , where ˜ j = jg i ( n -1) 2 d -1 mod n . And γ = [ γ 1 , · · · , γ n ] ⊤ with γ k = 1 n α ( ζ ( α, k i n ) + ζ ( α, n -k i n ) ) for k ∈ { 1 , · · · , n -1 } and γ n = 1 + 2 n α ζ ( α, 1) , where ζ ( · , · ) denotes the Hurwitz zeta function.

Remarks: The term H = h 0 ⊙···⊙ h 2 d -2 -1 -( h 1 ⊙···⊙ h d -1 -1 ) ⊙ h 0 ⊙ ( h -1 ⊙···⊙ h -( d -1) -1 ) has a regular pattern because of { g 0 , g n -1 2 d -1 , g 2( n -1) 2 d -1 , · · · , g ( d -1)( n -1) 2 d -1 , · · · , g (2 d -2)( n -1) 2 d -1 } mod n forms a subgroup of { 1 , · · · , n -1 } mod n . According to the Lagrange's theorem in group theory [Dummit and Foote, 2004], the vector h 0 ⊙··· ⊙ h 2 d -2 has n -1 2 d -1 different elements.

4 Experiments

We replace the i.i.d. Gaussian sampling of the INGO [Lyu and Tsang, 2021] with our RLTS. We evaluate our RLTS by comparing it with the standard INGO and the CMAES [Hansen, 2006]. In all the experiments, we keep the number of batch samples and the initialization the same for RLTS, INGO and CMAES. For all the methods, we initialize the µ = 0 . For INGO and RLTS, we set the step-size parameter β = 0 . 2 in all experiments. For RLTS, we set the parameter η = 1 in all experiments.

4.1 Evaluation on Benchmark Functions

We first evaluate our RLTS on challenging benchmark test functions: Rosenbrock, Rastrigin, and Nesterov. Rastrigin and Rosenbrock are smooth multi-mode functions, and Nesterov is a non-smooth function. These functions are very challenging benchmarks for black-box optimization. We offset the optimum by setting x = x -5 of the test functions. This increases the distance between the optimum and the initial point µ = 0 , which makes the test problems more challenging. We implement INGO by ourselves. For CMAES, we use the publicly available code 2 . We initialized Σ = I for all the methods.

0

20000

40000

60000

80000

Number of Evaluations

0.0

2.5

5.0

7.5

10.0

12.5

15.0

Loss

RLTS

INGO

CMAES

(a) DBpedia

0

20000

40000

60000

80000

Number of Evaluations

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

Loss

RLTS

INGO

CMAES

(b) SS2

0

20000

40000

60000

80000

Number of Evaluations

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Loss

RLTS

INGO

CMAES

(c) SNLI

0

20000

40000

60000

80000

Number of Evaluations

0.0

0.5

1.0

1.5

2.0

2.5

Loss

RLTS

INGO

CMAES

(d) AG's News

0

20000

40000

60000

80000

Number of Evaluations

0.25

0.50

0.75

1.00

1.25

1.50

1.75

2.00

Loss

RLTS

INGO

CMAES

(e) MRPC

0

20000

40000

60000

80000

Number of Evaluations

0.5

1.0

1.5

2.0

Loss

RLTS

INGO

CMAES

(f) RTE

We evaluate RLTS on 50 and 500-dimensional problems. The batchsize of all the methods are set to 200 and 2000 for 50 and 500-dimensional problems, respectively. All the experiments are performed in ten independent runs. The experimental results are shown in Figure 2. From Figure 2, we can observe that RLTS consistently converge faster than INGO on all the test functions on both 50-dimensional and 500-dimensional cases. It shows that our RLTS significantly improves the query efficiency of INGO, which verifies the effectiveness of RLTS. Moreover, we can see that RLTS outperforms CMAES on all the test functions on both 50-dimensional and 500-dimensional cases. In addition, we see that CMAES converge slowly on the 500-dimensional benchmark problems, while RLTS converges faster.

4.2 Evaluation on Black-box Prompt Fine-tuning Tasks

Prompt fine-tuning of large language models is a promising direction to achieve expertise models efficiently for downstream tasks. We evaluate our RLTS on black-box prompt fine-tuning tasks.

We employ the deep model in [Sun et al., 2022a] with publicly available code 3 as the backbone model for black-box prompt fine-tuning. It has 24 layers. For each layer, we set the dimension of the continuous prompt to 500 . Thus, the total dimension is 24 × 500 . We employ the hinge loss of training data as the black-box objective. Six benchmark datasets for different language tasks are employed for evaluation: DBpedia, SS2, SNLI, AG's News, MRPC and RTE. The SST2 [Socher et al., 2013] dataset is a dataset for the sentiment analysis task. AG's News and DBPedia datasets [Zhang et al., 2015] are used for topic classification tasks. SNLI [Bowman et al., 2015] and RTE [Wang et al., 2019] are employed for natural language inference. MRPC dataset [Dolan and Brockett, 2005] is used for the paraphrasing task.

In all the experiments, we keep the number of batch samples and the initialization the same for RLTS, INGO and CMAES. We set the number of batch samples to 2000 . Specifically, our RLTS employs 1999 rank-1 lattice QMC Gaussian samples and one sample from targeted sampling. INGO employs 1999 rank-1 lattice QMC Gaussian samples and one Gaussian sample. CMAES employs 2000 Gaussian samples. We initialize the µ = 0 and Σ = 0 . 2 I for all the methods. For INGO and RLTS, we set the step-size parameter β = 0 . 2 in all experiments. For RLTS, we set the parameter η = 1 in all experiments. All the experiments are performed in five independent runs with seeds in { 1 , 2 , 3 , 4 , 5 } . The layer-wise coordinate descent update approach in [Sun et al., 2022a] is employed for all the methods.

0

10000

20000

30000

40000

50000

Number of Query Evaluations

0

1

2

3

4

Objective Value

1e7

RLTS

TuRBO

(a)

0

10000

20000

30000

40000

50000

Number of Query Evaluations

0

50000

100000

150000

200000

250000

Objective Value

RLTS

TuRBO

(b)

0

10000

20000

30000

40000

50000

Number of Query Evaluations

0

1000

2000

3000

4000

5000

6000

7000

Objective Value

RLTS

TuRBO

(c)

The experimental results of mean objective ± std v.s. the number of queries are shown in Figure 3. From Figure 3, we can observe that our RLTS decreases the objective significantly faster than INGO and CMAES on all six fine-tuning tasks, which shows the superior query efficiency of our RLTS.

4.3 Additional Comparison with High-dimensional Bayesian Optimization

We further compare our RLTS with the high-dimensional BO method TuRBO [Eriksson et al., 2019]. We evaluate RLTS on the three benchmark functions: the Rosenbrock function, the Rastrigin10 function, and the Nesterov function. We offset the optimum by setting x = x -5 of the test functions. The dimension is set to 500. The number of initial points of TuRBO is set to 2000. The batch size of both RLTS and TuRBO is set to 2000. The maximum number of queries is set to 50,000. We employ the default box boundary for TuRBO, i.e., [ -5 , 10] d . The initial parameter µ of RLTS is set to µ = 0 , and Σ is set to Σ = I . For TuRBO, we employ the official code provided in the paper [Eriksson et al., 2019]. All the methods are performed in three independent runs.

The convergence performance regarding the number of query evaluations is shown in Figure 4. We can observe that RLTS converges faster than TuRBO on the benchmark test problems, demonstrating that RLTS improves query efficiency.

We further report the running time of RLTS and TuRBO on the same machine for evaluation. The results are shown in Table 1. We can observe that RLTS performs significantly faster than TuRBO, achieving around 300 times speedup regarding running time. The computation time of Bayesian Optimization usually grows cubically fast as the number of queries increases. In contrast, our RLTS reduces the expensive O ( n 3 ) operation to O ( n log n ) time complexity, which enables a fast plug-in of the ES-type algorithms.

5 Conclusion

We proposed a novel Rank-1 Lattice Targeted Sampling technique in this paper. Our RLTS has a O ( n log n ) time complexity w.r.t. n batch samples, which is fast for plugging into stochastic optimization methods to improve query efficiency while scaling up to high-dimensional problems. Empirically, we plugged our RLTS into the sampling phase of INGO, significantly improving the query efficiency on benchmark test functions and black-box prompt fine-tuning tasks. Moreover, we proposed a closed-form rank-1 lattice by analyzing the integral approximation error of functions in Korobov space. Our closed-form rank-1 lattice provides an efficient way for QMC Gaussian sampling, with properties enabling fast exact GP training and inference with a O ( n log n ) time complexity, which is critical for our RLTS to be a fast internal step for stochastic optimization. In addition, our closed-form rank-1 lattice is a fundamental tool that may have potential applications beyond the black-box optimization task.

References

Thomas Back, Frank Hoffmeister, and Hans-Paul Schwefel. A survey of evolution strategies. In Proceedings of the fourth international conference on genetic algorithms , volume 2. Morgan Kaufmann Publishers San Mateo, CA, 1991.

Juan Cruz Barsce, Jorge A Palombarini, and Ernesto C Martínez. Towards autonomous reinforcement learning: Automatic setting of hyper-parameters using bayesian optimization. In Computer Conference (CLEI), 2017 XLIII Latin American , pages 1-9. IEEE, 2017.

Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. A large annotated corpus for learning natural language inference. Proceedings of the 2015 conference on empirical methods in natural language processing , 2015.

Krzysztof Choromanski, Aldo Pacchiano, Jack Parker-Holder, and Yunhao Tang. From complexity to simplicity: Adaptive es-active subspaces for blackbox optimization. arXiv:1903.04268 , 2019.

Josef Dick, Frances Y Kuo, and Ian H Sloan. High-dimensional integration: the quasi-monte carlo way. Acta Numerica , 22:133-288, 2013.

Josef Dick, Peter Kritzer, and Friedrich Pillichshammer. Lattice Rules . Springer, 2022.

Bill Dolan and Chris Brockett. Automatically constructing a corpus of sentential paraphrases. In Third International Workshop on Paraphrasing (IWP2005) , 2005.

David Steven Dummit and Richard M Foote. Abstract algebra , volume 3. Wiley Hoboken, 2004.

David Eriksson, Michael Pearce, Jacob Gardner, Ryan D Turner, and Matthias Poloczek. Scalable global optimization via local bayesian optimization. Advances in neural information processing systems , 32, 2019.

Jacob Gardner, Chuan Guo, Kilian Weinberger, Roman Garnett, and Roger Grosse. Discovering and exploiting additive structure for bayesian optimization. In Artificial Intelligence and Statistics , pages 1311-1319. PMLR, 2017.

Robert M Gray et al. Toeplitz and circulant matrices: A review. Foundations and Trends® in Communications and Information Theory , 2(3):155-239, 2006.

Nikolaus Hansen. The cma evolution strategy: a comparing review. In Towards a new evolutionary computation , pages 75-102. Springer, 2006.

José Miguel Hernández-Lobato, James Requeima, Edward O Pyzer-Knapp, and Alán Aspuru-Guzik. Parallel and distributed thompson sampling for large-scale accelerated exploration of chemical space. In International conference on machine learning , pages 1470-1479. PMLR, 2017.

Nikolai Mikhailovich Korobov. Properties and calculation of optimal coefficients. In Doklady Akademii Nauk , volume 132, pages 1009-1012. Russian Academy of Sciences, 1960.

Daniel J Lizotte, Tao Wang, Michael H Bowling, and Dale Schuurmans. Automatic gait optimization with gaussian process regression. In IJCAI , volume 7, pages 944-949, 2007.

Yueming Lyu and Ivor W Tsang. Black-box optimizer with stochastic implicit natural gradient. In Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13-17, 2021, Proceedings, Part III 21 , pages 217-232. Springer, 2021.

Yueming Lyu, Yuan Yuan, and Ivor Tsang. Subgroup-based rank-1 lattice quasi-monte carlo. Advances in Neural Information Processing Systems , 33:6269-6280, 2020.

Wesley Maddox, Qing Feng, and Max Balandat. Optimizing high-dimensional physics simulations via composite bayesian optimization. arXiv preprint arXiv:2111.14911 , 2021.

Seyedali Mirjalili and Seyedali Mirjalili. Genetic algorithm. Evolutionary Algorithms and Neural Networks: Theory and Applications , pages 43-55, 2019.

Sarah Müller, Alexander von Rohr, and Sebastian Trimpe. Local policy search with bayesian optimization. Advances in Neural Information Processing Systems , 34:20708-20720, 2021.

Amin Nayebi, Alexander Munteanu, and Matthias Poloczek. A framework for bayesian optimization in embedded subspaces. In International Conference on Machine Learning , pages 4752-4761. PMLR, 2019.

Diana M Negoescu, Peter I Frazier, and Warren B Powell. The knowledge-gradient algorithm for sequencing experiments in drug discovery. INFORMS Journal on Computing , 23(3):346-363, 2011.

Yurii Nesterov and Vladimir Spokoiny. Random gradient-free minimization of convex functions. Foundations of Computational Mathematics , 17(2):527-566, 2017.

Quan Nguyen, Kaiwen Wu, Jacob Gardner, and Roman Garnett. Local bayesian optimization via maximizing probability of descent. Advances in neural information processing systems , 35: 13190-13202, 2022.

Ingo Rechenberg and M. Eigen. Optimierung technischer Systeme nach Prinzipien der biologischen Evolution . PhD thesis, 1973.

Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever. Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864 , 2017.

I Sloan and A Reztsov. Component-by-component construction of good lattice rules. Mathematics of Computation , 71(237):263-273, 2002.

Ian H Sloan. Multiple integration is intractable but not hopeless. The ANZIAM Journal , 42(1):3-8, 2000.

Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical bayesian optimization of machine learning algorithms. In NeurIPS , pages 2951-2959, 2012.

Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing , pages 1631-1642, 2013.

Mandavilli Srinivas and Lalit M Patnaik. Genetic algorithms: A survey. computer , 27(6):17-26, 1994.

Niranjan Srinivas, Andreas Krause, Sham M Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In ICML , 2010.

Tianxiang Sun, Zhengfu He, Hong Qian, Yunhua Zhou, Xuanjing Huang, and Xipeng Qiu. Bbtv2: Towards a gradient-free future with large language models. In Proceedings of EMNLP , 2022a.

Tianxiang Sun, Yunfan Shao, Hong Qian, Xuanjing Huang, and Xipeng Qiu. Black-box tuning for language-model-as-a-service. In Proceedings of ICML , 2022b.

Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. In 7th International Conference on Learning Representations, ICLR , 2019.

G Gary Wang and Songqing Shan. Review of metamodeling techniques in support of engineering design optimization. Journal of Mechanical design , 129(4):370-380, 2007.

Daan Wierstra, Tom Schaul, Tobias Glasmachers, Yi Sun, Jan Peters, and Jürgen Schmidhuber. Natural evolution strategies. The Journal of Machine Learning Research (JMLR) , 15(1):949-980, 2014a.

Daan Wierstra, Tom Schaul, Tobias Glasmachers, Yi Sun, Jan Peters, and Jürgen Schmidhuber. Natural evolution strategies. The Journal of Machine Learning Research , 15(1):949-980, 2014b.

Christopher KI Williams and Carl Edward Rasmussen. Gaussian processes for machine learning , volume 2. MIT press Cambridge, MA, 2006.

Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in neural information processing systems , 28, 2015.

A Proof of Theorem 1

Theorem 1. Let n be a prime number such that (2 d -1) | ( n -1) . Suppose the integrand function f ∈ H k , ∥ f ∥ H k ≤ 1 , the square worst-case integral approximation error of rank-1 lattice P constructed by Eq.(30) is given as Eq.(33):

where ⊙ denotes the element-wise product, symbol 1 denotes the vector with elements all ones, and h i = F i γ with F as the discrete Fourier matrix, i.e., F jk = exp(2 π i jk n ) , and F i denotes the matrix after permutation of the rows of F such that the j th row of F i equals to the ˜ j th row of F , where ˜ j = jg i ( n -1) 2 d -1 mod n . And γ = [ γ 1 , · · · , γ n ] ⊤ with γ k = 1 n α ( ζ ( α, k i n ) + ζ ( α, n -k i n ) ) for k ∈ { 1 , · · · , n -1 } and γ n = 1 + 2 n α ζ ( α, 1) , where ζ ( · , · ) denotes the Hurwitz zeta function.

To prove our main Theorem 1, we begin with several Lemma.

Lemma 1. For ∀ x , y ∈ [0 , 1] d and α > 1 , define a reproducing kernel as Eq.(34)

where γ α ( k ) = ∏ d j =1 γ α ( k j ) with γ α ( k ) given in Eq.(35)

̸

Let P = [ x 1 , · · · , x n ] be a rank-1 lattice constructed by the generating vector z with a prime number n . Then, for ∀ f ∈ H k , ∥ f ∥ H k ≤ 1 associated with the reproducing kernel Eq.(34), we have the square worst-case integral approximation error of P as Eq.(36).

where L ⊥ denote the dual lattice defined in Eq.(37).

Proof. Given a point set P = { x 1 , · · · , x n } , the worst case approximation error for ∀ f ∈ H k , ∥ f ∥ H k ≤ 1 is

Appendix

Then, from the definition of the reproducing kernel K ( x , y ) in Eq.(34), we know that

̸

̸

In addition, the second term in Eq.(41) as follows

̸

̸

Moreover, from the definition of rank-1 lattice P with prime n and generating vector z , we have the third term in Eq.(41) as follows

̸

̸

̸

Put Eq.(44), Eq.(49) and Eq.(54) together , we know that

̸

Note that for a prime number n , we have

It follows that

where L ⊥ := { k | k ⊤ z ≡ 0 ( mod n ) , k ∈ Z d } denotes the dual lattice.

Lemma 2. Given a prime n , construct a rank-1 lattice P = [ x 1 , · · · , x n ] by the generating vector z = [ z 1 , · · · , z d ] , then we have that

where function χ ( · ) on domain { 1 , · · · , n } is given as Eq.(59)

where ζ ( · , · ) denotes the Hurwitz zeta function.

Proof. From Lemma 1, we know that

Now, we check the term ∑ k i ∈{ 1 , ··· ,n } ( ∑ q i ≡ k i mod n γ α ( q i ) ) . From the definition of the function γ α ( · ) , for ∀ k i ∈ { 1 , · · · , n } , we have that

Note that series ∑ ∞ m =1 1 ( mn ) α , ∑ ∞ m =0 1 ( k i + mn ) α and ∑ ∞ m =0 1 ( n -k i + mn ) α can be rewritten as

where ζ ( · , · ) denotes the Hurwitz zeta function.

Plug them into Eq.(64), we know that

Plug Eq.(68) into Eq.(63), we have that

Lemma 3. Let n be a prime number. Let γ = [ γ 1 , · · · , γ n ] ⊤ be a vector with γ k = χ ( k ) for k ∈ { 1 , · · · , n } , where χ ( · ) is defined in Lemma 2. The square worst-case integral approximation error of rank-1 lattice P constructed by generating vector z = [ z 1 , · · · , z d ] can be rewritten in a matrix form as Eq.(70)

where ⊙ denotes the element-wise product, symbol 1 denotes the vector with elements all ones, and h i = F i γ with F as the discrete Fourier matrix, i.e., F jk = exp(2 π i jk n ) , and F i denotes the matrix after permutation of the rows of F such that the j th row of F i equals to the ˜ j th row of F , where ˜ j = jz i +1 mod n .

Proof. Define h i as Eq.(71)

where F as the discrete Fourier matrix, i.e., F jk = exp(2 π i jk n ) , and F i denotes the matrix after permutation of the rows of F such that the j th row of F i equals to the ˜ j th row of F , where ˜ j = jz i +1 mod n , and g denotes the primitive root modulo n .

From Lemma 2, we know that

Note that γ = [ γ 1 , · · · , γ n ] ⊤ is a vector with γ k = χ ( k ) for k ∈ { 1 , · · · , n } , it follows that

Lemma 4. Let n be a prime number such that (2 d -1) | ( n -1) . Let γ = [ γ 1 , · · · , γ n ] ⊤ be a vector with γ k = χ ( k ) for k ∈ { 1 , · · · , n } , where χ ( · ) is defined in Lemma 2. Given a rank-1 lattice P constructed by generating vector in Eq.(30), then we have Eq.(76)

where ⊙ denotes the element-wise product, symbol 1 denotes the vector with elements all ones, and h i = F i γ with F as the discrete Fourier matrix, i.e., F jk = exp(2 π i jk n ) , and F i denotes the matrix after permutation of the rows of F such that the j th row of F i equals to the ˜ j th row of F , where ˜ j = jg i ( n -1) 2 d -1 mod n , and g denotes the primitive root modulo n .

Proof. Note that h i = F i γ is a permutation of h 0 . From the definition of permutation F i , we know that the j th row of F i equals to the ˜ j th row of F with ˜ j = jg i ( n -1) 2 d -1 mod n . Note that (2 d -1) | ( n -1) and n is a prime number, we know { 1 , g 1( n -1) 2 d -1 , · · · , g (2 d -2)( n -1) 2 d -1 } modulo n forms a subgroup of { 1 , · · · , n -1 } modulo n . Thus, we know { h 0 , h 1 , · · · , h 2 d -2 } forms a group, and h 0 = h 2 d -1 . Furthermore, we know that h k is a permutation of h i such that j th row of F k equals to the ¯ j th row of F i with ¯ j = jg ( k -i )( n -1) 2 d -1 mod n . Thus, we know that

Note that h 0 = h 2 d -1 . It follows that

In addition, we have that

It follows that

Plug Eq.(85) into Eq.(79), we know that

Lemma 5. Let n be a prime number such that (2 d -1) | ( n -1) . Let γ = [ γ 1 , · · · , γ n ] ⊤ be a vector with γ k = χ ( k ) for k ∈ { 1 , · · · , n } , where χ ( · ) is defined in Lemma 2. Given a rank-1 lattice P constructed by generating vector in Eq.(30), then we have Eq.(87)

where ⊙ denotes the element-wise product, symbol 1 denotes the vector with elements all ones, and h i = F i γ with F as the discrete Fourier matrix, i.e., F jk = exp(2 π i jk n ) , and F i denotes the matrix after permutation of the rows of F such that the j th row of F i equals to the ˜ j th row of F , where ˜ j = jg i ( n -1) 2 d -1 mod n , and g denotes the primitive root modulo n .

Proof. Similar to the proof of Lemma 4 , we have that

It follows that

Lemma 6. Let n be a prime number such that (2 d -1) | ( n -1) . Let γ = [ γ 1 , · · · , γ n ] ⊤ be a vector with γ k = χ ( k ) for k ∈ { 1 , · · · , n } , where χ ( · ) is defined in Lemma 2. Given a rank-1 lattice P constructed by generating vector in Eq.(30), then we have Eq.(94)

where ⊙ denotes the element-wise product, symbol 1 denotes the vector with elements all ones, and h i = F i γ with F as the discrete Fourier matrix, i.e., F jk = exp(2 π i jk n ) , and F i denotes the matrix after permutation of the rows of F such that the j th row of F i equals to the ˜ j th row of F , where ˜ j = jg i ( n -1) 2 d -1 mod n , and g denotes the primitive root modulo n .

Proof. Plug Eq.(76) in Lemma 4 into Eq.(87) in Lemma 5, we know that

Now we check the last two terms in Eq.(95). Note that

It follows that

Because { h 0 , h 1 , · · · , h 2 d -2 } forms a group, and h 0 = h 2 d -1 with a modulo period 2 d -1 , we know that

Plug Eq.(102) into Eq.(101), we have that

Now, we are ready to prove our main Theorem 1.

Proof. From Lemma 3, we know that

From Lemma 6, we know that

Plug Eq.(105) into Eq.(104), we have that

Note that h 0 = Fγ and F denotes the discrete Fourier matrix, we have that

where b = [0 , 0 , · · · , 0 , n ] ⊤ .

Note that the n th element in γ is γ n = 1+ 2 n α ζ ( α, 1) , where ζ ( · , · ) denotes the Hurwitz zeta function. It follows that

Plug Eq.(109) into Eq.(106), we achieve the result in Theorem 1

B Benchmark Test Functions

The benchmark test functions employed in section 4.1 are listed in Table 2, which contains multi-mode functions and non-smooth functions that are challenging for optimization.

C Training Time and Fast Coordinate Search Time

0.0

0.2

0.4

0.6

0.8

1.0

Number of Samples

1e6

0

10

20

30

40

50

Time (s)

Rank-1 Lattice GP

(a)

Training Time (seconds)

0.0

0.2

0.4

0.6

0.8

1.0

Number of Samples

1e6

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

Time (s)

Fast Coordinate Search

(b)

Fast Coordinate Search Time (seconds)

We provide the training time of our rank1 lattice GP and the time of our fast coordinate search for targeted sampling in Figure 5(a) and Figure 5(b), respectively. The dimension of the rank-1 lattice data is set to d = 50 . The number of samples n is set to the parameter in { 1783 , 5347 , 10099 , 51283 , 100189 , 501139 , 1000099 } . The number of samples n is a prime number such that (2 d -1) | ( n -1) to achieve our closed-form rank-1 lattice construction. The number of epochs of training is set to 2000 . The number of iterations of fast coordinate search is set to T = 50 . All the experiments are performed in 50 runs on a single NVIDIA A40 Card.

We report the mean value ± std in Figure 5. The standard deviation of the time is small. From Figure 5(a), we can see that it takes around 50 seconds for our rank-1 GP training with one million lattice data. Moreover, our fast coordinate search for targeted sampling takes around 1.5 seconds to optimize rank-1 lattice GP posterior prediction conditioned on one million lattice data.

D Additional Experiments of Black-box Prompt Fine-tuning

0

2500

5000

7500

10000

12500

15000

17500

Number of Evaluations

0

2

4

6

8

10

12

14

16

Loss

RLTS

INGO

CMAES

(a) DBpedia

0

2500

5000

7500

10000

12500

15000

17500

Number of Evaluations

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

Loss

RLTS

INGO

CMAES

(b) SS2

0

2500

5000

7500

10000

12500

15000

17500

Number of Evaluations

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Loss

RLTS

INGO

CMAES

(c) SNLI

0

2500

5000

7500

10000

12500

15000

17500

Number of Evaluations

0.5

1.0

1.5

2.0

2.5

Loss

RLTS

INGO

CMAES

(d) AG's News

0

2500

5000

7500

10000

12500

15000

17500

Number of Evaluations

0.25

0.50

0.75

1.00

1.25

1.50

1.75

2.00

Loss

RLTS

INGO

CMAES

(e) MRPC

0

2500

5000

7500

10000

12500

15000

17500

Number of Evaluations

0.50

0.75

1.00

1.25

1.50

1.75

2.00

2.25

Loss

RLTS

INGO

CMAES

(f) RTE

We provide additional experimental results of black-box prompt fine-tuning for large language models. We employ the deep model in [Sun et al., 2022a] as the backbone. It has 24 layers. For each layer, we set the dimension of the continuous prompt to 50 . Thus, the total dimension is 24 × 50 . We employ the hinge loss of training data as the black-box objective.

In all the experiments, we keep the number of batch samples and the initialization the same for RLTS, INGO and CMAES. We set the number of batch samples to 200 . Our RLTS employs 199 rank-1 lattice QMC Gaussian samples and one sample from targeted sampling. INGO employs 199 rank-1 lattice QMC Gaussian samples and one Gaussian sample. CMAES employs 200 Gaussian samples. We initialize the µ = 0 for all the methods. For INGO and RLTS, we set the step-size parameter β = 0 . 2 in all experiments. For RLTS, we set the parameter η = 1 in all experiments. All the experiments are performed in five independent runs with seeds in { 1 , 2 , 3 , 4 , 5 } .

The experimental results of mean objective ± std v.s. the number of queries are shown in Figure 6. From Figure 6, we can observe that RLTS decreases the loss consistently faster than INGO and CMAES on all benchmark datasets. More importantly, RLTS decreases the loss significantly faster than INGO. Note that RLTS employs INGO as the backbone algorithm, which shows that RLTS improves the query efficiency of INGO.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents the Rank-1 Lattice Targeted Sampling (RLTS) technique as a solution to the challenge of efficiently scaling up black-box optimization in high-dimensional problems. The primary objective of this technique is to enhance the query efficiency while ensuring fast Gaussian process (GP) training and targeted sampling. Additionally, the authors derive a closed-form rank-1 lattice that facilitates rapid lattice sampling. By leveraging these contributions, the paper addresses the need for efficient and scalable optimization methods in high-dimensional settings.\n\nSTRENGTHS:\nThe authors of this paper address a crucial issue regarding the performance of Quasi-Monte Carlo methods in high-dimensional problems. They emphasize the challenges associated with effectively applying these methods in such scenarios, as high-dimensional optimization poses significant difficulties and requires a larger number of samples for accurate results. To overcome these challenges, the authors propose an alternative method that aims to enhance efficiency and mitigate the slow convergence of traditional Monte Carlo methods in high dimensions. Their proposed approach offers a solution to improve the performance of Quasi-Monte Carlo methods in high-dimensional optimization tasks.\n\nWEAKNESSES:\n- While the paper explores important concepts, it could benefit from improved cohesion and clarity in terms of presenting technical details and providing necessary context.\n   - The paper lacks in-depth descriptions of certain important details. For instance, a more comprehensive discussion on lattice methods and low-discrepancy sequences like Sobol sequence, would provide a solid introduction to the topic and enable readers to grasp the advantages of the rank-$1$ lattice approach more clearly. While the advantages of rank-$1$ lattices might be apparent to the authors, it is crucial to explicitly explain these benefits to the readers, ensuring that the significance of this method is clearly conveyed.\n   - Another aspect that requires clarification is the initial set of points, denoted as $x \\in \\mathcal{X}$, and the input box boundaries of the problem. The paper does not explicitly discuss how these initial points should be obtained or provide information regarding the problem's boundaries. Moreover, the process of obtaining the observed values $y$ is only mentioned in Algorithm 2, leaving questions regarding the data acquisition process in the main text.\n- The paper acknowledges the challenges faced by Bayesian optimization (BO) when applied to high-dimensional problems. However, the paper lacks a thorough comparison between the proposed RLTS technique and existing BO methods designed for addressing such problems, such as 1) trust regions, 2) structured kernels, 3) subset of active dimensions, or random projections [1, 2, 3]. The paper have not clearly articulated the unique features or advantages that make the RLTS approach competitive with established BO methods. Without explicit confirmation or evidence, it is unclear whether the proposed method performs better, equally, or worse than existing BO techniques in high-dimensional scenarios. Consequently, the comparative performance and potential advantages of the proposed RLTS technique in relation to BO methods remain uncertain.\n- The paper does not provide sufficient details regarding their experimental setup, which raises questions about key aspects such as the selection of parameter $T$, the number of data points $n$ involved in the problem, and the specifics of Gaussian process (GP) training. In order to substantiate their claim of accelerated methods, it would be beneficial for the authors to include plots that demonstrate the performance with respect to both time and iterations.\n\n[1] Binois, Mickael and Wycoff, Nathan, \"A survey on high-dimensional Gaussian process modeling with application to Bayesian optimization\", 2022, https://arxiv.org/abs/2111.05040 \\\n[2] Diouane, Youssef and Picheny, Victor and Riche, Rodolophe Le and Perrotolo, Alexandre Scotto Di, \"TREGO: a trust-region framework for efficient global optimization\", 2023, https://arxiv.org/abs/2101.06808 \\\n[3] Eriksson, David and Pearce, Michael and Gardner, Jacob and Turner, Ryan D and Poloczek, Matthias, \"Scalable global optimization via local Bayesian optimization\", 2019, https://arxiv.org/abs/1910.01739\n\nQUESTIONS:\n- Although the paper mentions the complexity of the RLTS technique as $n \\log(n)$, it does not provide a comprehensive analysis of the overall computational costs. For instance, the targeted search (Algorithm 1) exhibits a time complexity of $O(T d n \\log(n))$, and Algorithm 2 involves computationally expensive cubic operations, such as computing the Cholesky decomposition or the square root of the covariance matrix denoted as $\\Sigma^{1/2}$, for calculating the weights $w$ (as shown in Equation 25) and updating $\\Sigma$. These operations contribute to an overall time complexity of $O(n^3)$ of a sigle iteration of Algorithm 2. Given these complexities, the question arises regarding the specific contribution of the RLTS speed-up to the efficiency of the overall algorithm.\n- Could the authors please provide a more detailed explanation of the derivation process for equations 18 and 19? The absence of a thorough explanation and references raises concerns regarding the steps involved in combining the Fourier transform of $y$ with the Fourier transform of the kernel $k$. Alternatively, if a reference exists that explains the derivation, kindly provide the appropriate citation.\n- It would be beneficial for the authors to conduct a comprehensive comparison study to evaluate and elucidate the strengths and limitations of their method relative to existing BO techniques in high-dimensional settings. It is also improtant to demonstrate that the RLTS addresses the high-dimensionality problem.\n- I would appreciate clarification from the authors regarding the stationarity requirement in the paper. While they assert that the kernel $k_\\Delta$ should be stationary (shift invariant or translation invariant), it seems that the stationarity property should be satisfied by the function $\\phi$ instead. Since the kernel $k_\\Delta$ depends on the distance between data points, which is determined by the output of the $\\phi$ function.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies high-dimensional black-box optimization. In order to achieve high efficiency, this paper proposes a Rank-1 Lattice Targeted Sampling (RLTS) technique, using random rank-1 lattice Quasi-Monte Carlo, to perform fast local exact Gaussian processes (GP) training and inference with O(n log n) complexity w.r.t. n batch samples. In addition, this paper developed a fast coordinate searching method with O(n log n) time complexity for fast targeted sampling, improving the query efficiency while scaling up to high dimensional problems. The authors also proposed a closed-form construction to construct rank-1 lattices efficiently.\n\nSTRENGTHS:\nThis paper is generally well-structured, and the presentation is clear to the readers. The proposed sampling method for the Black-box optimization problem is novel in this area, which leads to low time complexity and scalability to high dimensional problems. The authors also provide theoretical analysis to show the error pattern of their rank-1 lattice construction. The authors further conduct an empirical evaluation to verify the effectiveness and efficiency of the proposed method on realistic tasks compared to other approaches.\n\nWEAKNESSES:\nSince my evaluation of this submission is an educated guess, I would like to modify my rating according to other reviewers' feedback and the authors' response to other reviewers' questions. I have one question regarding this work: In this experiment section, the authors demonstrate the comparison between different approaches via the number of evaluations. Can the authors explain if RLTS still has a faster convergence than other compared methods in terms of physical time?\n\nQUESTIONS:\nPlease see the above.\n\nLIMITATIONS:\nThe Paper Checklist Guidelines in https://neurips.cc/public/guides/PaperChecklist encourage authors to discuss the limitation, especially in a separate \"Limitations\" section if possible. But I was unaware of a detailed discussion on the limitations in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 1, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-27"}

comments: SUMMARY:
For the setting of black-box optimization where only function queries are available, this paper propose a rank-1 lattice targeted sampling technique to scale up to high-dimensional problems with good query efficiency. Based on the special structure, this paper use FFT/IFFFT to do fast coordinate search with O(n\log n) time complexity.

STRENGTHS:
The authors have a clear background introduction for black-box optimization. Meanwhile, introducing the lattice structure to optimization is new to me. Finally, applying FFT/IFFT with the Lattice structure to get an O(n\log n) time complexity is also impressive.

WEAKNESSES:
The narrative of this paper assumes that the readers have good knowledge of Lattice, but I think that many readers don't have this kind o knowledge. Some concept appears without explanation such as "dual lattice". As a result, it is better to give a more detailed introduction to Lattice, including why this kind of structure can work in the machine-learning context.

Meanwhile, the concept of mod 1 is not clear --- what is the meaning of nonnegative fractional part??? What is the value of a decimal mod 1? Also, what is the correlation between "spaced more evenly " with fast targeted sampling? 

Finally, this paper gives a new and interesting structure to reduce complexity, but it doesn't either provide strong convergence results and a strong theoretical verification or provide any meaningful experiments.  




,

QUESTIONS:
See the Weaknesses

LIMITATIONS:
No
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel Rank-1lattice targeted sampling (RLTS) strategy to significantly reduce the GP training & inference time complexity and also enable $O(n\\log(n))$ time complexity in order to find a targeting reference point at each iteration. Combined with the evolution strategies, the method proposed in this paper is proven to be efficient in handling high-dimensional problems. The effectiveness of the method is also demonstrated in some benchmark testing functions and a prompt fine-tuning task.\n\nSTRENGTHS:\n(1) The rank-1 lattice sampling strategy reduces the computational/time complexity of GP training/inference significantly in several aspects, which should be interesting to the community\n\n(2) The author also provides a theoretical result to show that the approximation error of this new sampling method is comparable to the integral approximation error.\n\n(3) The effectiveness of the new approach is supported by sufficient experiments.\n\nWEAKNESSES:\n(1) Some notation is confusing, for example, $x_i$ in eq(10) and $x_i$ in line 8 of algorithm 2 are different but share the same notation.\n\n(2) For better presentation, the author can consider claiming the assumptions explicitly (for example in sec 3.3 the kernel is required to have a decomposition structure in order to make sure the coordinate searching works)\n\n(3) In GP training, in addition to considering the complexity of computing the value of likelihood, isn't it also important to consider the complexity cost in optimizing the likelihood? Such a result seems to be missing in the paper.\n\nQUESTIONS:\n(1) What is the overall query complexity of Algorithm 2 or how much overall query complexity can RLTS saves compared with INGO? Even if it is a plug-in version of INGO such a result will be helpful for readers to understand the advantage of RLTS over INGO quantively.\n\n(2) How does the author define \"time complexity\" and \"query complexity\" in this paper? Are they share the same definition or not? If time complexity refers to the computation cost of solving a concrete optimization sub-problem in the overall algorithm while the query complexity refers to the total number of zeroth-order information that we need, then there is a inconsistency between the motivation and the contribution claimed by the author. (The motivation of using rank-1 sampling seems to target at reducing time complexity while the experiment try to demonstrate that rank-1 sampling can actually reduce query complexity).\n\nLIMITATIONS:\nThere no limitation in this paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-08-02"}