Title: PROTES: Probabilistic Optimization with Tensor Sampling
Venue: NeurIPS 2023
Accepted: yes
Keywords: Tensor Train

Black Box Optimization

Sampling

Optimal Control
Abstract: We developed a new method PROTES for black-box optimization, which is based on the probabilistic sampling from a probability density function given in the low-parametric tensor train format. We tested it on complex multidimensional arrays and discretized multivariable functions taken, among others, from real-world applications, including unconstrained binary optimization and optimal control problems, for which the possible number of elements is up to $2^{1000}$. In numerical experiments, both on analytic model functions and on complex problems, PROTES outperforms popular discrete optimization methods (Particle Swarm Optimization, Covariance Matrix Adaptation, Differential Evolution, and others).

PROTES: Probabilistic Optimization with Tensor Sampling

Abstract

We developed a new method PROTES for black-box optimization, which is based on the probabilistic sampling from a probability density function given in the low-parametric tensor train format. We tested it on complex multidimensional arrays and discretized multivariable functions taken, among others, from real-world applications, including unconstrained binary optimization and optimal control problems, for which the possible number of elements is up to 2 1000 . In numerical experiments, both on analytic model functions and on complex problems, PROTES outperforms popular discrete optimization methods (Particle Swarm Optimization, Covariance Matrix Adaptation, Differential Evolution, and others).

1 Introduction

The multidimensional optimization problem is one of the most common in machine learning. It includes the important case of gradient-free discrete optimization [30, 20, 21, 41]:

where d is the dimensionality of the problem, and N 1 , N 2 , . . . , N d are the numbers of items for each dimension. Such settings arise when searching for the minimum or maximum element in an implicitly given multidimensional array (tensor 1 ), including when considering the discretization of functions from a continuous argument. Multidimensional discrete optimization problems are still computationally difficult in the case of complex target functions or large dimensions, and efficient direct gradient-free optimization procedures are highly needed.

The development of methods for low-rank tensor approximations has made it possible to introduce fundamentally new approaches for the approximation, storage, and operation with multidimensional

tensors [13, 6, 7, 40]. One of the common methods for low-rank approximation is the tensor train (TT) decomposition [26], which allows bypassing the curse of dimensionality. Many useful algorithms (e. g., element-wise addition, multiplication, solution of linear systems, convolution, integration, etc.) have effective implementations for tensors given in the TT-format. The complexity of these algorithms turns out to be polynomial in the dimension d and the mode size N . It makes the TT-decomposition extremely popular in a wide range of applications, including computational mathematics [29, 2] and machine learning [31, 19].

In the last few years, new discrete optimization algorithms based on the TT-format have been proposed: TTOpt [37], Optima-TT [5], and several modifications [34, 35, 24, 36]. However, the development of new, more accurate, and robust TT-based methods for multidimensional discrete optimization is possible. In this work, we extend recent approaches for working with probability distributions and sampling in the TT-format [10, 25] to the optimization area. That is, we specify a multidimensional discrete probability distribution in the TT-format, followed by efficient sampling from it and updating 2 its parameters to approximate the optimum in a better way. This makes it possible to build an effective optimization method, and the contributions of our work are as follows:

We develop a new method PROTES for optimization (finding the minimum or maximum 3 value) of multidimensional data arrays and discretized multivariable functions based on a sampling method from a probability distribution in the TT-format;

We apply 4 PROTES for various analytic model functions and for several multidimensional QUBO and optimal control problems to demonstrate its performance and compare it with popular discrete optimization algorithms (Particle Swarm Optimization, Covariance Matrix Adaptation, Differential Evolution, and NoisyBandit) as well as TT-based methods (TTOpt and Optima-TT). We used the same set of hyperparameters of our algorithm for all experiments and obtained the best result for 19 of the 20 problems considered.

2 Optimization with probabilistic sampling

Our problem is to minimize the given multivariable discrete black-box function f (1). It can be formulated in terms of the multi-index search in an implicitly given d -dimensional tensor

for all n i = 1 , 2 , . . . , N i ( i = 1 , 2 , . . . , d ). The essence of our idea is to use a probabilistic method to find the minimum x min . We propose establishing a discrete distribution p ( x ) from which the minimum could be sampled with high probability. This distribution can be specified as a tensor P θ ∈ R N 1 × N 2 × ... × N d in some low-parametric representation with a set of parameters θ , having the same shape as the target tensor Y .

We start from a random non-negative tensor P θ and iteratively perform the following steps until the budget is exhausted or until convergence (see graphic illustration in Figure 1):

Sample K candidates of x min from the current distribution P θ : X K = { x 1 , x 2 , . . . , x K } ;

Compute the corresponding function values: y 1 = f ( x 1 ) , y 2 = f ( x 2 ) , . . . , y K = f ( x K ) ;

Select k best candidates with indices S = { s 1 , s 2 , . . . , s k } from X K with the minimal objective value, i. e., y j ≤ y J for all j ∈ S and J ∈ { 1 , 2 , . . . , K } \ S ;

Update the probability distribution P θ ( θ → θ ( new ) ) to increase the likelihood of selected candidates S . We make several ( k gd ) gradient ascent steps with the learning rate λ for the tensor P θ , using the following loss function

Sample

K multi-indices

Select

k top items

Update

for selected multi-indices

Compute

black box values

Black

Box

...

Probability

tensor

...

...

...

...

After a sufficient number of iterations, we expect the tensor P θ to represent an almost Kronecker delta-function with a pronounced peak in the value of the minimum of the target function (or several peaks if the minimum is not unique). Therefore, this value will be sampled during the steps of our algorithm since the probability of sampling other values will be sufficiently small.

From a low-parameter representation P θ we expect an efficient sampling algorithm and efficient calculation procedure for the logarithms in (2) with automatic differentiation capability to enable gradient ascent methods. As will be shown below, the TT-representation of tensors satisfies these requirements. Further, we will omit the index θ , assuming that the parameterized representation of the tensor P corresponds to the TT-format. Note that the values K , k , k gd , λ and the number of parameters in θ (i. e., rank of the TT-decomposition) are the hyperparameters of our algorithm.

3 Basic properties of the tensor train format

Let us dwell on the concept of the TT-format. A d -dimensional tensor P ∈ R N 1 × N 2 × ... × N d is said to be in the TT-format [26] if its elements are represented by the following formula

where ( n 1 , n 2 , . . . , n d ) is a multi-index ( n i = 1 , 2 , . . . , N i for i = 1 , 2 , . . . , d ), integers R 0 , R 1 , . . . , R d (with convention R 0 = R d = 1 ) are named TT-ranks, and three-dimensional tensors G i ∈ R R i -1 × N i × R i ( i = 1 , 2 , . . . , d ) are named TT-cores. The TT-decomposition (3) (see also an illustration in Figure 2) allows to represent a tensor or a discretized multivariable function in a compact and descriptive low-parameter form, which is linear in dimension d , i. e., it has less than d · max i =1 ,...,d ( N i R 2 i ) ∼ d · N · R 2 parameters, where N and R are effective ('average') mode size and TT-rank respectively.

Linear algebra operations (e. g., element-wise addition, solution of linear systems, convolution, integration, etc.) on tensors in the TT-format, respectively, also have complexity linear in dimension if the TT-ranks are bounded. The TT-approximation for a tensor or discretized multivariable function may be built by efficient numerical methods, e. g., TT-SVD [27], TT-ALS [4], and TT-cross [28]. A detailed description of the TT-format and related algorithms are given in the works [26, 6, 7]. Below, we discuss only three operations in the TT-format, which will be used later in the work.

Construction of the random TT-tensor. In order to build a random non-negative TT-tensor of a given size ( N 1 , N 2 , . . . , N d ) with a constant TT-rank R , it is enough to generate d TT-cores G 1 , G 2 , . . . , G d ( 3 -dimensional tensors) with random elements from the uniform distribution on the interval (0 , 1) . We will refer to this method as tt_random ( R, [ N 1 , N 2 , . . . , N d ]) .

Computation of the log-likelihood in the TT-format. To calculate the logarithm log P [ x ] in a given multi-index x = ( n 1 , n 2 , . . . , n d ) , we can use the basic formula (3) and then take the logarithm of the result. It can be shown that this operation has complexity O ( d · R 2 ) , because, roughly speaking, we ( d -1) times multiply a vector of length R by a matrix of size R × R to get the result. The corresponding method will be called tt_log ( P , x ) .

Sampling from the tensor in the TT-format. We use the approach proposed in the work [10] to generate a multi-index x with a probability proportional to the corresponding value p = P [ x ] of the TT-tensor P . The method is based on the sequential calculation of univariate conditional densities with efficient integration in the TT-format. The estimate for its complexity turns out to be the following: O ( K · d · ( N + R ) · R + K · d · α ( N ) ) , where K is a number of requested samples, and α ( n ) is a complexity of sampling from generalized Bernoulli distribution with n outcomes. Note that the algorithm allows sampling in the case of the initially non-normalized tensor, so we do not have to calculate the normalization factor. We will refer to this method as tt_sample ( P , K ) .

4 Optimization method PROTES

With the formal scheme of the proposed approach given in Section 2 and the description of operations tt_random , tt_log , and tt_sample given in Section 3, we can formulate our method PROTES for gradient-free discrete optimization in the TT-format, as presented in Algorithm 1. We denote as adam , a function that performs k gd steps of gradient ascent for the TT-tensor P at multi-indices X by the well-known Adam method [18]. In this case, the learning rate is λ , the loss function is given in (2), and tt_log with automatic differentiation support is used for the log-likelihood computation.

Computational complexity of the method. Let us estimate the complexity of the proposed algorithm, assuming that the number of requests to the target function (black-box) M is fixed. With the known estimate for the complexity of the tt_sample function, we can obtain the complexity of the sampling operations: O ( M K · K · d · ( ( N + R ) · R + α ( N ) )) . Assuming that the complexity of one gradient step coincides with the complexity of calculating the differentiated function and using the estimate for the tt_sample function, we can estimate the total complexity of the tensor updates: O ( M K · k · k gd · d · R 2 ) . Combining the two above estimates we obtain the complexity of the method

Algorithm 1 Method PROTES in the TT-format for multidimensional discrete black-box optimization

Data: the function f ( x ) , that computes the value of the target tensor Y ∈ R N 1 × N 2 × ... × N d at the multi-index x = [ n 1 , n 2 , . . . , n d ] ; the maximum number of requests M ; the number of generated samples per iteration K ; the number of selected candidates per iteration k ; the number of gradient ascent steps k gd ; the gradient ascent learning rate λ ; the TT-rank of the probability tensor R .

Result: d -dimensional multi-index x min , which relates to the minimum value of the tensor Y .

1 Initialize target multi-index and tensor value: x min = None , y min = ∞

2 Generate random non-negative rankR TT-tensor: P = tt_random ( R, [ N 1 , N 2 , . . . , N d ])

3 for iter = 1 to M/K do

4 Generate K samples from P : x 1 , x 2 , . . . , x K = tt_sample ( P , K )

5 Compute related tensor values: y 1 = f ( x 1 ) , y 2 = f ( x 2 ) , . . . , y K = f ( x K )

6 Find indices S = { s 1 , s 2 , . . . , s k } for topk minimum items in the list [ y 1 , y 2 , . . . , y K ]

7 Collect k-top candidates: X = { x s 1 , x s 2 , . . . , x s k } , Y = { y s 1 , y s 2 , . . . , y s k }

8 If Y contains a value less than y min , then update x min and y min

9

Update the TT-tensor:

10 return x min .

However, it is important to note that this estimate does not consider the complexity of calculating M times the objective function f , which in practical applications can be significant and many times greater than the estimate (4).

The intuition behind the method. The proposed method PROTES, like most gradient-free optimization approaches, is empirical; however, we can establish its connection with a well-known REINFORCE trick algorithm [43]. Let make a monotonic transformation F [ f ]( x ) of the target function f to be minimized that transforms minimum to maximum. A reasonable choice for F [ · ] is the Fermi-Dirac function

where y min is an exact or approximate minimum of f , T > 0 is a parameter and E > 0 is some small threshold. With the function F [ f ] we can find a maximum of the expectation max θ E ξ θ F [ f ]( ξ θ ) , where a family of random variables ξ θ has a parameterised distribution function p θ ( x ) . Using REINFORCE trick, we can estimate the gradient of the expectation by the following Monte-Carlo-like expression

where { x i } M 1 are independent realizations of the random variable ξ θ . If we find the optimal values of θ , then we expect the optimal distribution p θ to peak at the maximum point for function F [ f ] . Thus, we can obtain the argument of its maximum by sampling from this distribution. For very small values of T , only a few terms contribute to the sum (5), namely those x i for which f ( x i ) -y min < E is hold. For these values of x , F [ f ] is close to 1 , while for the other samples its value is 0 . Hence, we can discard all other samples and keep a few with the best values. So, we come to the loss function (2), where instead of the parameter E we use a fixed number k of the best samples, i. e., the samples for which the value of the target function f is the smallest.

Application of the method to constrained optimization. A very nice property of the proposed method is that it can be adapted to efficiently handle constraints such as a specified set of admissible multi-indices. One option is just to remove invalid samples from the top-k values, but in some cases, the probability of sampling multi-indices that are admissible is very low, so this approach will not work. Instead, if the constraint permits, we use the algorithm from the work [32] for the constructive building of tensors in the TT-format by a known analytic function, which defines the constraints. Once the indicator tensor ( 1 if the index is admissible and 0 if it is not) is built in the TT-format, we can just initialize the starting distribution P by it, and it will be guaranteed that the samples almost always belong to the admissible set.

P

adam

(

,

P

L,

̂

, k

X

gd

, λ

)

//

with the loss function

(2)

and the method tt_log

5 Related work

Below, we briefly analyze classical approaches for discrete optimization and then discuss the methods based on the low-rank tensor approximations, which have become popular in recent years.

Classical methods for gradient-free optimization. In many situations, the problem-specific target function is not differentiable, too complex, or its gradients are not helpful due to the non-convex nature of the problem [1], and standard well-known gradient-based methods cannot be applied directly. The examples include hyper-parameter selection, training neural networks with discrete weights, and policy optimization in reinforcement learning. In all these contexts, efficient direct gradientfree optimization procedures are highly needed. As for high-dimensional black-box optimization, evolutionary strategies (ES) [9] are one of the most advanced methods. This approach aims to optimize the parameters of the search distribution, typically a multidimensional Gaussian, to maximize the objective function. Finite difference schemes are commonly used to approximate gradients of the search distribution. Numerous works proposed techniques to improve the convergence of ES [23]; for example, second-order natural gradient [42] or the history of recent updates (Covariance Matrix Adaptation Evolution Strategy; CMA-ES) [15] may be used to generate updates. There is also a large variety of other heuristic methods for finding the global extremum. In particular, we note such popular approaches as NoisyBandit [33], Particle Swarm Optimization (PSO) [17], Simultaneous Perturbation Stochastic Approximation (SPSA) [22], Differential Evolution (DE) [38] and scrambled-Hammersley (scr-Hammersley) [14].

Tensor-based methods for gradient-free optimization. Recently, the TT-decomposition has been actively used for multidimensional optimization. An iterative method TTOpt based on the maximum volume approach is proposed in the work [37]. TTOpt utilizes the theorem of sufficient proximity of the maximum modulo element of the submatrix having the maximum modulus of the determinant to the maximum modulo element of the tensor. Based on this observation, tensor elements are sampled from specially selected successive unfoldings of the tensor. Dynamic mapping of the tensor elements is carried out to find the minimum element, which converts the minimum values into maximum ones. The authors applied this approach to the problem of optimizing the weights of neural networks in the framework of reinforcement learning problems in [37] and to the QUBO problem in [24]. A similar optimization approach was also considered in [34] and [35]. One more promising algorithm, Optima-TT, was proposed in recent work [5]. This approach is based on the probabilistic sampling from the TT-tensor and makes it possible to obtain a very accurate approximation for the optimum of the given TT-tensor. However, this method is intended for directly optimizing the TT-tensors, which means that its success strongly depends on the quality of the TT-approximation for the original multidimensional data array. Therefore, one of the related methods in the TT-format (TT-SVD, TT-ALS, TT-cross, etc.) should be additionally used for approximation.

6 Numerical experiments

To evaluate the effectiveness of the proposed method, we carried out a series of 20 numerical experiments for various formulations of model problems. The results are presented in Table 1, where we report the approximation to the minimum value for each model problem (P-1, P-2, . . . , P-20) and all considered optimization methods (PROTES, BS-1, BS-2, . . . , BS-7). Taking into account the analysis of discrete optimization methods in the previous section as baselines we consider two tensor-based optimization methods: TTOpt 5 (BS1) and Optima-TT 6 (BS2), and five popular gradientfree optimization algorithms from the nevergrad framework [3]: 7 OnePlusOne (BS3), PSO (BS4), NoisyBandit (BS5), SPSA (BS6), and Portfolio approach (BS7), which is based on the combination of CMA-ES, DE, and scr-Hammersley methods. The model problems and obtained results will be discussed in detail below in this section.

For all the considered optimization problems, we used the default set of parameters for baselines, and for PROTES we fixed parameters as K = 100 , k = 10 , k gd = 1 , λ = 0 . 05 , R = 5 (the description

of these parameters was presented in Algorithm 1). For all methods, the limit on the number of requests to the objective function was fixed at the value M = 10 4 . As seen from Table 1, PROTES, in contrast to alternative approaches, gives a consistently top result for almost all model problems (the best result for 19 of the 20 problems considered).

6.1 Multivariable analytic functions

First, we consider the optimization task for various tensors arising from the discretization of multivariable analytic functions. We select 10 popular benchmarks: Ackley (P-01), Alpine (P-02), Exponential (P-03), Griewank (P-04), Michalewicz (P-05), Piston 8 (P-06), Qing (P-07), Rastrigin (P-08), Schaffer (P-09) and Schwefel (P-10). These functions have a complex landscape and are often used in problems to evaluate the effectiveness of optimization algorithms [8, 16], including tensor-based optimizers [4, 39]. We consider the 7-dimensional case (since this is the dimension of the Piston function) and discretization on a uniform grid with 16 nodes.

As follows from Table 1 (benchmarks P-1, P-2, . . . , P-10), our method, like the other two tensor approaches (BS-1 and BS-2), gave the most accurate solution for all model problems. The most sophisticated approach from the nevergrad package (BS-7) was the next in accuracy (the method did not converge only in two cases out of ten).

6.2 Quadratic unconstrained binary optimization

QUBO is a widely known NP-hard problem [12] which unifies a wide variety of combinatorial optimization problems from finance and economics applications to machine learning and quantum computing. QUBO formulation in a very natural manner utilizes penalty functions, yielding exact model representations in contrast to the approximate representations produced by customary uses of penalty functions. The standard QUBO problem can be formulated as follows

where x is a vector of binary decision variables of the length d and Q ∈ R d × d is a square matrix of constants. In all our experiments, we fixed the number of dimensions as d = 50 .

We consider the following QUBO problems from the qubogen package: 9 Max-Cut Problem (P-11; which refers to finding a partition of an undirected graph into two sets such that the number of edges between the two sets is as large as possible), Minimum Vertex Cover Problem (P-12; which refers to finding a cover with a minimum number of vertices in the subset of the graph vertices such that each edge in the graph is incident) and Quadratic Knapsack Problem (P-13; which refers to finding a subset of maximum profit that satisfies the budget limitations from a set of potential projects with specified interactions between pairs of projects).

We also consider one more benchmark (P-14) from the work [11] (problem k 3 ; d = 50 ), where angle-modulated bat algorithm (AMBA) was proposed for high-dimensional QUBO problems with engineering application to antenna topology optimization. This is the ordinary binary knapsack problem with fixed weights w i ∈ [5 , 20] , profits p i ∈ [50 , 100] ( i = 1 , 2 , . . . , d ), and the maximum capacity C = 1000 . In experiments, we used the same values of the weights and profits as in [11].

For all four considered problems (P-11, P-12, P-13, P-14), the proposed method PROTES gives the best result, as seen from Table 1, and the baseline BS-7 again turned out to be the next in accuracy. We also note that several optimization methods were compared in [11] for the P-14 problem: BPSO (with the result -2854 ), BBA (with the result -2976 ), AMBA (with the result -2956 ), A-AMBA (with the result -2961 ), P-AMBA (with the result -2989 ), and the solution obtained using the PROTES method (the result -3079 ) turns out to be more accurate.

To study the stability of the proposed optimization method in the essentially multidimensional case, we also consider d = 1000 for the benchmark P-12. Additionally, we vary the value of the vertex connection probability in the generated random graph (i.e., the parameter in the corresponding function from the well-known networkx package), getting 5 different optimization problems with p = 0 . 1 , p = 0 . 2 , p = 0 . 3 , p = 0 . 4 , and p = 0 . 5 (note that in Table 1, we used the value p = 0 . 5 ). We performed the calculations with PROTES and baselines BS1 - BS7 with the same settings as above. In Table 2, we report the results averaged over 5 runs of each optimizer for each value of p . In all cases, PROTES demonstrates the best result. We note that BS2 failed for this problem because the selected dimension value is too large to construct the TT-approximation of the tensor.

In Figure 3, we plotted the convergence curves for p = 0 . 1 , p = 0 . 3 , p = 0 . 5 , and all optimizers (except the BS2, since it failed), showing the average result and the spread of values over 5 restarts for P-12 ( d = 1000 ). Our optimizer PROTES has a slight variance of the solution and consistently shows the best result when the number of iterations exceeds 2000 .

6.3 Optimal control

Suppose we have a state variable z ∈ R controlled by a binary variable x called control (i.e., it is just a switch with modes 'off' = 0 and 'on' = 1) over some discrete interval of time [0 , T ] . The state z ( t +1) at time t +1 depends on the control x ( t ) at time t and is obtained from the solution of the following differential equation ˙ z ( τ ) = g ( z ( τ ) , x ( t )) , t ≤ τ < t +1 , where the function g is called an equation function. The optimal control problem is to find such a sequence of controls x ∗ = [ x ∗ (0) , x ∗ (1) , . . . , x ∗ ( T )] (optimal solution) over the given time interval [0 , T ] that minimizes the given objective function G .

(a)

p

= 0

.

1

(scaled

×

10

-

5

)

(b)

p

= 0

.

3

(scaled

×

10

-

6

)

(c)

p

= 0

.

5

(scaled

×

10

-

6

)

-3.00

-1.0

-1.6

-3.25

PROTES

BS-1

BS-1

PROTES

-1.7

BS-1

PROTES

BS-3

1.1

BS-3

-1.8

BS-3

-3.50

BS-4

BS-4

BS-4

-3.75

BS-5

BS-6

-1.2

BS-5

BS-6

-2.0

-1.9

BS-5

BS-6

-4.00-

BS-7

BS-7

BS-7

2.1 -

-1.3

-4.25-

-2.2

4.50

1.4

-2.3

4.75

2.4

5.00+

-1.5+

2.5+

10°

101

10²

103

10

10°

101

10²

103

104

10°

101

10²

103

104

Number of requests

Number ofrequests

Number of requests

Formulating the problem mathematically, we need to find such a solution

where z = [ z (0) , z (1) , . . . , z ( T )] is a state variable path. In numerical experiments, we consider the nonlinear equation function g ( z, x ) = z 3 -x , and since it is nonlinear, finding an optimal solution

The initial and the reference state are fixed at values z 0 = 0 . 8 , z ref = 0 . 7 . For a fixed initial value z 0 and fixed equation function g , the objective function G can be represented as a binary multidimensional tensor, whose elements are calculated using the following function: f ( x ) = G ( z ( x ) , x ) , hence we can apply discrete optimization methods to find x min , which approximates the optimal solution x ∗ .

We considered several values for variable T , such as 25 , 50 , and 100 (benchmarks P-15, P-16, and P-17, respectively). As follows from the results presented in Table 1, PROTES gives the most accurate solution in all three cases. Note that a result comparable in accuracy to our method is obtained only in one case when using baselines (i.e., P-16, BS-7).

6.4 Optimal control with constraints

In practical applications, some conditions or constraints may be imposed on the solution of the optimal control problem. In this work, we consider the following control constraint P : the control variable x ∈ { 0 , 1 } N can take the value '1' no less than 3 times in a row during the whole time interval . Formally, this can be written as follows:

To account for this condition in the PROTES algorithm, we constructively build the initial distribution in the form of an indicator tensor as described in Section 4 in the constrained optimization subsection. The details of this construction are presented in the Appendix. The numerical results for T = 25 (P-18), T = 50 (P-19), and T = 100 (P-20) are reported in Table 1. In two cases out of three (P-19, P-20), our method showed the best result, and in one case (P-18) slightly yielding to the TTOpt method (BS-1), which, however, in two other cases, gave a significantly worse result.

TT-rank (R)= 3

TT-rank (R) = 5

TT-rank (R) = 7

10

10

10

Batch size (K)= 50

Batch size (K)= 100

Batch size (K)= 150

Batch size (K)=200

102

10

-2

10

-2

10-3

5

10

15

20

25

10

E-

5

10

15

20

25

10

5

10

15

20

25

Top candidates (k)

Top candidates (k)

Top candidates (k)

6.5 Robustness and performance of the PROTES

The results in Table 1 relate to the 'intuitive' selection of the hyperparameters for the PROTES method (as was mentioned above, we have used the values: K = 100 , k = 10 , k gd = 1 , λ = 0 . 05 , and R = 5 ). In Figure 4, we present an analysis of the dependence of the optimization result for the benchmark P-14 on the choice of hyperparameters K , k , and R , with fixed k gd = 1 and λ = 0 . 05 . We report the relative error of the result for all combinations K = 50 , 100 , 150 , 200 , 250 ; k = 5 , 10 , 15 , 20 , 25 ; R = 3 , 5 , 7 . As we can see from the plots, the hyperparameters used in the main calculations are not optimal for this particular problem, that is, additional fine-tuning of the method for specific problems or classes of problems is possible. At the same time, according to the results in Figure 4, the method remains stable over a wide range of hyperparameter values. We also note that all computations were carried out on a regular laptop, while the operating time of the considered optimizers was commensurate; for example, for the benchmark P-17, the measured operating time (in seconds) is: PROTES (641), BS-1 (607), BS-2 (4245), BS-7 (780).

A more detailed analysis of the PROTES performance and the dependence of optimization results on the values of hyperparameters are presented in the Appendix. Also, in the Appendix, we consider another promising application of our optimizer for several popular reinforcement learning problems from Mujoco / OpenAI-GYM collection.

7 Conclusions

In this work, we presented an optimization algorithm PROTES based on sampling from the probability density defined in the tensor train format. We used the same set of hyperparameters for all considered numerical experiments, so our algorithm is rather universal To take into account the constraints, as in the problem of optimal control with constraints, we only considered them in the form of a specially selected initial approximation (a special form of an indicator tensor in the tensor train format); further on, the algorithm did not consider the constraints explicitly. This approach allows us to extend the algorithm's capabilities by using the properties of the tensor train representation. Numerical experiments show that we outperform many popular optimization methods.

The main direction in our future work is scaling the method to large dimensions. For d ≥ 1000 , we have encountered numerous technical difficulties, which can be alleviated by other tensor formats (such as hierarchical Tucker, which can be parallelized over d ) and more efficient implementations of the optimization method (now we use standard automatic differentiation without special tensor optimization methods such as Riemannian optimization).

References

Stephane Alarie et al. 'Two decades of blackbox optimization applications'. In: EURO Journal on Computational Optimization 9 (2021), p. 100011.

Boian Alexandrov et al. 'Challenging the curse of dimensionality in multidimensional numerical integration by using a low-rank tensor-train format'. In: Mathematics 11.3 (2023), p. 534.

Pauline Bennet et al. 'Nevergrad: black-box optimization platform'. In: SIGEVOlution 14.1 (2021), pp. 8-15.

Andrei Chertkov, Gleb Ryzhakov, and Ivan Oseledets. 'Black box approximation in the tensor train format initialized by ANOVA decomposition'. In: SIAM Journal on Scientific Computing 45.4 (2023), A2101-A2118.

Andrei Chertkov et al. 'Optimization of functions given in the tensor train format'. In: arXiv preprint arXiv:2209.14808 (2022).

Andrzej Cichocki et al. 'Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions'. In: Foundations and Trends in Machine Learning 9.4-5 (2016), pp. 249-429.

Andrzej Cichocki et al. 'Tensor networks for dimensionality reduction and large-scale optimization: Part 2 applications and future perspectives'. In: Foundations and Trends in Machine Learning 9.6 (2017), pp. 431-673.

Johannes Dieterich and Bernd Hartke. 'Empirical review of standard benchmark functions using evolutionary global optimization'. In: Applied Mathematics 3.10 (2012), pp. 15521564.

Benjamin Doerr and Frank Neumann. 'A survey on recent progress in the theory of evolutionary algorithms for discrete optimization'. In: ACM Transactions on Evolutionary Learning and Optimization 1.4 (2021), pp. 1-43.

Sergey Dolgov et al. 'Approximation and sampling of multivariate probability distributions in the tensor train decomposition'. In: Statistics and Computing 30 (2020), pp. 603-625.

Jian Dong, Zhiyu Wang, and Jinjun Mo. 'A phase angle-modulated bat algorithm with application to antenna topology optimization'. In: Applied Sciences 11.5 (2021), p. 2243.

Fred Glover et al. 'Quantum bridge analytics I: a tutorial on formulating and using QUBO models'. In: Annals of Operations Research (2022), pp. 1-43.

Lars Grasedyck, Daniel Kressner, and Christine Tobler. 'A literature survey of low-rank tensor approximation techniques'. In: GAMM-Mitteilungen 36.1 (2013), pp. 53-78.

John Hammersley. 'Monte Carlo methods for solving multivariable problems'. In: Annals of the New York Academy of Sciences 86.3 (1960), pp. 844-874.

Nikolaus Hansen. 'The CMA Evolution Strategy: A Comparing Review'. In: Towards a new evolutionary computation: advances in the estimation of distribution algorithms. Springer Berlin Heidelberg, 2006, pp. 75-102.

Momin Jamil and Xin-She Yang. 'A literature survey of benchmark functions for global optimization problems'. In: Journal of Mathematical Modelling and Numerical Optimisation 4.2 (2013), pp. 150-194.

James Kennedy and Russell Eberhart. 'Particle swarm optimization'. In: Proceedings of ICNN'95-international conference on neural networks. Vol. 4. IEEE. 1995, pp. 1942-1948.

Diederik P Kingma and Jimmy Ba. 'Adam: a method for stochastic optimization'. In: arXiv preprint arXiv:1412.6980 (2014).

Kirandeep Kour et al. 'Efficient structure-preserving support tensor train machine'. In: Journal of Machine Learning Research 24.4 (2023), pp. 1-22.

Caio Kalil Lauand and Sean P Meyn. 'Approaching quartic convergence rates for quasi-stochastic approximation with application to gradient-free optimization'. In: Advances in Neural Information Processing Systems. 2022.

Tianyi Lin, Zeyu Zheng, and Michael Jordan. 'Gradient-free methods for deterministic and stochastic nonsmooth nonconvex optimization'. In: Advances in Neural Information Processing Systems 35 (2022), pp. 26160-26175.

John Maryak and Daniel Chin. 'Global random optimization by simultaneous perturbation stochastic approximation'. In: Proceedings of the 2001 American Control Conference.(Cat. No. 01CH37148). Vol. 2. IEEE. 2001, pp. 756-762.

Yurii Nesterov and Vladimir Spokoiny. 'Random gradient-free minimization of convex functions'. In: Foundations of Computational Mathematics 17.2 (2017), pp. 527-566.

Artyom Nikitin et al. 'Are quantum computers practical yet? A case for feature selection in recommender systems using tensor networks'. In: arXiv preprint arXiv:2205.04490 (2022).

Georgii Novikov, Maxim Panov, and Ivan Oseledets. 'Tensor-train density estimation'. In: Uncertainty in artificial intelligence. PMLR. 2021, pp. 1321-1331.

Ivan Oseledets. 'Tensor-train decomposition'. In: SIAM Journal on Scientific Computing 33.5 (2011), pp. 2295-2317.

Ivan Oseledets and Eugene Tyrtyshnikov. 'Breaking the curse of dimensionality, or how to use SVD in many dimensions'. In: SIAM Journal on Scientific Computing 31.5 (2009), pp. 37443759.

Ivan Oseledets and Eugene Tyrtyshnikov. 'TT-cross approximation for multidimensional arrays'. In: Linear Algebra and its Applications 432.1 (2010), pp. 70-88.

Olivier Parcollet et al. 'Learning Feynman diagrams with tensor trains'. In: Bulletin of the American Physical Society (2023).

Gary Parker and Ronald Rardin. Discrete optimization. Elsevier, 2014.

Jun Qi et al. 'Exploiting low-rank tensor-train deep neural networks based on Riemannian gradient descent with illustrations of speech processing'. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing 31 (2023), pp. 633642.

Gleb Ryzhakov and Ivan Oseledets. 'Constructive TT-representation of the tensors given as index interaction functions with applications'. In: 11th International Conference on Learning Representations, ICLR. 2023.

Jonathan Scarlett, Ilija Bogunovic, and Volkan Cevher. 'Lower bounds on regret for noisy gaussian process bandit optimization'. In: Conference on Learning Theory. PMLR. 2017, pp. 1723-1742.

Cheryl Selvanayagam et al. 'Global optimization of surface warpage for inverse design of ultra-thin electronic packages using tensor train decomposition'. In: IEEE Access 10 (2022), pp. 48589-48602.

Suhan Shetty et al. 'Tensor train for global optimization problems in robotics'. In: arXiv preprint arXiv:2206.05077 (2022).

Micheline Soley et al. 'Global optimization with the iterative power algorithm via quantum computing and quantics tensor trains'. In: Bulletin of the American Physical Society (2023).

Konstantin Sozykin et al. 'TTOpt: a maximum volume quantized tensor train-based optimization and its application to reinforcement learning'. In: Advances in Neural Information Processing Systems. 2022.

Rainer Storn and Kenneth Price. 'Differential evolution - a simple and efficient heuristic for global optimization over continuous spaces'. In: Journal of Global Optimization 11.4 (1997), pp. 341-359.

Christoph Strössner, Bonan Sun, and Daniel Kressner. 'Approximation in the extended functional tensor train format'. In: arXiv preprint arXiv:2211.11338 (2022).

Minghua Wang et al. 'Tensor decompositions for hyperspectral data processing in remote sensing: a comprehensive review'. In: IEEE Geoscience and Remote Sensing Magazine (2023).

Xiaoxing Wang et al. 'ZARTS: on zero-order optimization for neural architecture search'. In: Advances in Neural Information Processing Systems 35 (2022), pp. 12868-12880.

Daan Wierstra et al. 'Natural evolution strategies'. In: Journal of Machine Learning Research 15.27 (2014), pp. 949-980.

Ronald Williams. 'Simple statistical gradient-following algorithms for connectionist reinforcement learning'. In: Machine learning 8.3-4 (1992), pp. 229-256.

Vitaly Zankin, Gleb Ryzhakov, and Ivan Oseledets. 'Gradient descent-based D-optimal design for the least-squares polynomial approximation'. In: arXiv preprint arXiv:1806.06631 (2018).

Supplementary Material

A1 Stability of the PROTES

To check the stability of the optimization result, a series of 10 calculations were performed for each method (PROTES, BS1 - BS7) with random initializations. We consider the binary knapsack problem from the work [11] (benchmark P-14, described in detail in the main text of the work) with the known exact minimum -3103 . For all methods, the limit on the number of requests was fixed at the value M = 10 5 . All other parameters were the same as in the computations from the main text, i.e., the PROTES parameters are K = 100 , k = 10 , k gd = 1 , λ = 0 . 05 , R = 5 . In Table 3 we present the average and best results over 10 runs for each optimization method. As follows from the reported results, only PROTES and Portfolio method (BS-7) managed to successfully find the exact optimum, while the average result for PROTES is significantly better than that of all the baselines.

A2 Reinforcement learning with PROTES

As in the work [37], we consider maximization of the cumulative reward of the episode for several popular reinforcement learning problems from Mujoco / OpenAI-GYM collection: InvertedPendulum (P-21), Swimmer (P-22), Lunar Lander (P-23), and Half Cheetah (P-24). The policy is represented by a neural network with three layers and tanh activations. Parameters of the neural network are discretized on a grid with limits from -1 to +1 and 256 nodes (i.e., mode size N ). The total number of neural network parameters for the problems under consideration turns out to be from 50 to 68 (i.e., dimension d ). Thus, as in the work [37], we obtain a discrete on-policy learning problem (search for the values of the neural network parameters that lead to the maximum reward). To speed up the calculations, we took only two baselines (BS1, and BS7) for comparison, which showed the best results in the main experiments presented in the paper. We used the same optimizer settings as in the main calculations described in the text of the paper; however, in this case, we chose the budget value equal to M = 10 5 , as in [37]. For PROTES, BS1, and BS7, we repeated the calculations three times and reported the average result for found maximum and computation time in Table 4. PROTES shows the best result in terms of accuracy in 3 cases out of 4 , and once it is in second place, slightly inferior to the BS7.

Table 4: Maximization results for InvertedPendulum (P-21), Swimmer (P-22), Lunar Lander (P-23) and Half Cheetah (P-24) benchmarks from Mujoco collection. We report the reward and calculation time (seconds), averaged over 3 runs, for the proposed method PROTES and baselines BS1 and BS7.

A3 Choice of hyperparameters for the PROTES

We conduct additional experiments with varying the values of the hyperparameters of the PROTES method for benchmarks P-01 (Figure 5 and Figure 6; we report the optimization result) and P-14 (Figure 4 in the main text and Figure 7 below; we report the relative error of the optimization result). In the first series of experiments (Figure 5 below and Figure 4 in the main text), we fixed the values k gd = 1 and λ = 0 . 05 and tried all combinations of the remaining hyperparameters:

TT-rank (R)= 3

TT-rank (R)= 5

TT-rank (R) = 7

×10-5+ 8.306

x10-5+ 8.306

×10-5 + 8.306

8.0

8.0-

8.0

7.8

7.8

7.8

7.6

7.6

7.6

7.4

Batch size (K)= 50

7.4

7.4

Batch size (K) = 100

Batch size (K) = 150

7.2

7.2

7.2

Batch size (K) = 200

Batch size (K) = 250

7.0

7.0

7.0

5

10

15

20

25

5

10

15

20

25

5

10

15

20

25

Top candidates (k)

Top candidates (k)

Top candidates (k)

TT-rank (R) = 3

TT-rank (R) = 5

TT-rank (R) = 7

15

15

15

LR =5.0e-03

14

LR = 1.0e-02

14

14

LR = 5.0e-02

13

LR = 1.0e-01

13

13

LR =5.0e-01

12

12

12

11

11

11

10

10

10

9

9

9

8

110

25

50

100

110

25

50

100

110

25

50

100

Gradient steps (k-gd)

Gradient steps (k-gd)

Gradient steps (k-gd)

TT-rank (R) = 3

TT-rank (R) = 5

TT-rank (R) = 7

10

10

10

102

10

2

10~

LR =5.0e-03

LR = 1.0e-02

LR = 5.0e-02

LR = 1.0e-01

LR = 5.0e-01

10

11025

50

100

10

E-

11025

50

100

11025

50

100

Gradient steps (k_gd)

Gradient steps (k_gd)

Gradient steps (k_gd)

K = 50 , 100 , 150 , 200 , 250 ; k = 5 , 10 , 15 , 20 , 25 ; R = 3 , 5 , 7 . In the second series of experiments (Figure 6 and 7), we fixed the values K = 100 and k = 10 , and tried all combinations: k gd = 1 , 10 , 25 , 50 , 100 , λ = 0 . 005 , 0 . 01 , 0 . 05 , 0 . 1 , 0 . 5 ; R = 3 , 5 , 7 .

As follows from the presented results, for problem P-01, the dependence on the choice of hyperparameters turns out to be extremely weak, except for outliers for the small values of the learning rate λ = 0 . 005 and λ = 0 . 01 . For more complex problem P-14, the dependence of the result on the

choice of hyperparameters is more complex, but it can be seen that the method remains stable over a wide range of hyperparameter values.

A4 Performance comparison of the optimization methods

All the results described in the main text and presented there in Table 1 were obtained on a regular laptop. In Table 5 we report the related computation time for each method (PROTES, BS1 - BS7) and each model problem (P01 - P20). As follows from the results, the PROTES works much faster than classical optimization methods (BS3 - BS7), as well as faster than tensor-based methods (BS1 and BS2) for several optimal control problems (P-15 - P-20). However, the TTOpt (BS-1) and Optima-TT (BS-2) methods are faster for simpler analytic (P-01 - P-10) and QUBO (P-11 - P-14) problems. This is due to the fact that within the framework of the dynamic TT-rank refinement procedure in these methods, the TT-rank, and hence the computation time, turn out to be significantly higher for the optimal control problems. We note that the very short running time of the TTOpt method for P-20 is because most of the requests of the method did not satisfy the imposed constraints, and in this case, the differential equation was not solved.

A5 Derivative functions in the optimal control problem

We use the following derivative functions for the constructive building of the tensor described in [32]

for all TT-cores except the last one ( k = 1 , 2 , . . . , d -1 ), and for the last TT-core

A tensor in the TT-format built on such derivative functions is equal to 0 if there are less than l ones among its vector argument in a row, and is equal to 1 in all other cases.

Let us briefly explain why such derivative functions give a tensor of the restriction condition (i.e., the constraint in the considered optimal control problem). Recall that the upper index in the derivative function notation corresponds to the index number of the tensor argument, and the lower index corresponds to the value of this argument index. In our scheme, the argument of the derivative functions has the meaning of the number of ones that already stand to the left of the given index.

First, let us focus on the functions for all indices except the rightmost one ( k = 1 , 2 , . . . , d -1 ). If the current index is one, then we simply increase the value of the argument by 1 and pass it on, to the input of the next derivative function. This is what the function f k 1 does (we take the maximum, so if the number of ones has already reached l , we do not care if it is greater than or equal to l ; note that the maximum operation reduces the TT-rank but does not affect the result). For the zero value of the current index, if the argument is zero (which means that the previous index is also zero), the function f k 0 also returns zero as nothing have changed. If the argument of this function is l , it means that there are l or more ones in a row to the left of the considered index, so the given condition is not violated, and the function simply returns zero, which means that there are no ones. If the argument is greater than zero but less than l , it means that the condition is violated because the previous sequence of ones of length less than l is cut off at the current index. In this case, the function f k 0 returns None , which means that the value of the tensor will be 0 regardless of the subsequent indices.

The functions corresponding to the last ( k = d ) index behave similarly. Namely, if the last index is zero, the function f k 0 returns 1 if its argument is 0 or l since it does not violate the condition as just explained. Otherwise, it returns 0 . If the last index is 1 , the function f k 1 returns 1 only if its argument is equal to ( l -1) or l since this means that the current one is an element of a sequence of ones of length at least l . Otherwise, the function f k 1 returns 0 .

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present the algorithm PROTES for solving discrete optimization problems. The method uses probabilistic sampling to obtain a probability function of minimal (or maximal) values of a target function. This probability function takes the same shape as the target tensor. Alternatively, this could be described as a likelihood score for the optimal values of the discrete variables. To make the algorithm more efficient, the authors use the tensor train (TT)-representation for tensors. This is a low-rank approximation of a tensor, which makes the iterative computations faster. This TT-representation is used for the probability function.\n\nThe authors test their method on 20 different problem instances and compare their method to seven other methods. The results show that PROTES works well, and is stable for the instances it was tested on.\n\nSTRENGTHS:\nThe paper has a clear structure and is well written.\n\nWEAKNESSES:\nThe method was not well motivated and the update rule was unclear to the reader. This makes is it very hard to assess whether there is sufficient contribution to the research area.\n\nThe comment on the application of the method on constrained optimization is too short-sighted. In other words, such an important topic deserves more explanation. The current version does not clearly explain how general constrained optimization problems can be handled, particularly the feasibility concerns.\n\nThe instance sizes of the QUBO problems are very small compared to what is now possible with the constrained version of these problems.\n\nAlthough the method was tested on 20 different problems, compared to seven benchmarks, only one instance per problem (size) was tested. For instance, for P-11 to P-13, it looks like only one instance was selected, which was not specified whether random problem parameters or something else was used as input. Therefore, the reader cannot tell if the proposed method works well in general for the tested problems or only on the tested instance.\n\nQUESTIONS:\n- What is the reason for using gradient ascent for updating the the probability distribution? Cannot alternatively a cheaper updating rule be used? Think for instance of increasing the likelihood of the selected candidates by some value in each iteration (similar to weighing).\n\n- In line 62 of the paper, it is mentioned that the probability distribution is expected to present a Kronecker delta-function. It sounds more like an indicator function with a peak in the value of the minimum target function. Moreover, why is this function expected?\n\n- Please address why only one instance was tested per problem (size), and why this is enough evidence of the efficacy of the method for the type of problem.\n\n- In line 182, it is mentioned that related methods in the TT-format should be used for approximation. Is the approximation of the optimum meant here? It is a bit confusing since in the sentence before the (TT-)approximation of the original tensor is mentioned.\n\n- What are the specifications (CPU etc) of the ‘regular laptop’?\n\nLIMITATIONS:\nNot applicabel.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-05"}

comments: SUMMARY:
This paper presents a study on black-box optimization. The authors propose a new method based on sampling from the probability density defined in the tensor train (TT) format (extension effort). The experiments have shown that PROTES outperforms existing efforts.

STRENGTHS:
(Overall) This paper is well organized. The technical soundness and practical performance are strong. 

1. This paper has given a detailed motivation and an analysis of the computational complexity of the method. This makes the work appear comprehensive.

2. The research on gradient-free optimization plays a crucial role in various fields. Despite its significance, it often receives relatively low attention compared to gradient-based optimization methods. This effort tries to add distribution sampling for new tech, "TT-tensor."

WEAKNESSES:
1. The method section "Optimization method PROTES" is hard to follow and what's the relationship between this section and others?

2. The experiment (P1-P20) seems too simple to fully demonstrate the method's effectiveness.

3. This paper also uses the gradient ascent for the TT-tensor, so there lacks more discussion about the model design and gradient-free optimization.

QUESTIONS:
1. Does dimension d have an impact on the experiment? Why was it fixed at 50?

LIMITATIONS:
No negative societal impact.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-07

comments: SUMMARY:
The paper proposes PROTES, a new method for multi-dimensinoal discrete black-box optimization based on adaptive probablistic sampling in the tensor train representation.

STRENGTHS:
- The technical method is simple and intuitive, with a clear presentation.
- Diverse benchmark problems and comprehensive comparisons to relevant SOTA baselines.
- Good empirical performance under the tested experimental setup.

WEAKNESSES:
- Besides presenting and describing the results, lack of intuition/discussion on why the proposed approaches outperform baselines (both classical and tensor-based). 
- It’s encouraged to have a wider spectrum of dimensionalities of the test problems besides 7 and 50 used.
- It's not the most efficient method for any given number of requests (evaluation budgets), as seen in Fig 3. The method gains more advantage as requests become closer to 10^4.

QUESTIONS:
N/A

LIMITATIONS:
The authors discussed the limitations of the paper which include the inability to handle large dimensions (>1000) and explicit constraint consideration.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-09

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel algorithm to estimate global extrema of discrete black-box target functions. The approach proposes learning a probability distribution, represented as a multidimensional tensor and efficiently stored using a TT-decomposition, that is trained to favor multi-indices that achieve high/low values in the target function. The approach, PROTES, draws inspiration from the REINFORCE algorithm, and can be trained using gradient-based optimization techniques. Extensive experiments are conducted to demonstrate the competitive performance and robustness of PROTES.\n\nSTRENGTHS:\n- The paper is quite self-contained and well written, concepts are introduced gradually and are easy to understand\n- The connection to REINFORCE is informative and helpful\n- Experiments are compelling, detailed ablation analyses are included demonstrating the robustness of PROTE\n\nWEAKNESSES:\nClarity/Notation:\n- \"Gradient-free\" is defined very late in the paper but is used frequently throughout the earlier sections. This is confusing since the optimization of the probability tensor is done with gradient based optimization. I would suggest providing a clear description of this in the introduction.\n- It is stated in the footnote on page 1 that upper case letters are used to denote matrices, but they are used several times in the paper to refer to scalars i.e. $N_i$, $R_i$, etc.\n- In figure 2, the tensor network diagram needs further explanation. It is not clear what is being conveyed from the figure and the current caption.\n\nComparison to Existing Approaches:\n- While the experiments empirically demonstrate the efficacy of the proposed approach, the computation complexities of prior works is not stated anywhere in the paper. Is it the case that the performance benefits are attained by adding significant computational costs? This should be discussed in the comparison.\n- Since the proposed approach is probabilistic, it would be useful to include error bars in the experimental section.\n\nQUESTIONS:\n- What are the computational complexities of the baselines?\n- Does this method work especially well when there is a particular structure in the target function? Clearly this method, and any of the baselines would fail if $\\mathcal{Y}$ is random.\n- How is the probability tensor parametrized? Are the values of the tensor learnt directly? Could there be a benefit to alternative parametrizations (i.e. having a multilayer structure)?\n\nLIMITATIONS:\nDiscussed in detail", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper targets the challenging gradient-free discrete multidimensional optimization. It is formalized as finding a vector of d indices whose associated entry in a d-dimensional tensor is minimized, where the tensor contains all the possible values for the black-box objective function. There are an exponential number of entries in the tensor for brute-force search. Instead of enumerating all of them, the paper proposes to optimize a low-rank tensor whose each entry represents the probability of selecting the corresponding entry as the minimum in the objective value tensor. The algorithm starts from a random probability tensor and updates it using a similar trick as REINFORCE algorithm. In experiments on benchmark tasks, it performs comparably or better than the baselines evaluated.\n\nSTRENGTHS:\n1. It targets an important and challenging problem interesting to many practical problems.\n2. The idea of using a low-rank tensor can reduce the cost of brute-force search. \n3. Evaluation of different tasks and robustness to the hyperparameters.\n\nWEAKNESSES:\n1. It is a combinatorial search problem from an exponential number of candidates so some structural information of the problem has to be used. However, it is not clear why and when the low-rank probability tensor assumption holds, and if so, for which classes of problems.\n2. The technical contribution is ad-hoc to some extent by combining tensor-train decomposition with REINFORCE trick.\n3. I like the explanation of the intuition starting at line 122 but the approximation in Eq. (5) does not hold rigorously without using the true Monte-Carlo samples as in REINFORCE (the proposed alg instead uses the best k samples)\n4. The complexity analysis assumes a fixed number of queries to the black-box function and does not study the number of iterations required to reach a sufficiently good local minimum. \n5. The proposed method does not show advantages to BS-1 and BS-2 in Table 1 on analytic functions. Figure 3 does not show a clear advantage in the sample efficiency.\n\nQUESTIONS:\n1. Can you elaborate on when and why the combinatorial optimization problem's structure is consistent with the low-rank probability assumption? Some concrete examples can be helpful.\n2. How to balance the exploration and exploitation in the black-box optimization problem? It seems like the proposed algorithm mainly focuses on exploitation (always selecting the top-k candidates).\n\nLIMITATIONS:\nThe author did not discuss the potential limitations. An efficient solver for combinatorial black-box optimization can be used for attacks such as threats to cybersecurity. It would be nice to have some discussions on that.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-08-01"}