Paper: ProGO: Probabilistic Global Optimizer (ICLR-2024)

Reviewer comment (Evaluation):
it is not clear if the methods are using the same number of objective function evaluations. If I understand Algorithm 1 correctly, the objective function is evaluated in line 5, in the condition of the while loop, hence the while loops and thus the number of objective function calls is not deterministic for each iteration of the algorithm. Comparing algorithms by number of iterations when algorithms call the objective a different number of times each is not a fair comparison.

Author response:
We acknowledge that the function evaluations are not once for each iteration of ProGO, and the count of function evaluations per iteration may differ among various algorithms due to variations in their optimization strategies as well. Hence, we employ runtime as a standardized metric to assess and compare different methods impartially.

Reviewer comment (Novelty):
in my view the framework of ProGO is not new. The use of this particular MCMC Latent Slice Sampler within an simulated annealing algorithm may be new. If so, Standard Metropolis Hasting with a Gaussian proposal and other MCMC methods should be baselines. Even so, a change of MCMC sampler is not sufficient for publication.

Author response:
We respectfully disagree with the view. Yes, there are similarities between our approach and simulated annealing in the sense that they both use sampling methods. Simulated annealing uses Metropolis-Hastings (MH) random walk while we utilize the Latent Slice Sampler (LSS), which is an alternative better than MH. However, it is well known in the literature, Random Walk-Metropolis Hastings (RW-MH) methods, particularly those based on Gaussian perturbations, are known to have slower rates of convergence compared to slice samplers, which enjoy a geometric rate of convergence. 

To the best of our knowledge, the theoretical convergence results for target functions, which are assumed only to be continuous (without any smoothness conditions) and possibly non-convex, have not been established when the domain is unbounded. Furthermore, our theoretical extension from compact to non-compact sets is nontrivial given the relaxation of conditions. This is our first novelty.  

Second, to the best of our knowledge, the use of LSS for optimization has not been available in the existing literature. 

We have not come across any relevant references, and we would be grateful if the reviewer could provide any if applicable.

Reviewer comment (Evaluation):
do all algorithms evaluate $f(x)$ exactly the same number of times per iteration? the results should plot total numebr of function $f(x)$ calls on the x axis.

Author response:
We acknowledge that the function evaluations are not once for each iteration of ProGO, and the count of function evaluations per iteration may differ among various algorithms due to variations in their optimization strategies as well. Hence, we employ runtime as a standardized metric to assess and compare different methods impartially.

Reviewer comment (Theory):
The authors only provides convergence property when the iteration $k$ goes to infinity. It seems promising that the new method out-performs classic algorithms on some functions, but what about the worst-case upper bound? It would be more interesting (and practical) if we have some non-asymptotic results like how many iterations we need to approximate an global minimizer within error $\epsilon$. If, in the worst case, the algorithm needs $(1/\epsilon)^d$ calls of function value oracle to find a global min of a (say, lipschitz) function, then it is no better than a brute force search.

Author response:
Our method of approximating the minima employs stochastic latent slice sampling from the minima distribution $m_k(x)$. While obtaining non-asymptotic bounds is a challenging task we aim to tackle in future research, it's important to highlight that our method fundamentally differs from traditional sampling methods, utilizing the theoretical property of the minima distribution and the practical efficiency of the LSS technique. In contrast, brute force search explores the entire space and is unsuitable for high dimensions. Thus, our algorithm's robust performance in low- and high-dimensional cases demonstrates its superiority over brute-force search.

We appreciate the insights to conduct further explorations to validate our algorithm's performance for different scenario. In response, we have included additional examples such as convex optimization in Appendix B and plan to explore other applications in our future research.

Reviewer comment (Theory):
How many samples does the LSS-ProGo algorithm need to approximate the distribution $m_k(x)$? Especially, in high-dimension scenerios? How accurate this approximation needs to be?

Author response:
The required Monte Carlo (MC) sample size to achieve a desired level of precision in approximating the minimum distribution from MC samples can be grounded in fundamental statistical principles, including Central Limit Theory (CLT), Strong Law of Large Numbers (SLLN), Strong Law of Iterated Logarithm (SLIL), and Hoeffding's Inequality (e.g., [1], [2], [3]). For example, when adopting a CLT-based approach to ensure an error no greater than $\epsilon$, it is essential to have a sample size of at least $N \geq 4 \hat{\sigma}^2$, where $\hat{\sigma}^2 = \frac{1}{N-1} \sum_{i=1}^N(f(x_i)-f(x_i^*))^2$. It's noteworthy that the stochastic method's MC sample size is dimensional-invariant, while deterministic methods may experience the curse of dimensionality.

[1] Caflisch, Russel E. "Monte carlo and quasi-monte carlo methods." Acta numerica 7 (1998): 1-49.
[2] Zaremba, Stanislaw K. "The mathematical basis of Monte Carlo and quasi-Monte Carlo methods." SIAM review 10.3 (1968): 303-314.
[3] Reich, Brian J., and Sujit K. Ghosh. Bayesian statistical methods. CRC Press, 2019.

Reviewer comment (Experiments):
I am wondering what would happen if we use Gaussian as $\pi(x)$ instead of uniform distribution. Will that bring us better results?

Author response:
Thank you for this insightful suggestion. Using a Gaussian distribution as an alternative to a uniform one introduces additional prior information, involving the specification of the mean and covariance. If the Gaussian prior's mean closely aligns with the global optimum and the covariance is appropriately set, it has the potential to expedite convergence. In response, we appreciate the idea of exploring Gaussian priors in our framework. We will consider conducting further experiments in future research to assess the impact of Gaussian priors on ProGO's performance.

Reviewer comment (Theory):
In order to apply the proposed algorithm, the optimal function value $f^*$ is supposed to be known. The assumption is not true for many practical applications.

Author response:
We do not need the knowledge of optimal function value $f^*$, and we have demonstrated this in Section 3 (also see the Remark 1 that we included right after defining the nascent minima distribution). We used $f^*$ only for the ease of theoretical proofs, but as we pointed out in our remark, the $exp(-f^*)$ cancels out in the numerator and denomination (and so in practice we do not need to know the $f*$. The requirement for all inputs is listed in “Input” of Algorithm 2. We only use $f^*$ when evaluating experimental performance on synthetic functions in Section 4. We will make it clearer for the modified version.

Reviewer comment (Experiments):
The algorithm is only tested on two test instances, which can be efficiently solved by a number of existing derivative-free optimization algorithms. More experiments are expected to convince the performance benefit of the algorithm, for example, see “N. Hansen, A. Auger, R. Ros, O. Mersmann, T. Tušar, D. Brockhoff. COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting, Optimization Methods and Software, 36(1), pp. 114-144, 2021”.

Author response:
We acknowledge the importance of conducting extensive experiments to validate its performance advantages as recommended. We have incorporated additional examples in Appendix B and intend to explore further applications in future research.

Additionally, our initial algorithm testing, while limited to two instances, is notable for its high-dimensional setting, spanning 1000 dimensions. This departure from the typical 100 dimensions in other studies is worth emphasizing.

Reviewer comment (Experiments):
The performance of the algorithm heavily depends on the efficiency of the latent slice sampler applied to the nascent minima distribution $m_k$.

Author response:
We appreciate the reviewer's insights into the significant role of the latent slice sampler (LSS). LSS is a part of the slice samplers (SS) family, which is well-known for its robust convergence efficiency: [1] demonstrated their nearly geometric ergodic behavior, and [2] outlined conditions for SS to achieve uniform ergodicity. Extending the SS framework, [3] describes the detailed rationale and practical convergence efficiency of LSS.

[1] Roberts, Gareth O., and Jeffrey S. Rosenthal. "Convergence of slice sampler Markov chains." Journal of the Royal Statistical Society Series B: Statistical Methodology 61.3 (1999): 643-660.
[2] Mira, Antonietta, and Luke Tierney. "Efficiency and convergence properties of slice samplers." Scandinavian Journal of Statistics 29.1 (2002): 1-12.
[3] Li, Yanxin, and Stephen G. Walker. "A latent slice sampling algorithm." Computational Statistics & Data Analysis 179 (2023): 107652.

Reviewer comment (Reproducibility):
How to calculate the denominator of Equations (2) and (3) for a general black-box $f(x)$?

Author response:
We want to emphasize that there's no need to compute the denominator in the equation (2). This is because $m_k(x) \propto {e^{-kf(x)} \cdot \pi(x) }$, and the integral serves as a constant. As it’s well-known within the MCMC literature, the normalizing constant of the density is not required to generate samples from the path of the Markov chain (by using the LSS). To clarify, for any black-box target function, all that are required are MCMC samples generated from the kennel of the density  ${e^{-kf(x)} \cdot \pi(x) }$ to approximate the minima distribution in equation (2). Hence, there’s no need to compute the denominator to implement the LSS.

Reviewer comment (Reproducibility):
In Algorithm 2, why we don’t use only a single large value for $k$? In Line 3, is $x^{(0)}$ the same for every t-th iteration? I don’t see any interaction between iterations; that is, the information of $x^{(1)}, …, x^{(N)}$ is not reused in next steps.

Author response:
In Algorithm 2, employing a single large k within the exponential can result in numerical instability in the sampling algorithm. Our goal is to find k sufficiently large enough that makes LSS numerically stable until the convergence of the minima is numerically achieved. On the other hand, the latent slice sampler demonstrates a geometric rate of convergence even when with smaller values of $k$. Despite no direct interaction between iterations, the algorithm can achieve the desired results effectively. This justifies our approach of conducting a sequential search for the optimal k, considering both practical and theoretical considerations.

Reviewer comment (Experiments):
What is the impact of selecting $k$ in Eq. (3) on the solution quality? What should be the appropriate value for $k$?

Author response:
To illustrate the effect of k, we provided a one-dimensional illustration in Figure 2, which visualized the impact of increasing k from 0 to 1, 3, and 9. As k increases, the density of $m_k(x)$ converges towards the global optima of $f(x)$, resulting in more accurate sampling around the global optima.

Determining the appropriate value of k depends on the specific characteristics of the target function, including its value and dimension. Hence, throughout our sequential search, we explored a wide range of k values, covering from $k=5$ to $k=5e^{200}$. As our method is shown to converge theoretically to the minima, our goal here is to approximate the minima by slowly increasing the k (maintaining the stability of the LSS) until a numerical convergence is observed.

Reviewer comment (Experiments):
In Algorithm 2, why do we need to fix $T = 200$?

Author response:
We chose the value 200 because we found in preliminary results that our method's regrets had already reached an extremely small value of $1\times10^{-17}$, far less than the threshold that can be accurately represented by floating-point numbers. Admittedly, the chosen value is a bit ad hoc, but this choice is purely driven by our experiments.

Reviewer comment (Theory):
Although the algorithm is based on the convergence of m_k, the rationale behind the proposed sampling procedure for m_k is not clear.

Author response:
>W1-1: “The rationale behind the proposed sampling procedure for $m_k$ is not clear.”

The rationale for our sampling procedure is based on the fact that the mode (and also the mean) of the density $m_k(x)$ converges to the minima when it is unique. Consequently, by sampling random variates from this density, we estimate the mode and the mean using Monte Carlo methods, as exact analytical calculations are often impractical. A better understanding of this rationale might be gained by referring to our one dimensional Illustration example presented in Figure 2 within Section 3.1.

In addition, following up with your question, we have made an attempt to make our rationale clearer at the beginning of Section 3. Thank you for your feedback.

Reviewer comment (Experiments):
Although theoretical result is nice, it does not explain why this algorithm performs better than alternative approaches. As a result, two numerical experiments might not be sufficient to support the paper's claim.

Author response:
> W2: “It does not explain why this algorithm performs better than alternative approaches”

There are very few algorithms that are comparable to our proposed method when the target function is assumed to be a continuous function (e.g., our method doesn’t make use of gradient vectors). Besides, our method provides a theoretical convergence guarantee for non-convex functions with a global optima.  It's important to note that our approach requires only the continuity of the target function f(x), and the results are proved essentially under very mild conditions, avoiding the need for any gradient or smoothness conditions (e.g., Lipstiz, etc.), which are typically prerequisites in most well-known methods. Also, most existing methods are based on hot (random) starting values, but our method doesn't require that, which is one of the novelties of our proposed method compared to other existing optimization methods. In essence, most other existing methods are not strictly comparable for the above reasons.

Reviewer comment (Theory):
The paper misses discussing model-based optimization algorithms as in [1]. Similar theoretical convergence results exist in this and other papers. For example, see Thm. 3 in [2]. The convergence holds under weaker assumptions for the objective function: lower-semicontinuous instead of continuous; and the strong separable condition seems to hold (see the discussion in Appendix C of [2]).

Author response:
> W3: “The paper misses discussing model-based optimization algorithms as in [1]. Similar theoretical convergence results exists in this and other papers. For example, see Thm. 3 in [2]. The convergence holds under weaker assumptions for the objective function: lower-semicontinuous instead of continuous; and the strong separable condition seems to hold (see the discussion in Appendix C of [2]).”

Our approach stands out significantly different from model-based methodologies. Model-based methods typically provide approximations of optimality through a two-step process: starting with an initial modeling assumption that approximates the target function and subsequently deriving estimators based on the assumed model. Our proposed method requires no such sugrrorgate function to approximate the target function but rather is based on the given target function.

In reference to [2], it explores the Bayes convergence of a model-based solutions, particularly in the context of ODE-PDE problems, which exhibit closed-form solutions akin to the minima distribution. However, the proof of Theorem 3 [2] in Appendix C relies on the assumption of second continuous differentiability on a compact set. In contrast, our method operates under a more relaxed continuity assumption on a non-compact set, without the need for gradient information, showcasing a novel approach in comparison.

[1] Hu, Jiaqiao, et al. "A survey of some model-based methods for global optimization." Optimization, Control, and Applications of Stochastic Systems (2012): 157-179.

[2] Zhang, Chi, Amirhossein Taghvaei, and Prashant G. Mehta. "A mean-field optimal control formulation for global optimization." IEEE Transactions on Automatic Control 64.1 (2018): 282-289.