Paper: Density Ratio Estimation-based Bayesian Optimization with Semi-Supervised Learning (ICLR-2024)

Reviewer comment (Writing):
The introduction is poor. It does not introduce properly the problem addressed in the paper.

Author response:
We sincerely appreciate your constructive comment to improve our work.

> The introduction is poor. It does not introduce properly the problem addressed in the paper.

> In general the writing of the paper has to be improved a lot. It does not introduce the concept of DRE-based BO right.

We will revise our paper more carefully in the final version.

Reviewer comment (Writing):
In general the writing of the paper has to be improved a lot. It does not introduce the concept of DRE-based BO right. If the reader is not familiar with it, they cannot understand it properly. The authors have failed in this task.

Author response:
We sincerely appreciate your constructive comment to improve our work.

> The introduction is poor. It does not introduce properly the problem addressed in the paper.

> In general the writing of the paper has to be improved a lot. It does not introduce the concept of DRE-based BO right.

We will revise our paper more carefully in the final version.

Reviewer comment (Theory):
The authors have to better explain DRE-based BO, why does it work, why is it interesting and why it is better than regression based BO.

Author response:
> The authors have to better explain DRE-based BO, why does it work, why is it interesting and why it is better than regression based BO.

We have included why DRE-based Bayesian optimization works in **Section 3**. Moreover, compared to GP-based Bayesian optimization, surrogate model learning is more efficient. In particular, while GP scales cubically with the number of points, learning a classifier is likely to be faster. Moreover, as shown in the experiment **Minimum Multi-Digit MNIST Search** of **Section 5.2**, the use of classifiers helps learn the representation of high-dimensional data.

Reviewer comment (Presentation):
Figure 1 is not explained properly.

Author response:
> Figure 1 is not explained properly.

> Please, explain better figure 1 and why it illustrates the over-confidence problem.

To accommodate your comment, we have updated **Section 1.1**. Please see **Section 1.1**.

Reviewer comment (Novelty):
The proposed method is not very well motivated. It seems it is simply using a better classifier. The authors claim that they use semi-supervised learning techniques to train the classifier. However, the semi-supervised data seems to be generated by sampling from a truncated Gaussian and then label propagation is used to generate the associated labels. Therefore, the proposed method can be understood simply as using a better classifier. Given this, I do not find that much novelty in the proposed method.

Author response:
> The proposed method is not very well motivated. It seems it is simply using a better classifier. The authors claim that they use semi-supervised learning techniques to train the classifier.

We do not agree with this comment. Our method offers a way to employ unlabeled points into the procedure of Bayesian optimization. In particular, in the scenario of pool-based Bayesian optimization, the vanilla Bayesian optimization does not utilize unlabeled points in a pool. Our method with semi-supervised learning enables us to utilize them in Bayesian optimization. Therefore, our contributions are more than simply using a better classifier.

Reviewer comment (Presentation):
Please, explain better figure 1 and why it illustrates the over-confidence problem.

Author response:
> Figure 1 is not explained properly.

> Please, explain better figure 1 and why it illustrates the over-confidence problem.

To accommodate your comment, we have updated **Section 1.1**. Please see **Section 1.1**.

Reviewer comment (Experiments):
Please, explain how the threshold value y_t is chosen.

Author response:
> It is not clear what is the motivation for DRE-based BO. It is also unclear how the threshold value $y^t$ is chosen.

> Please, explain how the threshold value $y_t$ is chosen.

We followed the previous work (Tiao et al., 2021; Song et al., 2022). In order to focus on the use of unlabeled points in the process of DRE-based Bayesian optimization, the selection of $y^t$ is built on the previous literature. Please refer to the prior work.

Reviewer comment (Evaluation):
The efficacy of the proposed method in the performance of optimization over iterations is hardly distinguishable in Figs. 5 or 6. While a clear victory is uncommon in comparisons of black-box optimization methods, the bigger problem is that it is unclear from the text under what conditions and why the proposed method outperforms existing approaches.

Author response:
We disagree with this comment. Our methods quickly converge to solutions compared to other baseline methods. In particular, **Figure 6** shows the superior performance of our methods.

Reviewer comment (Experiments):
Fixed-size pool — Does the *fixed-size pool* scenario mean some finite number of candidate points ${\bf x}_i$ are selected in advance and then optimized over these points? If true, is the set of points dynamically updated or fixed in advance?

Author response:
Yes, it is optimized over a fixed number of points. The set of points is not dynamically updated by following the general formulation of pool-based optimization.

Reviewer comment (Evaluation):
Definition of simple regret — What is the definition of simple regret? Is it $f({\bf x}_n) - f^*$ in the $n$th iteration, where $f^*$ is the minimizer of function $f$? Why is the regret monotonically decreasing in Fig. 2? Does this plot $\min_n f({\bf x}_n) - f^*$?

Author response:
The definition of simple regret is $\min\_{n} f(\mathbf{x}\_n) - f^*$, which is widely used in comparing Bayesian optimization algorithms. We have included this definition; please see **Equation (12)**.

Reviewer comment (Experiments):
Dimensionality of ${\bf x}$ in four benchmark tasks in Fig. 2 — Are they all two? This is crucial information on the difficulty of black-box optimization problems.

Author response:
Yes, they are two-dimensional problems. We have added the specifics of the synthetic benchmarks; see **Section D.2**.

Reviewer comment (Reproducibility):
Hyperparameter optimization of Fig. 6 — How do you define the distance $\|{\bf x}_i - {\bf x}_j \|$, especially when categorical values are involved such as activation function and learning rate schedule.

Author response:
To handle categorical and discrete variables, we treat them as integer variables by following the previous literature (Garrido-Merchan & Hernandez-Lobato, 2020). After converting them to integer variables, the distance between two integer variables is easily defined. We have updated **Section D.3** accordingly.

Reviewer comment (Presentation):
The presentation could be improved. Specifically, the focus on over-confidence in the beginning is confusing and the main point is only clear after reading section 3.1 where the relation to exploitation is mentioned. Perhaps this should be moved towards the front.

Author response:
Thank you for pointing this out. We moved Section 3.1 towards the front.

Reviewer comment (Theory):
Some limitations of the work should be explicitly mentioned/discussed. For example, assumption 4.1 seems to imply that the method is only intended to work on smooth functions.

Author response:
We have described the limitations and societal impacts in **Sections H and I**. Moreover, we think that the limitation on the smoothness of an objective, which is mentioned by the reviewer, is also related to kernel selection. In Bayesian optimization common stationary kernels work well when an objective is smooth. Therefore, Assumption 4.1 does not significantly add the additional limitation of Bayesian optimization.

Reviewer comment (Theory):
One doubt I have with this paper perhaps stems from unfamiliarity with semi-supervised algorithms in general. My understanding of such approaches is that they tend to assume that the unlabelled training points are nevertheless generated from the underlying x distribution.

Author response:
We sincerely appreciate your constructive comment to improve our work.

> One doubt I have with this paper perhaps stems from unfamiliarity with semi-supervised algorithms in general. My understanding of such approaches is that they tend to assume that the unlabelled training points are nevertheless generated from the underlying x distribution. In most applications of BO, however, this concept is nonsensical: the only x distribution is the points sampled by BO, which are (in some sense) arbitrary (depending on the acquisition function they may cluster around the optimum as time passes, but not necessarily). Am I missing a key point here?

We have solved two scenarios, pool-based Bayesian optimization and generic Bayesian optimization (defined on a continuous search space). Notably, the formulation of pool-based Bayesian optimization has access to a fixed-size pool, which can be used as unlabeled points. The availability of the fixed-size pool makes DRE-based Bayesian optimization naturally built on semi-supervised learning. On the other hand, as pointed out by your comment, for the scenario of generic Bayesian optimization, the generation of unlabeled points may be nonsensical. However, our sampling process based on the truncated Gaussian distribution allows us to focus on the proximity of the query points that have already evaluated. This is aligned with the nature of Bayesian optimization, i.e., the consideration of both exploitation and exploration.

Reviewer comment (Theory):
In most applications of BO, however, this concept is nonsensical: the only x distribution is the points sampled by BO, which are (in some sense) arbitrary (depending on the acquisition function they may cluster around the optimum as time passes, but not necessarily).

Author response:
We sincerely appreciate your constructive comment to improve our work.

> One doubt I have with this paper perhaps stems from unfamiliarity with semi-supervised algorithms in general. My understanding of such approaches is that they tend to assume that the unlabelled training points are nevertheless generated from the underlying x distribution. In most applications of BO, however, this concept is nonsensical: the only x distribution is the points sampled by BO, which are (in some sense) arbitrary (depending on the acquisition function they may cluster around the optimum as time passes, but not necessarily). Am I missing a key point here?

We have solved two scenarios, pool-based Bayesian optimization and generic Bayesian optimization (defined on a continuous search space). Notably, the formulation of pool-based Bayesian optimization has access to a fixed-size pool, which can be used as unlabeled points. The availability of the fixed-size pool makes DRE-based Bayesian optimization naturally built on semi-supervised learning. On the other hand, as pointed out by your comment, for the scenario of generic Bayesian optimization, the generation of unlabeled points may be nonsensical. However, our sampling process based on the truncated Gaussian distribution allows us to focus on the proximity of the query points that have already evaluated. This is aligned with the nature of Bayesian optimization, i.e., the consideration of both exploitation and exploration.