Paper: Bayesian Vector Optimization with Gaussian Processes (ICLR-2024)

Reviewer comment (Novelty):
There are a lot of existing works on preference learning with BO.

Author response:
**(... a lot of existing works ...)** In the revised version of the paper, we update the related works part of the introduction by including the existing works suggested by the reviewer.

Reviewer comment (Reproducibility):
It is not clear how to use cones for a practitioner.

Author response:
**(... how to use cones for a practitioner...)**  In [1], authors discuss the use of cones in practice with preferences of decision maker in the form of relative importance (equivalently trade-off constraint). They give an example similar to the farming example in our paper's introduction. In [2], this approach is generalized to arbitrary dimensions, and authors consider the case where a number of criteria are considered to be less important than others. They algebraically characterize the polyhedral ordering cones that correspond to preferences in the mentioned form. In their case, the decision-maker is required to pass on the trade-off constraint among objectives , which is practical.

[1] Hunt, B.J., Wiecek, M.M. (2003). Cones to Aid Decision Making in Multicriteria Programming. In: Multi-Objective Programming and Goal Programming. Advances in Soft Computing, vol 21. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-36510-5\_20

[2] Hunt, Brian J. et al. “Relative importance of criteria in multiobjective programming: A cone-based approach.” Eur. J. Oper. Res. 207 (2010): 936-945.

Reviewer comment (Novelty):
The links to a similar method, PAL, are not detailed enough. An empirical comparison with this method is needed.

Author response:
**(The links to a similar method, PAL ...)** In order to make the connection clearer, we added additional texts in the introduction. As it's mentioned there, when VOGP is considered with the componentwise order, it can be seen as a variant of PAL that can handle dependent objectives. Therefore, we suspected that a comparison with PAL in componentwise order would boil down to dependent versus independent modeling which would not be informative.

Reviewer comment (Reproducibility):
The empirical results are not reproducible.

Author response:
**(The empirical results are not reproducible.)** We note that the code of VOGP and its experiments was submitted through the supplementary material submission.

Reviewer comment (Theory):
It is unclear what the inclusion relation is between Pareto optimal solution and cone order optimal solution (“However, this approach can be restrictive, as it only permits a certain set of trade-offs between objectives.” and then “preference cones provide a way to bias the search toward certain regions of the Pareto front.”).

Author response:
**( ... It is unclear what the inclusion relation ...)** We say that a vector weakly dominates another with respect to $C$ if their difference lies in $C$. Since the 90 degree cone is a subset of the 135 degree cone, if the difference of two vectors lies in the 90 degree cone, then it will also lie in the 135 degree cone. Since the Pareto set consists of all non-dominated design , a design that is not dominated under the 135 degree cone will not be dominated under the 90 degree cone. Hence, the Pareto set with respect to the 135 degree cone is a subset of the Pareto set with respect to the 90 degree cone.

Reviewer comment (Reproducibility):
Can you describe the convex optimization problem used in the pessimistic Pareto front construction?

Author response:
**(Can you describe ...)** To see whether a point1 is in pessimistic pareto set, we check if all its vertices of another hyperrectangle (of point2) can be represented by summation of a help vector from the cone and a point from the  points1's hyperrectangle. This is to check the pessimistic Pareto definition. If this holds for any such point2, point1 is not in the pessimistic Pareto set. CVXPY library of python is used to solve the optimization problem.

Reviewer comment (Theory):
It is unclear where the cone properties appear in the theoretical results.

Author response:
**(... where the cone properties appear ...)** In both bounds found in both Theorem 1 and Theorem 2, $d(1)$ is a cone-dependent constant that reflects the effects of different ordering cones.

Reviewer comment (Experiments):
Could you provide timings? Progress curves rather than just fixed snapshots? Random search should be added as a baseline.

Author response:
**(Random search should be added as a baseline.)** We kindly refer the reviewer to Section C of the updated paper.

**(Progress curves rather than just fixed snapshots?)** In the updated paper, we provide progress curves. We kindly refer the reviewer to Figure 3 of the updated paper.

Reviewer comment (Reproducibility):
As I understand, the noise hyperparameter is not learned by the GP? What is multi-output covariance kernel used? Is it the same across all compared methods? Is the learning strategy shared among all methods (e.g., fixed hyperparameters)?

Author response:
**(... the noise hyperparameter ... )** In the problem definition, a fixed noise variance is assumed. In the experiments, the variance is assumed to be known and is not trained. 

**(What is multi-output covariance kernel used?)** The model used in the experiments is the Mulitask model proposed in [3]. The other methods mentioned in the paper were proposed and implemented with independent GPs. Therefore, the GP used for other methods is independent GP, where the relation among objectives are not modelled. 

 [3] Bonilla, Edwin V., Kian Chai, and Christopher Williams. "Multi-task Gaussian process prediction." Advances in neural information processing systems 20 (2007).

**(Is the learning strategy shared ...)** Yes.

Reviewer comment (Reproducibility):
Could you add an example with more than 2 objectives? Too many details are missing to reproduce the experiments: code used, number of samples, etc.

Author response:
**(Could you add an example with more than 2 objectives?)** We added a real-world dataset to the experiments section that has 6 design dimensions, 4 objective dimensions and 1000 datapoints. The performance results of VOGP on this dataset can be seen in Table 2 of the updated paper.

**(Too many details are missing to reproduce the
experiments: code used, number of samples, etc.)**
We note that the code to reproduce the results were shared through supplementary material submission. The number of samples are given in the experiments section under the name ``SC" (e.g., in Table 2).

Reviewer comment (Writing):
The authors can consider prioritizing intuitive understanding over exposition of all the math.

Author response:
**(... prioritizing intuitive
understanding ...)** In the updated paper, we include a remark (Remark 1) that clarifies what the success conditions mean intuitively. We also provide a diagram that visualizes the process and the steps of Algorithm 1 (Figure 3). We hope these additions will help improve the clarity of the paper.

Reviewer comment (Presentation):
The same goes for the algorithm itself; perhaps add a diagram on the procedure the policy goes through in Algorithm 1.

Author response:
**(... prioritizing intuitive
understanding ...)** In the updated paper, we include a remark (Remark 1) that clarifies what the success conditions mean intuitively. We also provide a diagram that visualizes the process and the steps of Algorithm 1 (Figure 3). We hope these additions will help improve the clarity of the paper.

Reviewer comment (Experiments):
The paper does a good job comparing the proposed algorithm against state-of-the-art multi-objective BayesOpt policies. However, from what I understand, only one algorithm from the vector optimization literature, Naïve Elimination, is included.

Author response:
**(... only one algorithm ...)** To the best of our knowledge, Naive Elimination in Ararat and Tekin (2023) is the only other algorithm that can handle vector optimization problems with polyhedral ordering cones (other than the positive orthant) in the stochastic K-armed bandit framework with observation noise.

In the literature on deterministic vector optimization, there are algorithms that can solve constrained linear and convex vector optimization problems under general polyhedral ordering cones. Below is an incomplete list of works where such algorithms were proposed:

- Löhne, Rudloff, Ulus. “Primal and dual approximation algorithms for convex vector optimization problems.” Journal of Global Optimization 60 (2014): 713-736.


- Hamel, Löhne, Rudloff. “Benson type algorithms for linear vector optimization and applications.” Journal of Global Optimization 59 (2014): 811-836.

- Ararat, Ulus, Umer. “A norm minimization-based convex vector optimization algorithm.” Journal of Optimization Theory and Applications 194 (2022): 681-712.

However, none of these algorithms are applicable for our problem as these algorithms assume a completely deterministic and fully observable objective functions that satisfy linearity/convexity assumptions. That is why we could compare our method only with Naive Elimination.

In addition, in the supplementary document, we compare VOGP with the other MOO algorithms in the stochastic bandit framework by post-processing their outputs (that are computed under the usual positive orthant cone) with respect to the cone $C=C_{135^o}$.

Reviewer comment (Experiments):
In many real-life settings, we don't have access to the correct hyperparameters, or even good priors for them.

Author response:
**(... GP always has access to the correct hyperparameters ...)** In the updated supplemental document, we share an ablation study where the empirical effects of unknown or partially known kernel hyperparameters are discussed. We kindly refer the reviewer to Section C.

Reviewer comment (Experiments):
The paper could benefit from studying the effects of the GP having the wrong hyperparameters on the performance of the algorithm.

Author response:
**(... GP always has access to the correct hyperparameters ...)** In the updated supplemental document, we share an ablation study where the empirical effects of unknown or partially known kernel hyperparameters are discussed. We kindly refer the reviewer to Section C.

Reviewer comment (Writing):
In Definition 1, my understanding is that the second condition (ii) specifies that \( x \in P \) is not dominated by another by more than \( 2 \epsilon \). What does the first condition (i) say? Perhaps the the background of vector optimization could benefit from more descriptive discussions.

Author response:
**(... What does the first condition (i) say? ...)** It is correct that (ii) specifies that  $x \in P$  is not dominated by another by more that $2\epsilon$. (i) specifies that for all $x^* \in P^*$, there exists a $x \in P$ such that $x$ dominates $x^*$ with a help of $\epsilon$ normed vector. So, (i) can be thought as the covering condition.

Reviewer comment (Theory):
The authors noted that once a point is added to the Pareto set, it will not be removed. This doesn't match my intuition well; isn't it possible that as we learn more about the objectives, we realize that some of the points already added aren't non-dominated?

Author response:
**(... Pareto set, it will not be removed ...)** Pareto Identification and discarding subroutines check the ordering relation for all the possibilities inside confidence regions (i.e., hyperrectangles). By doing so, given the hyperrectangles include the true objective values, VOGP algorithm's Pareto identification subroutine makes sure the added designs are in the Pareto set, no matter how early in the algorithm. This is one of the strengths of VOGP, as designs are labeled as Pareto without waiting for the whole algorithm to terminate. This opens up applications in pipelines where the early identified Pareto designs are moved through the pipeline immediately, increasing the efficiency. Another application that arises from this is the task of identifying only one Pareto design. While other methods might need the whole algorithm to terminate and pick one from the returned Pareto set, VOGP would most efficiently identify the first Pareto design.

Reviewer comment (Theory):
Could the authors comment on the possible difficulties of extending the proposed algorithm to continuous setting? I imagine the challenge lies in the discarding and Pareto identification phases. Can we try to discard and identify dominated and non-dominated regions, respectively?

Author response:
**(... extending the proposed algorithm to continuous setting ...)**  Yes, the main challenge is that the Pareto Identification and Discarding steps are one to one comparison based steps which does not work in continuous problems where infinitely many comparisons are required. An extension where regions instead of points are considered can indeed be the solution. A practically easy instance of this kind of solution is the method of adaptive discretization where the search space is intelligently divided with points that are representatives of regions around them [1].

[1] Nika, A., Bozgan, K., Elahi, S., Ararat, Ç., \& Tekin, C. (2020). Pareto active learning with Gaussian processes and adaptive discretization. arXiv preprint arXiv:2006.14061.

Reviewer comment (Experiments):
The experiments are done on very small datasets. The largest dimension is $4$ and all tasks have two objectives.

Author response:
**(The experiments are done on very small datasets ...)** We added a real-world dataset (MAR dataset) to the experiments section that has 6 design dimensions, 4 objective dimensions and 1000 datapoints. This dataset has the objective of optimizing bulk carrier architecture specifically for navigating the Panama Canal. The performance results of VOGP on this dataset can be seen in Table 2 of the updated paper.

Reviewer comment (Theory):
At this point, the method in the paper is restricted to discrete domains $\mathcal{X}$, which limits the application of the method: the constant $\beta_t$ in the algorithm depends on the cardinality of $\mathcal{X}$, the theorem statements assume finite domains, and the evaluation metrics needs a finite cardinality $|\mathcal{X}|$ as well. I would assume this is fixable by extending the results in the paper, but additional empirical evaluations are required.

Author response:
**(.... the method in the paper is restricted to discrete domains ...)** Indeed the extension of VOGP to continuous spaces is possible through a method called adaptive discretization like in [1] where the design space is intelligently discretized so that the continuous space is explored well. Though its implementation is simple, carrying theoretical guarantees over to the adaptive discretization version of VOGP requires a more spohisticated theoretical analysis. We view this extension of our work as a promising future research direction.

[1] Nika, A., Bozgan, K., Elahi, S., Ararat, Ç., \& Tekin, C. (2020). Pareto active learning with Gaussian processes and adaptive discretization. arXiv preprint arXiv:2006.14061.

Reviewer comment (Experiments):
The naive elimination is a theoretical construct in the bandit setting, which does not exploit the correlation in the GP model. A more meaningful baseline for comparison is other multi-objective BO algorithms. For example, as the angle $\theta$ changes, how do the Pareto precision and Pareto recall change comparing with regular multi-objective BO algorithms (which are designed specifically for a particular definition of Pareto optimality)? Another helpful experiment is to plot metrics w.r.t. the number of queries. This allows us to visually check the convergence. From the current tables, it is hard to tell if they are fully converged or not.

Author response:
**(... a more meaningful baseline for comparison  ...)** In the updated supplemental document, we included the comparison of VOGP with other MOO methods under 135 degree cone order. The results of this experiment shows the necessity of cone-dependent algorithm (VOGP) when the used cones are not positive orthant cones (the $90^\circ$ cone). We kindly refer the reviewer to Section A.2.

Reviewer comment (Experiments):
The experimental setup is non-standard. For example, "for each dataset, we learn the kernel hyperparameters by training on the entire dataset". However, the hyperparameters in BO are typically learned as more queries are added to the training data. Have the author tried the latter more commonly used setting?

Author response:
**(The experimental setup is non-standard ...)** In the updated supplemental document, we investigate the effects of the assumptions on kernel hyperparameters in detail. We consider the cases where kernel hyperparameters are unknown, known and partially known. We share the performance results of VOGP under these cases and discuss the effects. We kindly refer the reviewer to Section C.

Reviewer comment (Writing):
The following notations need more explicit definitions in the main text: the hyperrectangles $R_t(x)$, their diameters $\omega_t(x)$, the information gain $\gamma_t$ and the constant $\eta$.

Author response:
**(The following notations ...)** The equation that defines the hyperrectangles is now written more explicitly. The explicit definition of hyperrectangle diagonal is now provided in the explanation of the evaluating phase. The definition of $\eta$ is now provided in Theorem 1. We note that an already explicit definition of $\gamma_t$ was given in Definition 2. We would appreciate if the reviewer could provide details to be added to make it more explicit.

Reviewer comment (Theory):
In the definition of the polyhedral cone, it should be $C = \{\mathbf{x} \in \mathbb{R}^M: W \mathbf{x} \geq 0\}$ and $W$ should be $N \times M$. The cone is defined in the output space, not in the domain.

Author response:
**(In the definition of  ...)** Fixed.

Reviewer comment (Theory):
Theorem 1 and Theorem 2 need to add an extra technical assumption on the cone $C$. Otherwise the bounds may be vacuous in certain cases. For example, $C = \{(x_1, x_2) \in \mathbb{R}^2: x_1 = x_2\}$ is a well-defined polyhedral ordering cone. However $d(1) = \infty$ in this case and thus both bounds become vacuous.

Author response:
**(Theorem 1 and Theorem 2 ...)** By the definition of the ordering cone (proper cone), an ordering cone has non-empty interior (Section 2.4 of [2]). Therefore, the mentioned cone is not an ordering cone that is allowed in our setting.

[2] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge: Cambridge University Press, 2004.

Reviewer comment (Theory):
Can you share more intuition on Definition 4? My intuition is that $\mathbf{u}^*$ points to the "center" of the cone.

Author response:
**(... more intuition on Definition 4 ...)** Indeed, $\boldsymbol{u^*}$ points to the ``center" of the cone. More specifically, it points to the center of the sphere or a circle that is tangent to every halfspace that defines $C$. In two dimensions, this corresponds to the center of the incircle (though there are only two lines that define the cone in 2D: given the circle radius, the third edge of the triangle is implicitly specified).

Reviewer comment (Experiments):
Branin and Currin are continuous functions defined on continuous domains. Does the experiment in Table 1 discretize their domains?

Author response:
**(Branin and Currin are continuous functions ...)** Yes, the datapoints were randomly sampled from the continuous functions.

Reviewer comment (Experiments):
There might be more applications illustrated.

Author response:
(There might be more applications ...) In the experiments section, we added the application of VOGP on another real-world dataset that has six design dimensions and four objective dimensions. This dataset has the objective of optimizing bulk carrier architecture specifically for navigating the Panama Canal. The performance results of VOGP on this dataset can be seen in Table 2 of the updated paper.

For more possible applications of VOGP, we kindly refer the reviewer to our response to the second question of reviewer ahkf.