Paper: Optimistic Bayesian Optimization with Unknown Constraints (ICLR-2024)

Reviewer comment (Experiments):
The paper does not explicitly introduce the motivation of decoupling the function query, and seems that the chosen real-world benchmark does not has the property of decoupling the function queries.

Author response:
We would like to emphasize that our methodology and theoretical analysis are applicable to both coupled and decoupled queries, as mentioned in the final paragraph of Sec. 1: Introduction. Therefore, our research holds significance not only for the decoupled setting but also for the coupled setting of constrained BO. We will further enhance the motivation and potential application of decoupling the function queries in our revised paper as follows:

**Motivation:** In Remark 3.2, we elaborate on the motivation that decoupling the function queries results in a more query-efficient solution. Hence, leveraging the ability to evaluate the objective function and constraints independently becomes advantageous, particularly in situations where the evaluation of each component is resource-intensive. For instance, when there is a high probability that the input query is feasible, the evaluation of constraints at that input query becomes unnecessary (especially if the optimal solution is far from the boundary of the feasible region, as illustrated in Figure 2a). In such scenarios, significant cost savings can be realized in the decoupled setting compared to the coupled one.

**Potential applications:** Here are two potential applications that showcase the benefits of employing decoupled queries:

   + A practical scenario of decoupled queries arises in the automotive industry. When designing the body of a car, various approaches are employed to assess its performance. For instance, computational fluid dynamics (CFD) is utilized to model the interaction between the car body and air flows. On the other hand, finite element analysis (FEA) is employed to simulate the car body's response to various loads, such as those encountered in collisions. One may construct a constrained optimization problem to maximize the aerodynamic performance of a car while adhering to safety constraints during collisions. CFD and FEA can be performed independently and performing each of them is expensive.

   + Another situation arises when selecting an ML model for deployment across different edge devices. These devices have diverse hardware configurations, leading to variations in prediction times and battery consumption. The goal is to optimize the model's prediction accuracy while meeting constraints on prediction times and battery consumption for various edge devices. Evaluating the prediction time and battery consumption on different devices and the prediction accuracy of the ML model can be performed separately, and these evaluations are time-consuming.

Reviewer comment (Evaluation):
In Figure 2 (h), the standard error of UCB-D is much larger than other baselines. Can you give some insights of why this happens?

Author response:
This is due to the logarithmic scale used in the plot, which accentuates small values. For example, on the log plot, the distance between 0.01 and 0.1 (a span of 0.09) is visually the same as the distance between 0.1 and 1.0 (a span of 0.9). Consequently, although the standard error of UCB-D appears larger in Figure 2h compared to other methods, it is indeed smaller than most of the methods. In Figure 2h, at the end of the line plot, the numerical value of the standard error for UCB-D is 0.015506. In comparison, the standard errors for ADMMBO, CMES-IBO, EIC, and UCB-C in Figure 2h are 0.37287908, 0.02479963, 0.01956778, and 0.01082636, respectively.

Reviewer comment (Evaluation):
In plotting the UCB-C's regret against the number of queries (e.g., Figure 2g), as evaluations are conducted at both the objective function and constraints in each BO iteration, how are they plotted? Does the plot advance by jumping every #constraints + 1?

Author response:
You are correct in noting that for UCB-C (involving coupled queries), the plot advances by jumping every #constraints + 1 queries. For UCB-D (involving decoupled queries), the plot advances with a jump every single query.

Reviewer comment (Theory):
Just curious, in cases where the query point is feasible, why don’t we evaluate the constraints together with the objective function? What are the advantages of strictly adhering to UCB-D?

Author response:
As our objective is to minimize the number of queries, we aim to minimize unnecessary queries to either the objective function or the constraints. Consequently, if we know that a query point is feasible, querying the constraints does not provide additional information on its feasibility. Therefore, we propose to query only the objective function to determine whether the query point is suboptimal (i.e., indicated by a low objective function evaluation). However, when the cost of querying constraints is not prohibitive or the queries are coupled, we may choose to evaluate both the constraints and the objective function simultaneously, as proposed in the UCB-C algorithm.

Reviewer comment (Novelty):
In Introduction, the motivation of studying decoupled constrained BO (CBO) is unclear to me. What are the real-world applications of decoupled CBO? In which case should we evaluate objective function and constraints at different inputs? In that case, can we run several independent standard BOs to solve all problems separately? Or can we run multi-objective BO to solve it?

Author response:
> 1. In Introduction, the motivation of studying decoupled constrained BO (CBO) is unclear to me. What are the real-world applications of decoupled CBO? In which case should we evaluate objective function and constraints at different inputs? In that case, can we run several independent standard BOs to solve all problems separately? Or can we run multi-objective BO to solve it?

   **Unified approach to both coupled and decoupled queries:** We would like to highlight that our approach and theoretical analysis work for both coupled and decoupled queries (see last paragraph of Sec. 1: Introduction). Thus, our work is important to not only the decoupled setting but also the coupled setting of constrained BO.
   
   **In which case should we evaluate objective function and constraints at different inputs?**
   We should evaluate objective function and constraints at different inputs when we can evaluate the objective function and constraints independently and the evaluation of each of them is expensive. In Remark 3.2, we also explored the circumstances under which employing decoupled queries could significantly improve efficiency in terms of the number of queries, as compared to coupled queries. For instance, when there is a high probability that the input query is feasible, the evaluation of constraints at that input query becomes unnecessary (especially if the optimal solution is far from the boundary of the feasible region, as illustrated in Figure 2a). In such scenarios, significant cost savings can be realized in the decoupled setting compared to the coupled one.

   **Real-world applications of decoupled queries:**

   + A practical scenario of decoupled queries arises in the automotive industry. When designing the body of a car, various approaches are employed to assess its performance. For instance, computational fluid dynamics (CFD) is utilized to model the interaction between the car body and air flows. On the other hand, finite element analysis (FEA) is employed to simulate the car body's response to various loads, such as those encountered in collisions. One may construct a constrained optimization problem to maximize the aerodynamic performance of a car while adhering to safety constraints during collisions. CFD and FEA can be performed independently and performing each of them is expensive.

   + Another situation arises when selecting an ML model for deployment across different edge devices. These devices have diverse hardware configurations, leading to variations in prediction times and battery consumption. The goal is to optimize the model's prediction accuracy while meeting constraints on prediction times and battery consumption for various edge devices. Evaluating the prediction time and battery consumption on different devices and the prediction accuracy of the ML model can be performed separately, and these evaluations are time-consuming.
   
   We will incorporate the above examples into Sec. 1: Introduction to further motivate the decoupled queries.

> In that case, can we run several independent standard BOs to solve all problems separately? Or can we run multi-objective BO to solve it?

   + We think that performing several independent standard BOs to solve a constrained BO problem with decoupled queries is a solution that is orthogonal to all existing constrained BO solutions. However, based on our current understanding, it is unclear (at least to us) how this can be accomplished effectively.
     
   + Similarly, it is unclear to us how multi-objective BO can be used to solve constrained BO if the queries are decoupled. We would appreciate the reviewer's guidance in providing further elucidation on these two approaches.

Reviewer comment (Theory):
I’m surprised to see definition of regret in eq (3) by combining objective function evaluation together with constraint violations. They may sit in totally different function ranges. Let F denote the range of objective function and let C denote the range of constraints. If C is much greater than F, then regret has little information about convergence. Also, the optimal point x* is defined w.r.t. objective function and $r_c$ is independent to x*. Why is $r_c$ is part of the regret?

Author response:
> 2. I'm surprised to see definition of regret in eq (3) by combining objective function evaluation together with constraint violations. They may sit in totally different function ranges. Let $F$ denote the range of objective function and let $C$ denote the range of constraints. If $C$ is much greater than $F$, then regret has little information about convergence. Also, the optimal point $x^\ast$ is defined w.r.t. objective function and $r_c$ is independent to $x^\ast$. Why is $r_c$ part of the regret?


   + **Why $r_c$ is part of the regret:**

     > Also, the optimal point $x^\ast$ is defined w.r.t. objective function and $r_c$ is independent to $x^\ast$. Why is $r_c$ part of the regret?

     It is important to clarify that the determination of $x^\ast$ is **not solely** based on the objective function in our context, as we are dealing with a constrained optimization problem. Consequently, $x^\ast$ is defined by considering both the objective function and the constraints, seeking to maximize the former while adhering to the latter. Therefore, the incorporation of $r_c$ (which quantifies the degree of constraint violation for a given input) into the regret is crucial. The regret, in this context, serves to evaluate the performance of an estimator $\tilde{x}_t$ in approximating the optimal solution. As a result, this regret must account for both suboptimalities arising from the objective function and the violation of constraints. Hence, our proposed regret that addresses these aspects is a unique contribution of our work, particularly considering the limited existing literature on regret analysis in constrained BO.

   + **Different function ranges**

     > I'm surprised to see definition of regret in eq (3) by combining objective function evaluation together with constraint violations. They may sit in totally different function ranges. Let $F$ denote the range of objective function and let $C$ denote the range of constraints. If $C$ is much greater than $F$, then regret has little information about convergence.

     + From a theoretical analysis perspective, the no-regret definition exhibits asymptotic behavior, specifically, the simple regret tends to $0$ as the number of iterations $T$ approaches infinity. The impact on convergence as $T$ approaches infinity remains unaffected by the magnitude of the function range, provided that the function range is not unbounded. Our proposed regret serves as an upper bound for regrets arising from the objective function and constraints, as it represents the maximum or sum of these regrets. Consequently, if our proposed regret approaches $0$, it implies that both the regrets associated with the objective function and constraints must also approach $0$, even in cases where their function ranges may differ. This can be conceptualized as constraining the worst-case regret among all regrets incurred by the objective function and constraints. We would like to point out that in the classic GP-UCB work, the asymptotic convergence holds irrespective of the scale of the function range.

     + From a practical point of view, one can normalize the observations at every BO iteration so that the observations from the objective function and constraints belong to similar ranges.

Reviewer comment (Theory):
How do you solve optimization problems in Line 3 and 4 in Algorithm 1? Definition of $O_t$ seems like making solving Line 3 intractable.

Author response:
> 3. How do you solve optimization problems in Line 3 and 4 in Algorithm 1? Definition of $\mathcal{O}_t$ seems like making solving Line 3 intractable.

   + Line 3: This is a constrained optimization problem with a closed-form expression of the objective function and closed-form expressions of the constraints. Hence, it can be solved with any existing constrained optimization package.
     + Objective function: $u_{f,t-1}(x)$;
     + Constraints: $u_{c,t-1}(x) \ge \lambda_c \ \ \forall c \in \mathcal{C}$ (from the definition of $\mathcal{O}_t$ in equation 9);
     + As defined in equation 7, $u_{h,t-1}$ only consists of the GP posterior mean and GP posterior standard deviation, both of which have closed-form expressions for a Gaussian process model. The set $\mathcal{C}$ is a finite set of constraints.
     
   + Line 4: We simply go through all constraints and find the one with the largest $\lambda_c - l_{c,t-1}(x_t)$.

Reviewer comment (Theory):
How does the last equation hold in eq 8 by adding vertical exploration bonus?

Author response:
> 1. How does the last equation hold in eq 8 by adding vertical exploration bonus?

Equation 8 is directly adopted (without any modification) from the work of Srinivas et al. (2010). The term in the argmax is just the definition of the upper confidence bound of $f(x)$. The introduction of the name "vertical exploration bonus" for a term in equation 8 aims to emphasize the analogous concept of horizontal exploration, which becomes relevant in the context of constrained BO later on.

Reviewer comment (Theory):
In last paragraph of Section 3.1, how does Lemma 3.1 imply that $O_t$ is non-empty with probability $>1-\delta$?

Author response:
> 2. In last paragraph of Section 3.1, how does Lemma 3.1 imply that $\mathcal{O}_t$ is non-empty with probability $> 1-\delta$?

If the optimization problem is feasible, then there exists at least $1$ feasible input $x_0 \in \mathcal{X}$, i.e., for all $c \in \mathcal{C}$, $c(x_0) \ge \lambda_c$. Lemma 3.1 says that $u_{c,t-1}(x) \ge c(x)$ for all $x \in \mathcal{X}$, $t \ge 1$, and $c \in \mathcal{C}$ with probability $\ge 1 - \delta$. Hence, if the optimization problem is feasible, $u_{c,t-1}(x_0) \ge c(x) \ge \lambda_c$ for all $c \in \mathcal{C}$ and $t \ge 1$ with probability $\ge 1 - \delta$. In other words, $\mathcal{O}_t$ is non-empty with probability $\ge 1 - \delta$ for all $t \ge 1$. We will include this argument in the revised paper.

Reviewer comment (Novelty):
Though the author highlights the connection of the proposed method to active learning (AL), it lacks a discussion on the link to the existing AL methods. For example, the concepts, including the uncharted area and $\nu_t$-relaxed feasible confidence region, resonate with the concepts in [1], and the analysis also bears connections.

Author response:
We will include the discussion of [1] in the revised paper. The major differences with [1] are briefly discussed below.

   + Our approach is not simply a combination of active learning for level set estimation and Bayesian optimization. As shown in Figs. 2a-c, our method adaptively allocates input queries based on specific scenarios, eliminating the necessity for performing level set estimation across the entire feasible region. In contrast, applying active learning for level set estimation typically involves uniformly exploring the complete feasible region.
   + The work of [1] requires monotonically decreasing confidence region by intersecting successive confidence intervals. Practical applications may encounter challenges, particularly if the intersection results in an empty confidence region. In contrast, our approach aligns more closely with conventional GP-UCB method, omitting the requirement for such a monotonic decrease in the confidence region and thereby avoiding the issue of empty confidence regions.
   + In [1], manually setting the value of $\epsilon$ is mainly for the theoretical analysis from our perspective. In our work, the value of $\nu_t$ is not manually set, but it is adaptively chosen by our algorithm. Furthermore, it is to balance between the vertical and horizontal exploration, a concept that does not exist in [1].

Reviewer comment (Theory):
The definition of regret is unconventional and lacks sufficient discussion. Typically, in the CBO setting, the reward is only defined within the feasible region, as there is no reward incurred by querying the points that are infeasible. The regret here is defined on both the objective and constraints, which circumvent the problem of infinite instantaneous regret in cumulative regret analysis when querying points out of the feasible region.

Author response:
The conventional viewpoint is that a constrained minimization problem with known constraints is often formulated as an unconstrained minimization problem with infinite values for infeasible inputs. The constraints in constrained BO are unknown, and so is its feasible region. Assigning infinite instantaneous regret to infeasible inputs is problematic. For example, when there are equality constraints, it is impossible to estimate the feasible region exactly (e.g., a line) given noisy observations. In such a case, we will never know if an input is feasible. Hence, without a finite measure of constraint violation, analyzing the regret is very challenging. As a result, we believe it is a natural approach to assign finite regret based on constraint violation. We have discussed the challenges of equality constraints in Appendix A. We will incorporate this discussion in the revised paper.

   Additionally, the limited theoretical works on constrained BO (only 1 theoretical work with coupled queries and no theoretical work with decoupled queries) deprive us of a conventional regret definition capable of addressing both the unknown objective function and unknown constraints. Hence, we consider introducing this concept of regret as a distinctive contribution of our work.

Reviewer comment (Novelty):
Could the author include the line of work in constraint active search [2][3] in a discussion of related work? They are closely related to the CBO problem as it aims at searching feasible points efficiently within feasible regions defined by unknown constraints.

Author response:
We will include the line of work in constraint active search [2,3] together with the level set estimation [1] in the related works. They are related to constrained BO since these approaches explore the feasible regions (i.e., either the boundary in the level set estimation [1] or the interior in the constraint active search [2,3]). It is also noteworthy that these approaches differ from constrained BO by not taking into account an objective function.

Reviewer comment (Novelty):
There is a recent paper studying a similar CBO solution [4] to the proposed algorithm. It is unnecessary to include it in the paper due to its release publication time, but I encourage the author to explore it.

Author response:
We appreciate your update regarding the recent paper [4]. It appears to have been conducted concurrently with our research. The paper seems to investigate a different notion of regret and employ a different BO strategy. We plan to delve into the paper more thoroughly for a detailed comparison.