Paper: Constrained Bayesian Optimization with Adaptive Active Learning of Unknown Constraints (ICLR-2024)

Reviewer comment (Novelty):
The active learning component (the difference between the confidence bounds) is oddly presented and is not novel. The authors recognize this, but fail to recognize that the active learning method harks back to at least MacKay (1992) and has been widely applied since. Global variance reduction should not, in my opinion, be presented as a novel aspect of any algorithm. Moreover, it should be presented as Global variance reduction, or ALM (Active Learning MacKay), or otherwise, but not as a difference between confidence bounds. This simply disguises the method as something seemingly more complex than it is.

Author response:
We sincerely appreciate the reference pointed out by the reviewer, where the active data selection on the region of interest is thoroughly discussed. We acknowledge the similarities between our approach and the global variance reduction methods or ALM, as well as their roots in previous work. In the revision, we will clarify this connection and refrain from describing the active learning aspect of COBALT as entirely novel. However, we would like to highlight the difference between the active learning methods referred by the reviewer and the proposed algorithm for constrained BO tasks. 

We aim to propose a principled tradeoff of **active learning of multiple unknown constraints and the optimization of the objective in the feasible region**. To achieve this, we choose to unify the two different aspects---namely variance reduction and UCB, by framing them under the same mathematical framework as elaborated in Eq (5) and Eq (6). This framing ensures the comparability of the acquisition function of the unknown objective and the acquisition function of the unknown constraints. Further, it serves as the foundation for defining our acquisition function tailored to both needs. The other reason we use the interval between UCB and LCB is to follow the convention in previous works (AL-LSE[5] and BALLET[6]).

Another significant difference between the proposed method and the global variance reduction is that we are doing **combined variance reduction** on adaptively identified **intersected ROI**, which leverages the UCB or LCB as the threshold.  The **shrinkage of the intersected ROI** guarantees the exploitation perspective of the acquisition and avoids budget waste, as discussed with reviewer AopU. The **variance reduction on ROI (as in eq (6)) and standardized GP-UCB** on ROI (as in eq (6)) achieve the exploration for both active learning and optimization. With the combination of ROI intersection and variance reduction on both the objective and the constraints, the proposed method achieves an **efficient and principled tradeoff** between exploration and exploitation. This serves the purpose of constrained BO and resonates with the well-known tradeoff in conventional BO. 

As pointed out by other reviewers, given the **specific constrained BO reward** discussed in this paper, this is the **first theoretical result of the convergence of the constrained BO** algorithm we are aware of.

Reviewer comment (Theory):
Theoretical analysis contains substantial flaws, starting with Assumption 3: "Assumption 3 Given a proper choice of $\beta_t$ that is non-increasing, the confidence interval shrinks monotonically...". This is flawed for two reasons: (1) It appears trivial (beta is non-increasing, so the confidence interval will naturally be smaller as beta shrinks). If there is a nuance to this assumption that I am missing, I encourage the authors to enlighten me. (2) Assuming a non-increasing $\beta_t$ goes against a decade of BO convention, set by Srinivas et. al. (2009) that beta should scale as $\mathcal{O}(\log t)$.

Author response:
We’d like to point out that there is a subtle difference between a monotonically shrinking $2\beta_t^{1/2}(x)\sigma_{t-1}(x)$ and the consistency of confidence interval at different iterations, as discussed in assumption 3. The former is trivial given the monotonicity of $\\sigma_{t-1}(x)$, while the latter could possibly be violated even when $\beta_t$ is non-increasing. For example, if the observed $y_t$ actually lies on the confidence interval, the resulting posterior confidence interval at $x_t$ could disagree with the confidence interval of the prior. Assumption 3 actually assumes a consistency of prior and posterior confidence intervals, and it helps with the analysis by guaranteeing a consistently shrinking ROI.

If we only resort to the monotonicity of the pointwise $ \sigma_{t-1}(x)$ when having a constant $\beta$ for $\forall 1 \geq t \geq T$ where T is the known total budget, we could revise the $\beta$ to be dependent only on $\tilde{D}$ rather than  $\tilde{D}_{\hat{X}_t}$. In that case, the theoretical results still hold. We refer to the detailed discussion in Appendix A of the revised paper.

Reviewer comment (Theory):
The subsequent Lemma 1 and Theorem have numerous un-introduced quantities ($K, \pi_t$) and appear to run contrary to Assumption 3. In Theorem 1, $\beta_t =2 \log( 2K+1)|D_{X_t}|$ where $D_{X_t}$ should be increasing in the data, making $\beta_t$ increase over time — thus not non-increasing. Admittedly, $D_{X_t}$ is the intersection between a discretization of the search space and the data, which clearly contains only the data, at most. The non-increasing property of $\beta$ is re-stated in the Appendix, suggesting it is not a mistake. The theoretical results are difficult to properly assess due to these apparent issues and ambiguities, so I will refrain from commenting on the correctness of the proofs in the appendix until these issues are clarified.

Author response:
We appreciate the detailed question here. The reason $\beta_t=2\log(2(K+1)\vert \tilde{D}\vert \pi_t/ \delta)$ needs to grow is the need to guarantee the convergence of the series, which allows an asymptotical sublinear regret. Specifically, the requirement is $\sum_t{\pi_t} = 1$. Yet, in our problem setup, since cumulative regret could not be bounded as instantaneous regret could be infinite when the evaluated candidate $x_t$ violates any of the constraints (cf. Equation (3)), we aim to guarantee that after a certain horizon T, the *simple regret* could be bounded. Therefore, there is no need to have a convergence series $\sum_t{\pi_t} = 1$. As shown in the proof, we only need $\sum_{t=1}^T{\pi_t} = 1$ for a given budget/horizon T, and setting $\pi_t = 1/T$ satisfies it. We made revisions in the analysis section and the proof part to highlight this.

Reviewer comment (Writing):
The algorithm "Maximize the acquisition values from different aspects:" is informal, and line 11 is not well-defined. What does the $\text{argmax}$ over $\mathcal{G}$ (but not $x$) mean? Moreover, $x_{g, t}$ in line 12 has not been obtained from any prior step. Should the $g_t$ in line 11 really be $x_{g,t}$? As such, it is still not clear to me what the resulting acquisition function is, but I believe it is the $\text{argmax}_{g, x}$ over all constraint and BO acquisitions, jointly.

Author response:
We agree that the unconventional notation could be confusing and want to add additional clarification as the following. $\{x_{g,t}\}$ in line 11 are acquired in line 7 as $x_{\mathcal{C_k}, t}$ or line 9 as $x_{f, t}$, since $\mathcal{G}$ is composed of $\mathcal{C_k}$ and $f$. Roughly speaking, we are taking $arg\max_{g, x}$, yet we avoid using such notation for two reasons. (1) the domain where equation 5 and equation 6 are maximized are different; (2) the domain for equation 6 could even be empty. Therefore, we are currently taking the $arg\max$ of equation 5 and equation 6 over different domains (if not empty) separately and then taking the $arg\max$ of the corresponding acquisition function values as in line 11.

Reviewer comment (Presentation):
The figures are difficult to parse due to the odd color choices (bright grey and dark orange on orange in Fig. 1, tiny legends everywhere).

Author response:
We sincerely appreciate the constructive comments and will revise the color and legends for easier parse.

Reviewer comment (Reproducibility):
Competing methods do not seem to be properly implemented. SCBO (Eriksson and Poloczek, 2019) builds on TuRBO, one of the more robust BO methods around. Seeing it fail on most tasks is a warning sign, and suggests to me that there are implementation flaws. If the authors believe it has been correctly implemented and used, I would suggest they motivate its poor performance in their specific setting.

Author response:
We appreciate the concern raised by the reviewer. We’d like to point out that SCBO shows strong performance in the second and fifth columns of Figure 3 and in Car-Car-7D-8C in Figure 4.  We found its performance strongly depends on the initial center of the trust region. If the initial trust region is correctly identified, it demonstrates superior performance. The previous work [1] also shows a similar inconsistency of performance by TSC (a variant of SCBO) in Figure 3 of [1]. We would like to additionally cite the critics of SCBO by Takeno et al. (2022), that when the identified feasible region is empty, there is no rationale for selecting points of minimum constraint violations in the sense of TS. 

We appreciate the constructive comments by the reviewer and will accommodate additional discussion over SCBO performance in the revision.

Reviewer comment (Theory):
The constraint functions $g$ are assumed to be continuous-valued. Is this conventional, and is the method amenable to non-continuous (i.e. binary output) constraints?

Author response:
We follow the convention in constrained BO [1, 2] and active learning [4, 5] literature that assumes that the unknown constraint is sampled from a GP and there is the corresponding threshold defining the corresponding feasible area. The non-continuous constraints would require incorporating additional treatment, e.g., ROIAL, an ROI-based AL framework [3] that is designed for preferential or ordinal feedback. Their framework assumes that there is an underlying continuous utility function and uses information gain as the acquisition function, though no theoretical guarantee is provided.

Reviewer comment (Theory):
$x_t$ can be picked as $x_{\mathcal{C}_k,t} = \text{arg}\!\max_{x \in U_{\mathcal{C}_k,t}} \alpha_{\mathcal{C}_k,t}(x)$, so it may belong to $L_{\mathcal{C}_{k'},t}$ for some $k' \neq k$. In this case, it raises the question of why the objective and constraints are evaluated at an infeasible input.

Author response:
Yes, this motivates the design of the intersection of ROIs and different ROIs for objectives and constraints. We want to avoid inefficient queries. This is a very good point. Actually, the reviewer might refer to the code we provided (src/opt/constrained.py line 523-524); we are actually intersecting $U_{C_k,t}$ with $\hat{X_t} $ instead of maximizing on the full $U_{C_k,t}$ on line 7 in algorithm 1. This is actually a typo, and we apologize for the confusion. Also, please note that in the proof of theorem 1, we’ve been using the correct domain.

Reviewer comment (Theory):
In equation (5), when $\text{LCB}_{f,t,\max} = \infty$, we basically pick the most uncertain $f(x)$. However, there may exist uncertain $f(x)$ with a very low evaluation. Then, why do we need to pick this input if we are interested in large $f(x)$?

Author response:
Please note that we are picking the most uncertain $f(x)$ on ROI, where the $UCB_{f} (x)$ is guaranteed to be above a certain threshold. Intuitively, we are narrowing the confidence interval of $f(x^*)$. This serves as an efficient global optimization acquisition as the uncertainty maximization guarantees the exploration while the ROI shrinking guarantees the exploitation. When $LCB_{f,t,max}$ is infinity, it means there is no super level-set identified, and taking the maximum of UCB could be optimizing the objective function out of the feasible region, which doesn’t always contribute to the performance.

Reviewer comment (Novelty):
The paper asserts that existing constrained Bayesian optimization lacks theoretical analysis. However, the work titled "No-Regret Bayesian Optimization with Unknown Equality and Inequality Constraints using Exact Penalty Functions" by Lu and Paulson (2022) provides theoretical analysis.

Author response:
We’re grateful for the reference provided by the reviewer and would like to add additional discussion to the related work.  We would like to highlight that regret is different, and we provide theoretical justification in the task following the constrained BO problem setup where the infeasible candidate doesn’t incur a reward.

Reviewer comment (Theory):
The value of $\beta_t$ is defined based on $\hat{X}_t$ but $\hat{X}_t$ is defined based on $\beta_t$. Therefore, $\beta_t$ cannot be defined in Theorem 1.

Author response:
We appreciate the reviewer pointing out the typo here. Actually, the reason we introduce assumption is to guarantee a monotonically shrinking $\hat{X_t}$, meaning we could use $\hat{X}\_{t-1}$ as the conservative surrogate for $\hat{X_{t}}$. In the revision, we will avoid confusion by using $\tilde{D}$ instead.

Reviewer comment (Theory):
The paper may need more arguments to explain why the approach of Takeno et al. (2022) "violates the theoretical soundness".

Author response:
We acknowledge that the criticism could be overstated. Yet, as admitted in Takeno et al. (2022), there is no regret guarantee for the entropy-based method. It proves that the link to the UCB method in previous work could be an error.

Reviewer comment (Theory):
Theorem 1 does not address feasibility. Furthermore, Srinivas et al. (2009) analyze the regret of *some input* (for example $x_t$). While this paper introduces a definition of regret (or reward) of inputs, it does not use this definition in the theoretical analysis. This raises questions about the significance of Theorem 1 in terms of the algorithm's achieved reward (or regret).

Author response:
(1) In assumption 2, we assume the feasibility. Theorem 1 shows that if there are feasible points, we can identify them after sufficient query with a certain confidence. Otherwise, the empty super level-set would suggest that with a certain confidence, assumption 2 is violated. That is, there is no feasible area.

(2) With regards to actual reward, if the identified Superlevel set is non-empty, the acquisition function $acq_{f,t}$ on that would actually be $UCB_f(x)$, which shares the regret guarantee with Srinivas et al. (2009). If the user wants to guarantee a query on the argmax of $UCB_f(x)$, the user can enforce that at any time or afterward. Otherwise, the algorithm is likely to query at least the argmax of $UCB_f(x)$ once within a large interval. We also add an additional corollary of theorem 1 in the revision to show the guarantee of a simple regret.

Reviewer comment (Experiments):
The abstract highlights a setting where "the objective and constraints can be independently evaluated", yet the paper primarily focuses on the coupled setting where they are evaluated together. The decoupled setting is briefly explained in the appendix without any supporting experiments.

Author response:
We do provide the algorithm and analysis as an interesting side product for the decoupled setting. Yet, we do not intend to highlight the focus on the decoupled setting due to the limited scope of this work and motivating application. We would leave the comprehensive study of decoupled settings into future work. We would tune down the sentence in the revision.

Reviewer comment (Evaluation):
The chosen performance metric is simple regret. However, in practical applications, identifying the input with minimum regret is unknown. Thus, the paper does not address what input Bayesian Optimization recommends as the optimal solution.

Author response:
We appreciate the comment and add additional discussion over the guarantee of a simple regret by picking the $arg\max_{\hat{X_T}} LCB_{f, T}(x)$ in the revised appendix.

Reviewer comment (Experiments):
The experiment section is quite short. Surprisingly, in the Rastrigin-1D-1C, the discrete search space consists of $1000$ points but the proposed algorithm reaches the optimum with $2000$ iterations. Furthermore, the noise is small $\mathcal{N}(0,0.1)$ compared to the function range $[-40,0]$. This efficiency is questionable.

Author response:
We agree that on conventional BO tasks on the 1D Rastrigin without unknown constraints, the 2000 iteration seems overkill, especially given the small noise. Yet the challenge is poised by the tradeoff between learning the feasibility and optimizing the objective with noisy observation. The performance of the baselines after 2000 iterations reveals that the problem lacks a principled tradeoff. We consider a small perturbation on the constraint significant due to the distribution of Rastrigin-1D and the positions of the optimal area. Additionally, the simple regret actually drops fast in the beginning 400 iterations. It is expected that the algorithm could query suboptimal several times due to noise and the existence of multiple identical objective values.

Reviewer comment (Theory):
Assumption 2 does not hold for equality constraints.

Author response:
Yes, we believe this relates to the discussion in 3.d and 3.e., if the objective doesn't incur reward out of the feasible region, it is challenging to guarantee the performance without the knowledge of epsilon or when epsilon converges to 0. However, in practice, the user might slightly adjust the threshold $h_k$ to accommodate the $\epsilon_k$.

Reviewer comment (Novelty):
The name COBALT coincides with an existing Bayesian Optimization work titled 'COBALT: COnstrained Bayesian optimizAtion of computationaLly expensive grey-box models exploiting derivaTive information' by Paulson and Lu (2021).

Author response:
We sincerely appreciate the comments by the reviewer and will fix them correspondingly in the revision. We are grateful the reviewer pointed out the naming coincidence. We acknowledge this in the revision.

Reviewer comment (Reproducibility):
Some important experimental details are missing. It is unclear how the hyperparameter $\beta_t$ is set in the experiments. The constant $\beta_t$ balances the exploration and exploitation and has huge impact on the practical performance.

Author response:
```
The constant balances the exploration and exploitation and has a huge impact on the practical performance. It is unclear how it is set in practice.
```

We appreciate the constructive comment. We added additional results and corresponding scripts in the revision to demonstrate the robustness against different choices of $\beta$. In summary, among the choices of the theoretical results or constant values, including 0.1, 2, 4, and 8, we observe that only .1 could trap the algorithm in the sub-optimal area due to its lack of exploration.

Reviewer comment (Theory):
The exact condition on the discretization $\tilde D$ is unclear in the statements of Lemma 1 and Theorem 1. According to Lemma 1, $\tilde D$ can be an arbitrary discretization of the domain as long as it contains the global optimum. However, I don't quite think this can be true. For example, the discretization of Srinivas et al. (2009) has to be large enough (exponential) to roughly cover the entire domain. Otherwise, the concentration inequality only holds inside the discretization $\tilde D$ and cannot be generalized to the entire domain.

Author response:
```
The exact conditions of the discretization are unclear in the statement of lemma 1 and theorem 1.
```

We'd like first to clarify the discretization as it is related to the other concern raised by the reviewer. Throughout the design of the algorithm, we focus on dealing with the **finite discretization $\tilde{D}$ of the original search space**, which aligns with many engineering tasks where only the discretization is feasible or a known smoothness on locality alleviates the needs to scan the whole continuous space. 

As noted by the reviewer, Srinivas et al. (2009) start by studying a search space of finite discretization and then introducing the smoothness assumption to extend the results to a multi-dimensional continuous search space where the regret bound relies on the search space. We do not further extend the discussion to a continuous search space for two reasons. (1) The motivating application for us does not deal with a continuous search space; (2) There are recent advancements introducing **various treatments** of continuous search space given different assumptions on the structure of the problem. For example, with regard to the high-dimensional BO task as concerned by the reviewer, except directly creating discretization, which incurs the curse of dimensionality, various principled treatments [3,4,5] have been studied. They could be incorporated into our constrained BO framework to address the concern that the construction of a finite search space could be troubled by high dimensionality. A comprehensive discussion of the construction of a finite search space might go beyond the scope of this work.

Reviewer comment (Theory):
The proof relies on an additional assumption that the confidence interval has to shrink monotonically, which is somewhat non-standard in the analysis of BO algorithms. Though, the author claims they can apply a proof technique by Gotovos et al. (2013) even if this assumption is violated.

Author response:
```
The proof relies on an additional assumption that the confidence interval has to shrink monotonically, which is somewhat non-standard in the analysis of BO algorithms. However, the author claims they can apply a proof technique by Gotovos et al. (2013) even if this assumption is violated.
```

You are correct that our proof relies on the assumption, and as the reviewer pointed out, we can apply a proof technique by Gotovos et al. (2013) even if this assumption is violated. We can construct a new confidence interval by taking the maximum (minimum) of historical pointwise LCB (UCB). Then, the new confidence interval monotonically and consistently shrinks. Since we are aiming at a guarantee within a finite known $T$ iteration, the construction of this new confidence interval (taking union bound) would only incur a constant cost on the confidence level of the ultimate bound in theorem 1. Please note that such modification implies a slight change in the algorithm. However, we find in our empirical study that, even without these additional steps, our algorithm already exhibits compelling performance consistently across a variety of BO benchmarks.

Reviewer comment (Theory):
The Assumption 3 needs $\beta_t$ to be non-increasing. However, subsequently $\beta_t = 2 \log (2 (K + 1) |\tilde D| \pi_t / \delta)$ which is increasing in $t$, given that $\pi_t$ has to grow superlinearly (typically quadratically). Can you comment on this discrepancy?

Author response:
***Question***

```
The assumption 2 needs \beta_t to be non-increasing. 
```

We appreciate the detailed question here. The reason $\pi_t$ needs to grow is the need to guarantee the convergence of the series, which allows an asymptotical sublinear regret. Specifically, the requirement is $\sum_t{\pi_t}^{-1} = 1$. Yet, in our problem setup, since cumulative regret could not be bounded as instantaneous regret could be infinite when the evaluated candidate $x_t$ violates any of the constraints (cf. Equation (3)), we aim to guarantee that after a certain horizon T, the *simple regret* could be bounded. Therefore, there is no need to have a convergence series $\sum_t{\pi_t}^{-1} = 1$. As shown in the proof, we only need $\sum_{t=1}^T{\pi_t}^{-1} = 1$ for a given budget/horizon T, and setting $\pi_t = T$ satisfies it. We made revisions in the analysis section and the proof part to highlight this.

The setting in which $\sum_{t=1}^T{\pi_t}^{-1} = 1$ is used instead of $\sum_t{\pi_t}^{-1} = 1$ can be found in [1] as they also aim at a simple regret after a sufficiently large $T$. Similarly, in a multi-fidelity BO setting [2], the total budget is assumed to be given to attain a theoretical guarantee on the cumulative regret.

***Reference***

[1] Wang, Zi, and Stefanie Jegelka. "Max-value entropy search for efficient Bayesian optimization." In International Conference on Machine Learning, pp. 3627-3635. PMLR, 2017.

[2] Song, Jialin, Yuxin Chen, and Yisong Yue. "A general framework for multi-fidelity bayesian optimization with gaussian processes." In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 3158-3167. PMLR, 2019.

[3] Maus, Natalie, Haydn Jones, Juston Moore, Matt J. Kusner, John Bradshaw, and Jacob Gardner. "Local latent space Bayesian optimization over structured inputs." Advances in Neural Information Processing Systems 35 (2022): 34505-34518.

[4] Gardner, Jacob, Chuan Guo, Kilian Weinberger, Roman Garnett, and Roger Grosse. "Discovering and exploiting additive structure for Bayesian optimization." In Artificial Intelligence and Statistics, pp. 1311-1319. PMLR, 2017.

[5] Nayebi, Amin, Alexander Munteanu, and Matthias Poloczek. "A framework for Bayesian optimization in embedded subspaces." In International Conference on Machine Learning, pp. 4752-4761. PMLR, 2019.

Reviewer comment (Experiments):
Empirical evaluation is limited to 6 relatively low-dimensional tasks. Testing on more high-dimensional real-world problems would better showcase scalability.

Author response:
1. Thank you for raising this in the comments. Please note that the water converter is a 36-dimensional task, which is reasonably high-dimensional (i.e., d>10).

2. As the reviewer noted in the questions, the performance actually depends on the quality of the GP models. We found tuning a large number of GPs in high-dimensional, large-scale tasks to be more challenging. Especially when the prior or kernel learning is not properly tuned, the **misspecification of every single model accumulates in the ROI intersection**. Then, it is possible to result in an empty intersected ROI, which is not expected when models offer satisfying uncertainty quantification.

3. We do see the potential of further integrating additional ROI-based HDBO treatments, such as BALLET[1], into the COBALT framework to further enhance the scalability through (deep) kernel learning [6]. We want to highlight that, as noted by the reviewer, this submission **focused** on the principled tradeoff between learning and optimization in the constrained BO setting within the limited pages. Advancements in modeling and efficiency for large-scale constrained BO tasks would definitely be of interest in the future.

Reviewer comment (Theory):
The theoretical analysis relies on several assumptions that may not always hold in practice, such as the independence of the GPs and the existence of a feasible solution.

Author response:
1. This is a good point (question). Thank you for raising this concern. For the dependency assumption, we follow the convention in the constrained Bayesian optimization literature [2, 3]. As discussed in section F.3 in the appendix, we acknowledge the limitation of the current assumption and believe it is an interesting future direction to explore different dependency assumptions, which could lead to potentially novel modeling and acquisition function choices. 

2. We also appreciate the reviewer's concern about our assumption of the existence of a feasible solution, although it wasn't a primary consideration in our motivating applications. If this assumption does not hold, it implies two significant implications: (1) the task or application in question may encounter fundamental challenges; (2) as our analysis indicates, the algorithm is designed to identify a subset of the feasible region, should it exist. In the absence of such a region, the active learning component of our algorithm would converge to an empty intersected Region of Interest (ROI).

3. To provide further clarity on this aspect, we have extended our analysis in the appendix of the revised version. Our additional result further highlights the algorithm's capability to detect infeasibility and is presented as a corollary of our existing results.

Reviewer comment (Theory):
The theoretical analysis relies on assumptions like GP independence and feasible solutions. Could these be relaxed? What are the barriers to deriving more general guarantees?

Author response:
1. This is a good point (question). Thank you for raising this concern. For the dependency assumption, we follow the convention in the constrained Bayesian optimization literature [2, 3]. As discussed in section F.3 in the appendix, we acknowledge the limitation of the current assumption and believe it is an interesting future direction to explore different dependency assumptions, which could lead to potentially novel modeling and acquisition function choices. 

2. We also appreciate the reviewer's concern about our assumption of the existence of a feasible solution, although it wasn't a primary consideration in our motivating applications. If this assumption does not hold, it implies two significant implications: (1) the task or application in question may encounter fundamental challenges; (2) as our analysis indicates, the algorithm is designed to identify a subset of the feasible region, should it exist. In the absence of such a region, the active learning component of our algorithm would converge to an empty intersected Region of Interest (ROI).

3. To provide further clarity on this aspect, we have extended our analysis in the appendix of the revised version. Our additional result further highlights the algorithm's capability to detect infeasibility and is presented as a corollary of our existing results.

Reviewer comment (Experiments):
Have you tested on any high-dimensional (D>10) or large-scale real-world problems? This could better showcase scalability.

Author response:
1. Thank you for raising this in the comments. Please note that the water converter is a 36-dimensional task, which is reasonably high-dimensional (i.e., d>10).

2. As the reviewer noted in the questions, the performance actually depends on the quality of the GP models. We found tuning a large number of GPs in high-dimensional, large-scale tasks to be more challenging. Especially when the prior or kernel learning is not properly tuned, the **misspecification of every single model accumulates in the ROI intersection**. Then, it is possible to result in an empty intersected ROI, which is not expected when models offer satisfying uncertainty quantification.

3. We do see the potential of further integrating additional ROI-based HDBO treatments, such as BALLET[1], into the COBALT framework to further enhance the scalability through (deep) kernel learning [6]. We want to highlight that, as noted by the reviewer, this submission **focused** on the principled tradeoff between learning and optimization in the constrained BO setting within the limited pages. Advancements in modeling and efficiency for large-scale constrained BO tasks would definitely be of interest in the future.

Reviewer comment (Experiments):
Is there a principled way to set the β tradeoff parameter? Sensitivity analysis could elucidate its impact.

Author response:
We appreciate the constructive comment and add a brief study of the sensitivity of $\beta$ in Section E in the revision. In summary, among the choices of the theoretical results or constant values, including 0.1, 2, 4, and 8, we observe that only .1 could trap the algorithm in the sub-optimal area due to its lack of exploration. We acknowledge that there is a discrepancy in the theory and practice of $\beta$ as in the literature. In the case of COBALT, since the estimation of $\epsilon$ plays an important role while might not always be available in prior, we typically take a moderate value, as studied above, which shows robust performance.