Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This work suggests a two-step lookahead Bayesian optimization strategy in the presence of inequality constraints, which is non-myopic and computationally efficient. The previous work [1] applies unreliable derivative-free optimization of acquisition function in such a constrained optimization problem, but the method proposed in this paper lets us to leverage a gradient-based optimization framework in a two-step lookahead Bayesian optimization strategy. Moreover, the proposed method supports either a sequential setting or a batch setting. Finally, the authors show the differentiability of the proposed acquisition function and experimental results.
main_review: This paper solves an interesting topic, which can be applied in many practical scenarios in the presence of constraints. Considering a constrained optimization problem with a non-myopic approach is pretty important and the method using a gradient-based optimization framework has the advantage of computational efficiency. My only concern is about numerical experiments. I think real-world experiments should be added in order to demonstrate the usefulness of the proposed method.

Pros

+ Interesting topic.
+ Well-written and well-organized paper.
+ Clear description of contributions

Cons

- No real-world experiments.

Detailed Comments

1. I would like to see the benefit of the proposed method in terms of computational cost, compared to other existing methods.
2. How many times did you repeat the experiments? I cannot find it in the paper.
3. I would like to ask if non-myopic approaches are always better than myopic approaches. I do not think that it needs to describe in the paper, but I was just curious about it.

Minor issues

* In Line 123, q should be $q$.
* For Equation (2), a period should be added.
* For the equations in Line 155 and Line 161, commas should be added.
* In Line 191, q should be $q$.

[1] R. Lam and K. Willcox. Lookahead Bayesian optimization with inequality constraints. In Advances in Neural Information Processing Systems, volume 30, pages 1890–1900, 2017.
limitations_and_societal_impact: I do not have any comments on limitations and societal impact. Please see "Main Review" for the detailed reviews.
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: The paper studies look-ahead Bayesian optimization (BO) under inequality constraints. The authors
develop a principled acquisitions for two-step look-ahead and motivate why it is necessary, especially
in the constrained setting. They propose a method to optimize the acquisition based on an importance
sampling strategy and evaluate the approach against other methods on 3 synthetic problems.

While this direction of research is promising, I do not believe the paper is ready for publication
yet at Neurips.
main_review: 1. The paper does not do a good job of motivating the problem. The authors mention two applications
in passing in lines 26/27, without sufficient explanation. For instance, can you give some concrete
instances where people have maximized model accuracy under run time and fairness constraints? What
would the fairness constraints look like?
In any case, you should delve deep into one or two of these examples, explain the reasons for the
constraints clearly, and cite other sources which have used such methods. **

2. My main concern with the paper is its empirical evaluation. For a paper where the focus is on
method and evaluation (as opposed to theory), one expects a more beefed up empirical evaluation.
This includes 1-3 real benchmarks which ideally arise out of motivating applications, more baselines,
and ablation studies.
- The authors have cited a bunch of papers in 2.1 and you should evaluate your method against them.
  It is also usually a good idea to compare against random search when evaluating BO methods. Right
  now, the paper has just 2 myopic baselines and 1 non-myopic baseline.
- It is hard to make sense of Table 1 where all results except for 2-OPTC is obtained from another
  paper. BO methods tend to be quite sensitive to hyperparameter choices so it is not clear if the
  hyperparameters were set the same way. On the same note, how did you choose your GP
  hyperparameters?
In its current form, the empirical evaluation does not make a strong case for its use in practice.
Having one or two compelling real benchmarks would also address 1 above, as it will motivate the
setting well.

3. The claim that all BO methods are myopic is misleading. While EI is admittedly myopic,
methods such as UCB and Thompson sampling are known to optimally balance exploration and exploitation
to optimize a function over several rounds. The authors' method essentially corrects the EI
criterion to work in this setting. I acknowledge that UCB methods are trickier to apply in the
constrained setting, there is a simple and intuitive TS implementation (simply sample from f and g
conditioned on the posterior, say f', g', and choose the maximizer of f' under the g'>0 constraint).
- On this note, you could also include TS as a potential baseline in your evaluation.

Other comments/suggestions
- I liked Figure 1 and the explanation that follows on why look-ahead is especially important in
  constrained BO. It would recommend bringing this early on to motivate the method.
- You might want to consider including a separate Related work section. Right now, parts of it is
  included in the middle of the introduction, and parts in Section 2.1. A separate Related work
  section, allows a reader to quickly locate and identify how your work is different from prior
  work.
- Don't use citations as nouns in a sentence. E.g. in line 90, use "XYZ et al. propose .." instead
  of "[5] proposes ..". This can be achieved via the \citet latex command.
- Do you have only one inequality constraint or multiple? Line 24 says multiple, but later on in the
  text, you use only 1.

PS:
** One application that I have encountered in practice where such it is necessary to handle
inequality constraints is drug discovery, where you wish to maximize potency, subject to constraints
on the specificity and toxicity. Ideally, you would motivate the problem set up with an application
you use in your experimental evaluation. If this is something that interests you, you could check
out the ChemBL or PubChem databases.
limitations_and_societal_impact: N/A
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper provides a batch constrained Bayesian optimization (CBO) method in a non-myopic way for continuous domain and functions. Compared to the previous method [1], the proposed approach demands less computational burden to choose where to sample. The paper proposes a constrained two-step acquisition function 2-OPT-C for both sequential and batch CBO settings and its gradient estimator for efficient optimization. The proposed method outperforms the non-myopic rollout algorithm of [1].

[1] Lam and Willcox, Lookahead Bayesian Optimization with Inequality Constraints, NeurIPS (2017).
main_review: The paper's main contribution builds on an unconstrained version of the two-step look-ahead BO [2]; however, its optimization approach uses likelihood ratios different from IPA used in [2] to overcome the discontinuity problem inherent in the reparameterization trick. This discontinuity problem creates the same optimization challenge for SAA. The observation and analysis are pretty reasonable, and the proposed approach has a novelty and demonstrates its numerical efficiency compared to IPA and SAA on a set of synthetic problems.

However, the reviewer concerns that the experimental setting is too limited because no real datasets are considered; the paper only numerically compares the approach with related methods only on a set of synthetic problems from another reference [1]. At this point, the reviewer thinks the proposed method is a promising approach but needs more work on numerical experiments as in [3]. Also, in L66, the paper argues that the proposed method is better than the rollout algorithm [1] in high-dimensional problems. It seems there is no investigation nor numerical experiment on this.

The paper is generally well-written and given the technical content. The approach appears technically sound, and the reviewer considers the resulting algorithm quite interesting.

**Conclusion**
The method shows promise, but the method is not sufficiently evaluated on real-data problems. The reviewer argues for rejecting the work in its current form.

---
**Update**: as the authors could clarify some of my concerns during the rebuttal, I raise my score to 5.

[2] Wu and Frazier, Practical two-step lookahead Bayesian Optimization, NeurIPS (2019).

[3] Gardner et al., Bayesian Optimization with Inequality Constraints, ICML (2014).
limitations_and_societal_impact: The authors addressed the limitations of their work in the paper.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper is about a making computationally affordable a non-myopic (two step ahead) acquisition function for constrained Bayesian optimization. The contribution is relying on a likelihood ratio, akin to importance sampling, to obtain the required gradient. The instability of existing methods is detailed. Comparisons are offered against myopic, sequential, batch and rollout alternatives.
main_review: The paper is clear overall. Still, clarifications are required on some statements, as well as some additional details. The empirical evaluations is limited to toy functions and a few alternatives methods.

Detailed comments and questions:

- Section 3: why not considering another batch at stage 1 after a batch at stage 0?

- Page 6, line 220: “To the best of our knowledge, we are the first to adopt the likelihood ratio method to design a gradient estimator for BO algorithms.” while this is already done in [17, section 3.2]. This requires further discussion than just the paragraph in introduction (in addition to the similarity).

- How do you find x_2^* in practice (in Algorithm 2), which is the “bottleneck” of the approach (A.3)?

- The point made in [25] (authors and editors are swapped in the citation) is close to what is said at the beginning that non-feasible points can bring information. This makes it a very pertinent candidate to add to the empirical comparison and figure 1. Random search should be added as well for reference.

- How to deal with equality constraints in this framework? What about failures (no observation possible in non feasible regions) or 0-1 constraints? See e.g., Bachoc, F., Helbert, C., & Picheny, V. (2020). Gaussian process optimization with failures: classification and convergence proof. Journal of Global Optimization, 78(3), 483-506.


Minor:
- two titles appear (Constrained Two-step Look-Ahead Bayesian Optimization and Two-step lookahead Bayesian optimization with inequality constraints)


Typos : 
section A: recomendation 
limitations_and_societal_impact: Yes
review_title: null

Meta-review: This paper presents a  two-step lookahead Bayesian optimization (BO) strategy in the presence of inequality constraints, which builds on a previous work on an unconstrained version of the two-step lookahead BO and overcomes the weakness of an earlier multi-step lookahead BO. The paper is generally well-written and given the technical content. However, most of reviewers have concerns in limit empirical evaluations. It is required to include 1-3 real benchmarks which ideally arise out of motivating applications, more baselines, and ablation studies. This work received split reviews. After the authors' response, I had more than a few communications with reviewers. Unfortunately none of reviewers are willing to champion this work.



Decision: Accept (Poster)