Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: The authors use importance sampling to estimate the epistemic uncertainty of the decoder, and use this quantity to avoid latent space regions with high uncertainty when generating high-dimensional discrete data.
main_review: I believe that if certain concerns are addressed the authors method can be of practical use to important application domains. The paper is in general well-written and well-motivated.

My main concern with the paper is a fundamental one. The authors method corresponds to censoring the regions of the latent space into which not many encodings were made. While this may understandably result in more valid generated samples, is this not against the idea of using variational autoencoders in the first place? After all, when taken to extreme, the authors approach would turn the VAE to a classical autoencoder, where the latent space has no regularity or smoothness outside the already existing encodings. One of the main advantages of using a VAE is (hopefully) obtaining a 'regular' latent space, thereby allowing the creation of samples that are both novel _and_ valid. I understand that this is not always the case in terms of validity (hence the authors' paper) but the route the authors have taken runs the risk of defeating the purpose of using a VAE in the first place. After all, they might be sacrificing novelty for validity, since an autoencoder could also create very valid samples, but with no novelty at all.

A different approach that I believe the authors can compare their approach to is work on robust VAEs - I believe such approaches might be competitive with the authors' in terms of novelty / validity trade-off, remaining truer to the idea of using VAEs. Such approaches strive to create more 'smooth' and 'regular' latent spaces, example work include [1] and [2]. How would the authors compare their work to those who attempt to create a more smooth latent space?

I believe an empirical comparison would also be called for, both for the existing comparisons and a potential comparison with the approaches mentioned above. I think the authors should discuss any potentially useful metric that would characterize the novelty of the generated samples, and compare their work with previous ones in this respect as well. This comparison would especially be useful when made against robust VAE approaches.

I am looking forward to authors' response and am open to revising my score accordingly.

[1] https://arxiv.org/abs/2102.07559
[2] https://arxiv.org/abs/2012.03715
limitations_and_societal_impact: See my comments above regarding limitations. If published in NeurIPS the authors could include a section about potential ethical/societal implications of their work as a generative modeling method in the camera-ready version of their paper.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The author(s) note that modelling discrete objects with VAEs that use a continuous latent representation provides an easy way to optimize an auxiliary "design" function, which maps discrete data-space objects to a scalar fitness score, via indirectly optimizing the latent space by back propagating "design" function gradients through the VAEs decoder. In other words, this method circumvents discrete optimization methods in favor of simpler continuous optimization strategies. The author(s) correctly observe this can be problematic when such an optimization results in values that have low density under the variational aggregate posterior (i.e. latent codes that produce invalid objects since the decoder has never seen them before). The author(s) propose an importance-sampling-based estimate to measure decoder uncertainty as a way to discourage auxiliary "design" function optimization from converging on latent codes that decode into invalid objects. The author(s) integrate this uncertainty into both gradient ascent and Bayesian optimization regimes to experimentally showcase their proposals.
main_review: ### Originality:
- I am unfamiliar with the latest and greatest mutual information estimation techniques. However, I do know for certain there is a large body of related research on MI estimation. Could the author(s) please discuss which aspects of their approach are novel? From lines 103-106, it sounds the decomposition (equations 2-4) is not novel. The paper reads as if importance-sampling these quantities is novel. Can the author(s) please confirm this is the case? I think the paper would benefit on a relevant works related to MI estimation alone, since this is a major contribution of the paper.

### Quality:
- The author(s) mention a variety of related works the first paragraph of section 2.1. Yet, many of these methods do not appear as experimental baselines in section 5. Their absence makes it difficult to assess the proposals of this paper. In particular, how is a researcher to select between existing methods or the ones proposed?

### Clarity:
- While each section reads clearly on its own, I was unable to comprehensively understand the implementation of the manuscript's proposals. This difficulty is the principal reason for my low score. I hope the author(s) can address the following issues. 
- Conventional VAE lexicon uses x for data space z for latent space. Section 2.2 uses x for inputs and y for discrete outputs. Wouldn't the inputs be y as well since a VAE reconstructs the input from itself? Or is y reconstructed from x? How do x and y in section 2.2 relate to conventional VAE lexicon? The introduction encourages the reader think a VAE mindset, but the MI deviates from VAE convention.
- The author(s) want to improve optimization in the VAE latent space, but spend very little time formulating the optimization procedures in section 4. Lines 159-167 talk about several Bayesian Optimization methods but don't tell the reader which one is used.
- The paragraph spanning 168-183 seemingly discusses how to use a VAE to auxiliary a surrogate design function, yet never really discusses the specifics of how to incorporate the proposed MI estimation other than the vague, "we can further improve the
quality of the candidate set by censoring the moves in latent during gradient ascent that would result
in a value of uncertainty above a predefined threshold."
- The second and third paragraphs of section 5 address my two points above, but are very short on detail. I recommend combing these sections in an effort to provide more mathematical details of the optimization procedure (equations would be nice).
- The proposal to "compute the gradient of the auxiliary property network with respect to latent positions and accept proposal moves along these directions if the decoder uncertainty at the corresponding position in latent is below a predefined threshold (e.g., 99th percentile of decoder uncertainty on the training set" seems problematic if the latent space utilized by the training set is discontinuous, which can and does occur (see, Adversarial Auto-encoders). Are there other ways to incorporate uncertainty that allow for unconnected regions of latent space (other than doing multiple restarts)?

### Significance:
- The applications of this paper are numerous in computational biology and drug design. If it is state-of-the-art, which is unclear (see above), then it is of great significance to researchers in these fields.
limitations_and_societal_impact: The author(s) do not really address these issues. Can the author(s) please think of times their uncertainty estimation might produce false estimates or when their optimization proposals might avoid less probably but valid regions of latent space that might be associated with a less-represented group?
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper propose to improve the latent space optimization (LSO) framework through incorporating uncertainty of decoders. The current  latent sace optimization algorithms are based on (a) training an generative autoencoder of candidate solutions for the optimization problem and (b) searching over latent codes that corresponds to high objective function. Searching over latent codes that are unseen during training may lead to the decoder generating invalid outputs. The authors propose to constrain the search process based on (newly proposed) importance sampling-based estimation of decoder uncertainty. Experiments validate the useful-ness of the approach compared to the two baselines: naive LSO and LSO constrained by prior distribution of latent codes. 
main_review: Overall, I believe this paper propose a sensible approach to tackle an important problem. It seems clear that incorporating decoder uncertainty is beneficial to the latent space optimization framework. However, I believe additional experiments should be conducted to evaluate the useful-ness of the proposed importance sampling scheme to estimate decoder uncertainty.

Pros:
- This paper tackels an important problem of the latent space optimization framework that has gained interest. 
- The proposed approach is simple and sensible. 
- Experiments are conducted over various domains to demonstrate its general usage. 

Cons:
- My main concern is on the experiments missing baselines for measuring the decoder uncertainty. There are many works that propose algorithms for evaluating uncertainty of (sequential) generative model and detecting out-of-distribution samples. For example, the authors can compare to the algorithms listed in [Ren et al., 2019]. 
-  The authors should cite & discuss their position of their paper with respect to [Griffith et al., 2020] who performs a constrained Bayesian optimization for LSO. 
- In the supplemenent, it is mentioned that the authors use Monte Carlo dropout to estimate posterior distribution of the neural network parameters. I think this is an imporatant detail that should be shown in the main paper. 
- (minor) The authors could provide results on various score functions for the molecular optimization task. There are many scoring functions that are more relavent to the real applications like drug discovery. For example, the authors can consider the GuacaMol benchmark [Brown et al., 2019] or GSK & JNK [Li et al., 2019]

Question:
- I was able to understand that decoder uncertainty helps to generate latent codes that are decoded in a valid & realistic way (ratio of valid & high-quality molecules). However, I am curious what is the explanation for how improving the decoder uncertainty leads to better performance of LSO. 

[Ren et al., 2019] Likelihood Ratios for Out-of-Distribution Detection
[Griffith et al., 2020] Constrained Bayesian optimization for automatic chemical design using variational autoencoders
[Brown et al., 2019] GuacaMol: Benchmarking Models for De Novo Molecular Design
[Li et al., 2019] Multi-objective de novo drug design with conditionalgraph generative model
limitations_and_societal_impact: The authors adequately addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper considers the problem of black-box optimization of high-dimensional discrete objects where the inputs are embedded into a continuous latent space (typically using a deep generative model) and the optimization process takes place in this latent space. The main hypothesis in the paper is that existing methods for this problem might not be robust because they explore uncertain parts of the latent space with no training data in that region. The paper proposes to leverage the epistemic uncertainty of the decoder (measured in terms of the mutual information between model parameters and outputs) in order to navigate the latent space for black-box optimization. The mutual information is computed via an importance sampling estimate and employed in Bayesian optimization (BO) and gradient based search on the latent space. Experiments are performed on digit generation, arithmetic expression and molecular generation benchmarks.

main_review: - One nice thing about the paper is that the proposed approach is complementary to existing methods and can be employed in a drop-in replacement. 


- Although the importance sampling estimator gives principled uncertainty estimate, it is an extra computational overhead which can be substantial. The estimator requires sampling from the model posterior (accomplished in the paper via MC-dropout) which is already a very challenging and active research problem. Please discuss the implications of this challenging step in computing the estimator. Please provide a computational complexity analysis of using the proposed approach on top of existing methods. Some empirical analysis in terms of raw wall-clock time would be very useful. 


- The paper proposes two modifications to BO over latent space: censor points above an uncertainty threshold or training GP surrogate model by penalizing points with high uncertainty. However, in BO, the surrogate model's uncertainty is essential in exploring the function. How does the proposed method interact with the uncertainty of the surrogate model. It seems like it might push BO to over-exploit if the uncertainty threshold is not tuned properly. 


- Please provide a discussion on the criterion for selecting the right uncertainty threshold and how robust the proposed approach is to this parameter. 


- The experimental comparison in the paper is slightly weak because the baselines are a bit naive. The right method to compare for this setting is Tripp et al. [1] which also proposes a drop-in replacement approach for latent-space optimization. Please add some discussion discussing the pros and cons of the proposed approach with [1].


- A minor point but the references section can be improved significantly. Many references are missing even conference names. Please fix that.

References

[1] Tripp, A., Daxberger, E., & Hernández-Lobato, J. M. (2020). Sample-efficient optimization in the latent space of deep generative models via weighted retraining. NeurIPS, 2020
limitations_and_societal_impact: Yes, Societal impact is discussed in the appendix.
review_title: null

Meta-review: Generation of discrete objects from a conditional density (a decoder) with a continuous latent is of interest in many applications.
This work proposes to use  importance sampling to estimate the uncertainty of the decoder, "and use this quantity to avoid latent space regions with high uncertainty when generating high-dimensional discrete data." 

The key contribution of the work is an importance-sampling-based estimate to measure decoder uncertainty.
This is used to avoid latent codes that correspond to invalid objects. 

The paper is found solid and insightful by the reviewers. During the rebuttal, the authors successfully answered several concerns, and provided a detailed performance comparison with prior methods for molecular generation as well as for measuring performance on additional metric to illustrate the performance of the proposed method.

Overall, there is a consensus for acceptance and I also agree with this decision. 

Decision: Accept (Poster)