Inversion-based Latent Bayesian Optimization

Authors: jaewon  chu
jinyoung  park
seunghun  lee
hyunwoo j kim
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 13:17:14

Abstract: Latent Bayesian optimization (LBO) approaches have successfully adopted Bayesian optimization over a continuous latent space by employing an encoder-decoder architecture to address the challenge of optimization in a high dimensional or discrete input space. LBO learns a surrogate model to approximate the black-box objective function in the latent space. However, we observed that most LBO methods suffer from the `misalignment problem', which is induced by the reconstruction error of the encoder-decoder architecture. It hinders learning an accurate surrogate model and generating high-quality solutions. In addition, several trust region-based LBO methods select the anchor, the center of the trust region, based solely on the objective function value without considering the trust region's potential to enhance the optimization process. To address these issues, we propose $\textbf{Inv}$ersion-based Latent $\textbf{B}$ayesian $\textbf{O}$ptimization (InvBO), a plug-and-play module for LBO. InvBO consists of two components: an inversion method and a potential-aware trust region anchor selection. The inversion method searches the latent code that completely reconstructs the given target data. The potential-aware trust region anchor selection considers the potential capability of the trust region for better local optimization. Experimental results demonstrate the effectiveness of InvBO on nine real-world benchmarks, such as molecule design and arithmetic expression fitting tasks. Code is available at https://github.com/mlvlab/InvBO.

Main paper: 
 introduction: 
 Bayesian optimization (BO) has been used in a wide range of applications such as material science [1] , chemical design [2, 3] , ,and hyperparameter optimization [4, 5] . The main idea of BO is probabilistically estimating the expensive black-box objective function using a surrogate model to find the optimal solution with minimum objective function evaluation. While BO has shown its success on continuous domains, applying BO over discrete input space is challenging [6, 7] . To address it, Latent Bayesian Optimization (LBO) has been proposed [8] [9] [10] [11] [12] [13] [14] . LBO performs BO over a latent space by mapping the discrete input space into the continual latent space with generative models such as Variational Auto Encoders (VAE) [15] , consisting of an encoder q ϕ and a decoder p θ . Unlike the standard BO, the surrogate model in LBO associates a latent vector z with an objective function value by emulating the composition of the objective function and the decoder of VAE.In LBO, however, the reconstruction error of the VAE often leads to one latent vector z being associated with two different objective function values as explained in Figure 1 . We observe that the discrepancy between y and y ′ (or x and x ′ ) hinders learning an accurate surrogate model g and generating high-quality solutions. We name this the 'misalignment problem'. Most prior works [9, 11, 12] use the surrogate model g enc , which is trained with the encoder triplet (x, z, y), and Figure 1 : Misalignment problem. In LBO, a latent vector z can be associated with two function values y and y ′ due to the reconstruction error of the VAE, i.e., x ̸ = x ′ . (a) In Encoder triplet (x, z, y), latent vector z is associated with f (x), where x is the original input to the encoder, i.e., z = q ϕ (x). (b) In Decoder triplet (x ′ , z, y ′ ), z is associated with y ′ = f (x ′ ), which is the objective function value of reconstructed input value x ′ using the decoder, i.e., x ′ = p θ (z). The discrepancy between y and y ′ hinders learning the accurate surrogate model g. We name this the 'misalignment problem'. generate solutions x ′ via decoder p θ . Since g enc fails to estimate the composite function of p θ and f , this approach often results in suboptimal outcomes. Some works [13, 14] handle the misalignment problem by employing the surrogate model g dec trained with the decoder triplet (x ′ , z, y ′ ). However, they request a huge amount of additional oracle calls to obtain the decoder triplet, which leads to inefficient optimization. In addition, several existing LBO methods [13, 14, 16] adopt the trust region method to restrict the search space and have shown performance gain. Most prior works select the anchor, the center of the trust region, as the current optimal point. This objective function value-based anchor selection overlooks the potential to benefit the optimization performance of the latent vectors within the trust region.In this work, we propose an Inversion-based Latent Bayesian Optimization (InvBO), a plug-and-play module for VAE-based LBO methods. InvBO consists of two components: the inversion method and a potential-aware trust region anchor selection. The inversion method addresses the misalignment problem by inverting decoder p θ to find the latent code that yields x without any additional oracle call. We theoretically analyze that our inversion method decreases the upper bound of the error between the surrogate model and the objective function within the trust region. The potential-aware trust region anchor selection method selects the anchor considering not only the observed objective function value but also the potential to enhance the optimization process of the latent vectors that the trust region contains. We provide the experimental evaluation on nine different tasks, Guacamol, DRD3, and arithmetic expression fitting task to show the general effectiveness of InvBO. Specifically, plug-and-play results of InvBO over diverse prior LBO works show a large performance gain and achieved state-of-the-art performance.The contributions of our paper are as follows:• We propose the inversion method to address the misalignment problem in LBO by generating the decoder triplet without using any additional oracle calls.• We propose the potential-aware trust region anchor selection, aiming to select the centers of trust regions considering the latent vectors expected to benefit the optimization process within the trust regions.• By combining the inversion method and potential-aware trust region anchor selection, we propose Inversion-based Latent Bayesian Optimization (InvBO), a novel plug-and-play module for LBO, and achieve state-of-the-art performance on the nine different tasks.2 Related Works 
latent bayesian optimization: 
 The goal of Latent Bayesian Optimization (LBO) [8, 9, [11] [12] [13] [14] [16] [17] [18] [19] is to learn a latent space to enable optimization over a continuous space from discrete or structured input (e.g., graph or image). LBO consists of a Variational AutoEncoder (VAE) to generate data from the latent representation and a surrogate model (e.g., Gaussian process) to map the latent representation into the objective score. Some works on the LBO have designed new decoder architectures [8, [20] [21] [22] [23] to perform the reconstruction better, while other works have proposed learning mechanisms [9] [10] [11] [12] [13] [14] 17] to alleviate the discrepancy between the latent space and input space. LOL-BO [13] adapts the concept of trust-region to the latent space and jointly learns a VAE and a surrogate model to search the data point in the local region. CoBO [14] designs new loss to encourage the correlation between the distances in the latent space and objective function.Inversion has widely been applied to a variety of generative models such as Generative Adversarial Networks (GANs) [24, 25] and Diffusion models [26] [27] [28] [29] . Inversion is the process of finding the latent code z inv of a given image to manipulate images with generative models. Formally, given an image x and the well-trained generator G, the inversion can be written as:EQUATIONwhere d X (•, •) denotes the distance metric in the image space X , and Z is the latent space. To solve Eq. ( 1), most inversion-based works can be generally classified as two approaches: optimizationbased and learning-based methods. The optimization-based inversion [30] [31] [32] [33] iteratively finds a latent vector to reconstruct the target image x through the fixed generator. The learning-based inversion [25, 34, 35] trains the encoder for mapping the image x to the latent code z while fixing the decoder. In this work, we introduce the concept of inversion to find the latent vector that can generate a desired sample for constructing an aligned triplet and we use the optimization-based inversion.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: The paper provide the limitation after the Section 7.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
preliminaries: 
 Bayesian optimization. Bayesian optimization (BO) is a powerful and sample-efficient optimization algorithm that aims at searching the input x with a maximum objective value f (x), which is formulated as:EQUATIONwhere the black-box objective function f : X → Y is assumed expensive to evaluate, and X is a feasible set. Since the objective function f is unknown or cost-expensive, BO methods probabilistically emulate the objective function by a surrogate model g with observed datasetD = {(x i , y i )|y i = f (x i )} n i=1. With the surrogate model g, the acquisition function α selects the most promising point x n+1 as the next evaluation point while balancing exploration and exploitation. BO repeats this process until the oracle budget is exhausted.Trust region-based local Bayesian optimization. Classical Bayesian optimization methods often suffer from the difficulty of the optimization in a high dimensional space [36] . To address this problem, TuRBO [36] adopts trust regions to limit the search space to small regions. The anchor (center) of trust region T is selected as a current optimal point, and the size of the trust region is scheduled during the optimization process. At the beginning of the optimization, the side length of all trust regions is set to L init . When the trust region T updates the best score τ succ times in a row, the side length becomes twice until it reaches L max . Similarly, when it fails to update the best score τ fail times in a row, the side length becomes half. When L falls below a L min , the side length of the trust region is set to L init and restart the scheduling. Recently, LOL-BO [13] adapted trust region-based local optimization to LBO, and has shown performance gain.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes] Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: The paper provide anonymized code that satisfied the submission guidelines.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
method: 
 In this section, we present an Inversion-based Latent Bayesian Optimization (InvBO) consisting of an inversion and a novel trust region anchor selection method for effective and efficient optimization. We first describe latent Bayesian optimization and the misalignment problem of it (Section 4.1). Then, we introduce the inversion method to address the misalignment problem without using any additional oracle budgets (Section 4.2). Lastly, we present a potential-aware trust region anchor selection for better local search space (Section 4.3). [13, 14] solve the misalignment problem by the recentering technique that generates the aligned triplet (x ′ , z, y ′ ). However, it requests additional oracle calls as y ′ = f (x ′ ) is unevaluated, and does not fully use the evaluated function value y = f (x). (b) The inversion method (ours) aims to find z inv that generates the evaluated data x to get the aligned triplet (x, z inv , y) without any additional oracle calls.BO has proven its effectiveness in various areas where input space X is continuous, however, BO over the discrete domain, such as chemical design, is a challenging problem. To handle this problem, VAE-based latent Bayesian optimization (LBO) has been proposed [8, 11, 13, 14] that leverages BO over a continuous space by mapping the discrete input space X to a continuous latent space Z. Variational autoencoder (VAE) is composed of encoder q ϕ : X → Z to compute the latent representation z of the input data x and decoder p θ : Z → X to generate the data x from the latent z.Given the objective function f , latent Bayesian optimization can be formulated as:EQUATIONwhere p θ (z) is a generated data with the decoder p θ and Z is a latent space. Unlike the standard BO, the surrogate model g aims to emulate the function f • p θ : Z → Y. To the end, the surrogate model is trained with aligned dataset D = {(x i , z i , y i )} n i=1 , where x i = p θ (z i ) is generated by the decoder p θ : Z → X and y i = f (x i ) is the objective value of x i evaluated via the black box objective function f : X → Y. In the rest of our paper, we define that the dataset is aligned when all triplets satisfy the above conditions (i.e., all triplets are the decoder triplets explained in Figure 1 ), and the dataset is misaligned otherwise. We define the 'misalignment problem' as the misaligned dataset hinders the accurate learning of the surrogate model g. Most existing LBO works [11] [12] [13] [14] overlook the misalignment problem 1, which originates from two processes: (i) construction of initial dataset D 0 and (ii) update of VAE.Construction of initial dataset D 0 . Since initial dataset D 0 is composed of pairs of input data and its corresponding objective value {(x i , y i )|y i = f (x i )} n i=1 , LBO requires latent vectors {z i } n i=1to train the surrogate model. Most works compute a latent vector z i as z i = q ϕ (x i ) under the assumption that VAE completely reconstructs every data points (i.e., p θ (q ϕ (x i )) = x i ), which is difficult to be satisfied in every case. This results in the data misalignment (x i ̸ = p θ (z i )) during the construction of initial dataset D 0 .In LBO works, updating VAE during the optimization plays a crucial role in adapting well to newly generated samples. However, due to the update of VAE, the previously generated triplet (x, z, y) cannot ensure the alignment since z was computed by the VAE before the update. Previous LBO works [13, 14] solve the misalignment problem originating from the VAE update with a recentering technique that requests additional oracle calls to generate the aligned datasetD = {(x ′ i , q ϕ (x i ), f (x ′ i ))} n i=1where x ′ i = p θ (q ϕ (x i )) as shown in Figure 2 (Left). But, it has a limitation to consuming a huge amount of additional oracle calls while they do not update the best score (Figure 3 ). Note that prior works do not explicitly mention the additional oracle calls during the recentering technique, but it can be verified by the official GitHub code. Further details about oracle consumption of the recentering are provided in the supplement Section H.Our primary goal is training the surrogate model g to correctly emulate the composite function f • p θ : Z → Y via constructing an aligned dataset without consuming additional oracle calls. To the end, we propose an inversion method that inverts the target discrete data x into the latent vector z that satisfies x = p θ (z) for dataset alignment as shown in Figure 2 (Right). With a pre-trained frozen decoder p θ , the latent vector z can be optimized by:EQUATIONwhere x is a target data and d X is a distance function in the input space X . We use the normalized Levenshtein distance [37] as our distance function, d X , which can be applied to any string-form data.Our inversion method, however, is flexible and can utilize any task-specific distance functions, such as Tanimoto similarity [38] for molecule design tasks. To solve the Eq. ( 4), we iteratively update a latent vector z to find z inv that reconstructs the target data x. We provide the overall pseudocode of the inversion method in Algorithm 1.Input: Encoder q ϕ , decoder p θ , target data x, max iteration T , distance function d X , learning rate η, reconstruction loss L 1: Initialize z (0) ← q ϕ (x) 2: for t = 0, 1, ..., T -1 do 3:EQUATIONreturn z (t+1)end if 7: end for 8: return z (T )The initialization strategy of latent vector z plays a key role in the optimization-based inversion process. We set the initialization point of latent vector z as an output of a pre-trained encoder q ϕ (x) given target discrete data x as in line 1. We iteratively update the latent vector z with the cross-entropy loss used in VAE training in line 3 until it reaches the maximum number of iterations T . Before the iteration budget is exhausted, we finish the inversion process when the distance between the generated data p θ (z) and target data x is less than ϵ as our goal is finding the latent vector z inv that satisfies Eq. ( 4), which is denoted in line 4. The inversion method generates the aligned dataset during the construction of the initial dataset D 0 and the update of VAE to handle the misalignment problem.We theoretically show that optimizing the latent vector z to satisfy d X (x, p θ (z)) ≈ 0 with inversion plays a crucial role in minimizing the upper bound of the error between the posterior mean of the surrogate model and the objective function value within the trust region centered at z. Proposition 1. Let f be a black-box objective function and m be a posterior mean of Gaussian process, p θ be a decoder of the variational autoencoder, c be an arbitrarily small constant, d X and d Z be the distance function on input X and latent Z spaces, respectively. The distance function d X is bounded between 0 and 1, inclusive. We assume that f , m and the composite function of f and p θ are L 1 , L 2 , and L 3 -Lipschitz continuous functions, respectively. Suppose the following assumptions are satisfied:EQUATION)Then the difference between the posterior mean of the arbitrary point z ′ in the trust region centered at z with trust radius δ and the black box objective value is upper bounded as:EQUATIONwhered Z (z, z ′ ) ≤ δ.The proof is available in Section A. We assume that the black box function f , the posterior mean of Gaussian process m, and the objective function f • p θ are Lipschitz continuous functions, which is a common assumption in Bayesian optimization [39, 40] or global optimization [41] . Proposition 1 implies that the upper bound of the Gaussian process prediction error within the trust region can be minimized by reducing the distance between x and p θ (z), denoted as γ. Our inversion method reduces γ by generating an aligned dataset without additional oracle calls. In Eq. ( 6), the constant c represents the accuracy of the surrogate model, which can be improved during training. Some LBO works [12, 13] introduce regularization terms to learn a smooth latent space, implicitly reducing L 3 , the Lipschitz constant of the composite function f • p θ . CoBO [14] further proposes regularization losses that explicitly reduce L 3 with theoretical grounding. Since the surrogate model m emulates the composite function f • p θ , its Lipschitz constant L 2 can also be reduced along with L 3 .Here, we propose a potential-aware trust region anchor selection to consider both the objective function value y and the potential ability of the trust region to benefit the optimization. Previous trust region-based BO works [13, 14, 36, 42] select the anchor based on the corresponding objective function value only. However, this objective score-based anchor selection does not consider that the trust region contains latent points expected to enhance the optimization performance.We design a potential score to measure the potential ability to enhance the optimization process of the trust region. To compute it, we use the acquisition function value, which is generally employed to measure the potential ability to improve the optimization process of a given data point. Specifically, we employ Thompson Sampling, a well-established acquisition function used in previous trust regionbased methods [13, 14, 36] . Formally, the potential score of each trust region T i is computed as follows:EQUATIONwhere i is an index of the candidate anchor point, Z i cand is the candidate set sampled from the trust region T i and f is a sampled function from the surrogate model (e.g., GP) posterior.As the scale of objective function values Y = {y 1 , y 2 , ..., y n } and that of the potential ability of each trust region A = α 1 pot , α 2 pot , ..., α n pot is changed dynamically during the optimization process, we calculate a scaled potential score α i pot by adjusting the scale of α i max according to the Y :EQUATIONwhere A max = max i α i pot and A min = min i α i pot denote the maximum and minimum value of A, respectively. Y max = max i y i and Y min = min i y i indicate the maximum and minimum value of Y , respectively. Based on the scaled potential score α i scaled , the final score s i of each anchor is calculated as:EQUATIONOur final score takes into account the objective function value of the anchor (observed value) and the potential score (model's prediction) for the better local search space. Finally, we select the anchors with the highest final score s i . We summarize our potential-aware trust region anchor selection schema in Algorithm 2 of Section N. 
tasks: 
 We measure the performance of the proposed method named InvBO on nine different tasks with three Bayesian optimization benchmarks: Guacamol [43] , DRD3, and arithmetic expression fitting tasks [6, 8, [11] [12] [13] 44] . Guacamol and the DRD3 benchmark tasks aim to find molecules with the most necessary properties. For Guacamol benchmarks, we use seven challenging tasks, Median molecules 2 (med2), Zaleplon MPO (zale), Perindopril MPO (pdop), Amlodipine MPO (adip), Osimertinib MPO (osmb), Ranolazine MPO (rano), and Valsartan SMARTS (valt). To show the effectiveness of our InvBO in various settings, we also conducted experiments in a large budget setting, which is used in previous works [13, 14] . The goal of the arithmetic expression fitting task is to generate single-variable expressions that minimize the distance from a target expression (e.g., 1/3 + x + sin (x × x)). More details of each benchmark are provided in Section K. TuRBO-L 0.186±0.000 0.186±0.000 0.186±0.000 0.000±0.000 0.000±0.000 0.000±0.000 TuRBO-L + InvBO 0.186±0.000 0.194±0.002 0.202±0.001 0.000±0.000 0.024±0.017 0.212±0.092 LOL-BO 0.186±0.000 0.186±0.000 0.190±0.001 0.000±0.000 0.000±0.000 0.000±0.000 LOL-BO + InvBO 0.189±0.002 0.204±0.005 0.227±0.010 0.000±0.000 0.007±0.005 0.171±0.039 CoBO 0.186±0.000 0.188±0.002 0.191±0.003 0.000±0.000 0.000±0.000 0.000±0.000 CoBO + InvBO 0.187±0.001 0.203±0.004 0.214±0.006 0.000±0.000 0.042±0.013 0.348±0.107Figure 4 : Optimization results on Guacamol benchmark tasks. The lines and ranges indicate the average and standard error of ten runs under the same settings. A higher score is a better score.We compare to six latent Bayesian Optimization methods: LS-BO, TuRBO [36] , W-LBO [11] , LOL-BO [13] , CoBO [14] , and PG-LBO [17] . We also compare with GB-GA [45] , a widely used genetic algorithm for graph structure data. LOL-BO and CoBO employ the decoder triplet generated by the recentering technique, and the other baselines employ the encoder triplet during the optimization process. In the case of LOL-BO and CoBO, we substitute the recentering technique with the inversion method. We provide the details of each baseline in Section L.For the arithmetic expression fitting task, we follow other works [8, 11, 13, 14] to employ Grammar-VAE model [8] . For the de novo molecule design tasks such as Guacamol benchmark and DRD3 tasks, we use SELFIES VAE [13] following recent works [13, 14] . In all tasks, we adopt sparse variational GP [46] with deep kernel [47] as our surrogate model. In the inversion method, the learning rate is set to 0.1 in all experiments. The further implementation details are provided in Section M.We apply our InvBO to several trust region-based LBOs, TuRBO-L, LOL-BO [13] and CoBO [14] , and provide optimization results on two Guacamol benchmark tasks, med2 and valt. All results are average scores of ten runs under the identical settings. Table 1 demonstrates that our InvBO consistently improves all LBO models in two tasks by a large margin. In particular, in the valt task, all baseline models with InvBO demonstrate significant performance improvements and CoBO with InvBO achieves a 0.348 score gain while the baseline models without InvBO fail in optimization.More results of other baselines with InvBO on other tasks are provided in Section E.Figure 4 provides the optimization results on four Guacamol benchmark tasks including med2, zale, osmb, and valt. Each subfigure shows the number of evaluations of the black box objective function (Number of Oracle) and the corresponding average and standard error of the objective score (Best Score). Our InvBO built on CoBO achieves the best performance in all four tasks. Further results of other tasks are provided in Section F.We also conduct experiments to demonstrate the effectiveness of our InvBO on DRD3 and arithmetic expression fitting tasks and large-budget settings. The experimental results are illustrated in Fig- , respectively. Please note that both DRD3 and arithmetic expression tasks aim to minimize the score while the goal of Guacamol tasks is increasing the score. Figure 5 shows that our InvBO applied to CoBO achieves the best performance. We provide further optimization results on other tasks under the large budget settings in Section G. These results demonstrate that InvBO is effective in diverse tasks and settings.6 Analysis 
ablation study: 
 We conduct additional experiments to verify the contribution of each component in our InvBO: the inversion method (INV), and the potential-aware trust region anchor selection method (PAS). Figure 6 shows the optimization results of the ablation study on med2, zale, osmb, and valt tasks. From the figure, models with the inversion method (i.e., CoBO with INV and InvBO) replace the recentering technique as the inversion method, while models without the inversion method (i.e., vanilla CoBO and CoBO with PAS) employ the recentering technique. Notably, both components of our method contribute to the optimization process, and the combination work, InvBO, consistently obtains better objective scores compared to other models. Specifically, in osmb task, the average best score achieved by the methods with the PAS, INV and both (i.e., InvBO) shows 0.784, 0.792, and 0.804 score gains compared to vanilla CoBO, respectively.To further prove that the misaligned dataset hinders the accurate learning of the surrogate model (i.e., misalignment problem), we compare the performance of the surrogate model trained with aligned and misaligned datasets on the med2 task. Figure 7 shows the fitting results of the surrogate model trained with encoder triplets (g enc , left) and decoder triplets (g dec , right), respectively. Figure 7 demonstrates that the surrogate model g dec approximates the objective function accurately, while g enc fails to emulate the objective function. Further details of an experiment are provided in Section I.In Figure 8 , we compare the optimization results of CoBO using decoder triplets and encoder triplets on the valt and med2 tasks. Both models use the potential-aware anchor selection, and the decoder triplets are made by our inversion method. CoBO using the decoder triplets shows superior optimization performance over the CoBO using the encoder triplets on both tasks. This implies that the misaligned dataset hinders the accurate learning of the surrogate model, which leads to suboptimal optimization results, and the inversion method handles the problem by generating the decoder triplet.In Section 4.2, we provide the upper bound of the error between the predicted and ground-truth objective value. In Eq. ( 6), the Lipschitz constant of the objective function L 1 and the trust region radius δ is fixed or the hyper-parameter and the constant c can be improved by learning the surrogate model. In the end, we can reduce the upper bound of the objective value prediction error by reducing the three components: γ, L 2 , and L 3 , which implies the distance between x and p θ (z), the Lipschitz constant of function m and f • p θ , respectively. Our inversion method reduce γ by searching the latent vector z inv that satisfies x = p θ (z inv ). Previously, CoBO [14] proposed regularization losses that reduce L 3 . Since the surrogate model emulates the composite function f • p θ , these regularization losses can reduce L 2 along with L 3 .Figure 9 (left) shows the regularization losses of CoBO and our inversion method reduces the objective value prediction error. CoBO-based models (i.e., CoBO+PAS and CoBO+InvBO) employ the CoBO regularization losses, and models with InvBO (i.e., TuRBO-L+InvBO and CoBO+InvBO) employ our inversion method. TuRBO-L does not use the CoBO regularization losses nor our inversion method, and models with PAS (i.e., TuRBO-L+PAS and CoBO+PAS) employ encoder triplets.Applying the regularization losses and our inversion method reduces the objective value prediction error, respectively, but our inversion method shows a larger error reduction. The combination of regularization losses and our inversion shows the smallest prediction error, which implies our inversion method robustly complements existing methods. We provide the optimization results of each model on the med2 task in Figure 9 (right). These results demonstrate that reducing the objective function prediction error plays a crucial role in optimization performance.To further prove the importance of our potential-aware anchor selection method, we perform BO with the diverse anchor selection methods: random, acquisition α, and objective score y. Random indicates randomly selected anchors, and acquisition and objective indicate anchors are selected based on the max acquisition function value and objective score, respectively. All models use the inversion method, and the optimization results on valt and med2 tasks are in Figure 10 . Ours and objective score-based anchor selection rapidly find high-quality data compared to the random and acquisitionbased selections. However, objective score-based anchor selection shows inferior performance than our potential-aware anchor selection. This indicates that both the uncertainty of the surrogate model and objective function value need to be considered for exploration and exploitation.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: The paper provide the implementation details including the hyperparameter settings in Section 5.3, M and the details of baselines in Section L, and the details of benchmark tasks in Section K.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
conclusion: 
 We propose Inversion-based Latent Bayesian Optimization (InvBO), a plug-and-play module for LBO. We introduce the inversion method that inverts the decoder to find the latent vector for generating the aligned dataset. Additionally, we present the potential-aware trust region anchor selection that considers not only the corresponding objective function value of the anchor but also the potential ability of the trust region. From our experimental results, InvBO achieves state-of-the-art performance on nine LBO benchmark tasks. We also theoretically demonstrate the effectiveness of our inversion method and provide a comprehensive analysis to show the effectiveness of our InvBO.Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: The paper depict the errorbar and the error statistic on every performance results.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: The paper provide the computer resources including the time of excution in Section J.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: The paper provide the broader impacts after the Section 7.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [No] Justification: The paper does not include the safeguards for misusing.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: The paper provide the license in Section M and used the assets properly.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [Yes]Justification: The paper provide the code and documentation of it.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: One of the contributions of this paper is molecular design optimization, which requires careful consideration due to its unintentional applications such as the generation of toxic. We believe that our work has a lot of positive aspects to accelerate the development of chemical and drug discovery with an inversion-based latent Bayesian optimization method.The performance of the methods proposed in the paper depends on the quality of the generative model. For example, if the objective function is related to the molecule property, the generated model such as the VAE should have the ability to generate the proper molecule to have good performance of optimization.A A Proof of Proposition 1In this section, we provide the proof of Proposition 1. Proposition 1. Let f be a black-box objective function and m be a posterior mean of Gaussian process, p θ be a decoder of the variational autoencoder, c be an arbitrarily small constant, d X and d Z be the distance function on input X and latent Z spaces, respectively. The distance function d X is bounded between 0 and 1, inclusive. We assume that f , m and the composite function of f and p θ are L 1 , L 2 , and L 3 -Lipschitz continuous functions, respectively. Suppose the following assumptions are satisfied:EQUATION)Then the difference between the posterior mean of the arbitrary point z ′ in the trust region centered at z with trust radius δ and the black box objective value is upper bounded as:EQUATION)where d Z (z, z ′ ) ≤ δ.Proof. With L-Lipschitz continuity, we have:EQUATIONThus, the difference between the posterior mean of point within the trust region of length δ centered at z and the black-box objective value is upper bounded as follows:EQUATIONB Exploration of Potential-aware Trust Region Anchor Selection We measure the number of searched unique data samples for each iteration to validate the exploration ability of our potential-aware trust region anchor selection in 4.3. We compare it with the objective score-based anchor selection, which selects the best anchors based on their objective score, used in most prior works [13, 14, 36] . From the figure, our potential-aware anchor selection searches more diverse data compared to the objective score-based anchor selection on both tasks. These results demonstrate that our anchor selection method improves the exploration ability without the loss of the exploitation ability. We here provide the comparison of with and without inversion to empirically prove that the inversion generates aligned data. Given a pair of data sample x i and its corresponding latent vector z i , we measure the dissimilarity between the input x i and the data p θ (z i ) generated from the decoder. The latent vector generated with the inversion method z i is defined asz i = arg min z∈Z d X (x i , p θ (z)) (Sec- tion 4.2) The latent vector without inversion method z i = q ϕ (x i ) is constructed by feeding the input x i into the encoder q ϕ similar to prior LBO works [11, 17] .Figure 12 shows the dissimilarity comparison results without and with the inversion on med2 and valt tasks. We measure the dissimilarity as the normalized Levenshtein distance between two SELFIES tokens. The x-axis indicates the iteration, and the y-axis indicates the dissimilarity between x i and p θ (z i ). From the figure, the inversion achieves zero dissimilarity for every iteration on both tasks, which indicates that the inversion always generates aligned data. On the other hand, BO without the inversion mostly generates misaligned data. These results demonstrate the necessity of the inversion in LBO. We analyze the effectiveness of PAS along with standard BO approaches. We provide the optimization performance of TuRBO and TuRBO with PAS in the Ackley benchmark function with 40 dimension, with input ranges from [-32.768, 32.768]. Our implementation is based on the codebase of TuRBO [36] provided in the BoTorch [48] tutorial, and we use the RBF kernel with Automatic Relevance Determination (ARD) lengthscale [49] . The number of initial data is 80 and the number of total oracle calls is 1,000. Figure 13 shows that applying PAS on TuRBO consistently improves the optimization performance during the iteration. This result demonstrates that PAS is also effective in standard BO benchmark tasks.We provide further results of applying our InvBO to previous LBO works, LS-BO, TuRBO-L, W-LBO [11] , LOL-BO [13] , PG-LBO [17] and CoBO [14] on Guacamol and arithmetic fitting tasks. All results are average scores of ten runs under identical settings. Since LS-BO, W-LBO, and PG-LBO are not trust region-based LBO, we report the optimization results applying inversion (INV) only on these baselines. The experimental results are illustrated in Table 2 . The table shows that applying our InvBO to all previous LBO works consistently improve the optimization performance in all Guacamol and arithmetic expression fitting tasks. Figure 14 provides the optimization results on three Guacamol benchmark tasks including pdop, adip, and rano. The y-axis of each subfigure denotes the best-found score, and the x-axis denotes the number of the objective function evaluation. Our InvBO built on the CoBO shows the state-of-the-art performance on all three tasks. In particular, CoBO with our InvBO achieves 0.56 best score in the pdop task while the best scores of all other baselines are under 0.48.We provide further optimization results on five Guacamol benchmark tasks, adip, osmb, pdop, rano, and zale under the large budget setting to demonstrate the effectiveness of our InvBO in diverse budget settings, which are used in previous works [13, 14] . The experimental results are illustrated in Figure 15 . In the figure, our InvBO to CoBO achieves superior performance on large-budget settings. Specifically, InvBO built on the CoBO showed a large margin from the baseline and achieved a more than 0.9 best score in adip tasks. These results imply that our InvBO is effective in diverse settings and tasks.We here provide the details about additional oracle calls in the recentering technique [13, 14] . Firstly, CoBO explicitly mentioned the oracle call for recentering in L8-9 of Algorithm 1 in the paper. Secondly, LOL-BO's oracle calls for recentering can be verified by its official GitHub code. See, L188-228 of lolbo/lolbo/lolbo.py and L54 of lolbo/lolbo/latent_space_objective.py. PG-LBO 0.763±0.002 0.764±0.002 0.771±0.003 0.000±0.000 0.000±0.000 0.000±0.000 PG-LBO + INV 0.765±0.003 0.784±0.005 0.804±0.004 0.000±0.000 0.000±0.000 0.159±0.079TuRBO-L 0.762±0.000 0.762±0.000 0.762±0.000 0.000±0.000 0.000±0.000 0.000±0.000 TuRBO-L + InvBO 0.765±0.002 0.785±0.003 0.799±0.004 0.000±0.000 0.024±0.017 0.212±0.092 LOL-BO 0.762±0.000 0.769±0.002 0.777±0.003 0.000±0.000 0.000±0.000 0.000±0.000 LOL-BO + InvBO 0.775±0.002 0.797±0.006 0.807±0.005 0.000±0.000 0.007±0.005 0.171±0.039 CoBO 0.762±0.000 0.763±0.001 0.772±0.003 0.000±0.000 0.000±0.000 0.000±0. We here provide the experimental details about the Gaussian process fitting with the encoder triplets and the decoder triplets. We train the surrogate model on 300 train points and predict on 100 test points, all points are randomly sampled. The encoder triplets {(x i , q ϕ (x i ), f (x i ))} n i=1 and the decoder triplets {(x i , z i inv , f (x i ))} n i=1 share the same discrete data x, where x i = p θ (z i inv ). The dots in the figure denote the mean predictions of the surrogate model and bars denote 95% confidence intervals. Figure 17 provides the Gaussian process fitting results trained with the encoder triplets (Left) and decoder triplets (Right) that predict the training points on the med2 task. The figure implies that the Gaussian process trained with encoder triplet fails to fit the composite function f • p θ , while trained with decoder triplet emulates the function accurately.Although Bayesian optimization assumes the objective function is cost-expensive, it is still important to consider the efficiency of the algorithm. We present an efficiency analysis comparing with baseline methods [11, 13, 14, 17] , and TuRBO-L, LS-BO. We conducted experiments under the same condition for fair comparison: a single NVIDIA RTX 2080 TI with the CPU of AMD EPYC 7742. Wall-clock time comparison with the baseline model is in Table 3 . The table shows that applying our InvBO to CoBO achieves state-of-the-art performance not only under the same oracle calls but also under the same wall-clock time. We demonstrate the effectiveness of InvBO through three optimization benchmarks: Guacamol [43] , DRD3, and arithmetic expression fitting task [8, 11, 13, 14] . Guacamol benchmark tasks aim to design molecules with the desired properties, which are scored with a range between 0 to 1. The number of initial points is 100 and the number of oracle calls is 500 in a small-budget setting, and the number of initial points is 10,000 and the number of oracle calls is 70,000 in a large-budget setting. In the DRD3 benchmark task, we aim to design ligands (molecules that specifically bind to a protein) that bind to dopamine receptor D3 (DRD3). We evaluate the docking score of ligands with the Dockstring library [50] . The number of the observed data points is 100 and the number of max oracle calls is 3,000. Lastly, the arithmetic expression fitting task is to generate single-variable expressions that are close to a target expression (e.g., 1/3 + x + sin (x × x)). The number of initial access to oracle is 100 and the number of max oracle calls is 500. All tasks are assumed to be noiseless settings, and we select the initial points randomly from the dataset provided in [14] and the same initial data was used in all tasks.We compare our InvBO with six latent Bayesian Optimization methods: LS-BO, TuRBO [36] , W-LBO [11] , LOL-BO [13] , CoBO [14] , and PG-LBO [17] , and one graph-based genetic algorithm: GB-GA [45] . We adopt the standard LBO for LS-BO approach following other work [14] with VAE update during the optimization process. TuRBO leverages trust regions to alleviate the overexploration. To adapt TuRBO to the latent space, we employ TuRBO-L, LS-BO with trust region, following other latent Bayesian optimization baselines [13, 14] . W-LBO weights the data based on importance in order to focus on samples with high objective values. LOL-BO adapts the concept of the trust region to the latent space and proposes VAE learning methods to inject the prior of the sparse GP models for better optimization. CoBO designs novel regularization losses based on the Lipschitz condition to boost the correlation between the distance in the latent space and the distance within the objective function. PG-LBO utilizes unobserved data with a pseudo-labeling technique and integrates Gaussian Process guidance into VAE training to learn a latent space for better optimization.In PG-LBO, we sample the pseudo data by adding the Gaussian noise, and dynamic thresholding.Our implementation is based on the codebase of [13] . We use PyTorch3 , BoTorch4 [48] , GPy-Torch5 [52] , and Guacamol6 software packages. In the Guacamol tasks and DRD3 task [50] , we use the SELFIES VAE [13] which is pretrained in an unsupervised manner with 1.27M molecules from Guacamol benchmark. The Grammar VAE [8] is pre-trained with 40K expression data in an unsupervised manner. The dimension of latent space is 256 in the SELFIES VAE and 25 in the Grammar VAE. The size of the data obtained from the acquisition function and k are hyperparameters, which are presented in Table 4 .The learning rate used in the inversion method is 0.1 in all tasks, as we empirically observe that it always finds the latent vector that generates the target discrete data. The maximum iteration number of the inversion method is 1,000 in all tasks. The rest of the hyperparameters follow the setting used in [14] . Since arithmetic fitting tasks and Guacamol with the small budget tasks use different initial data numbers used in [14] , we set the number of top-k data and the number of query points N q same as DRD3 task, as they use the same number of initial data points.Here, we provide the pseudocode of the potential-aware trust region anchor selection method. We get the max acquisition function value, which is a Thompson Sampling, of each trust region T i given a Get a candidate set Z i cand with random points in the trust region T i around z i 3: EQUATIONEQUATIONs i ← y i + α i scaled ▷ Eq. (9) 10: end for 11: S = {s i } n 12: return z i where i is index of max (S) candidate set Z i cand in lines 2-3. After scaling the max acquisition function values, we calculate the final score of each trust region s i in lines 8-9. Finally, we sort the anchors in the descending order of the final score and select the anchor.We provide the pseudocode of CoBO applying our method, InvBO, which consists of the inversion method and the potential-aware anchor selection. The inversion method is used in lines 1 and 8, and the potential-aware anchor selection method is used in line 11. As in CoBO, we use the subset of data history, which is composed of the top k scored data and the most recently evaluated data during the optimization process in line 4. When we fail N fail times to update the best score, we train the VAE with CoBO loss L CoBO and update the dataset to be aligned using the inversion method. After that, we train the surrogate model with D t and negative log-likelihood loss L surr except for the first iteration and select the trust region anchor with the proposed potential-aware trust region anchor selection in line 11. We select N q next query points from the trust region in line 12. Then, we evaluate the next query points with the objective function and update the data history in lines 17, and 18.Input: Pretrained encoder q ϕ , decoder p θ , black-box function f , surrogate model g, acquisition function α, oracle budget T , latent update interval N fail , number of query point N q , initial dataD 0 = (x i , y i ) n i=1 , learning rate for inversion η, distance function d X 1: D 0 ← x i , z i inv , y i | x i , y i ∈ D 0 , z i inv ← Inversion x i , q ϕ , p θ , η, d X n i=1▷ Algorithm 1 2: n fail ← 0 3: for t = 1, 2, ..., T do 4:D t ← CONCAT D t-1 [-N q :], topk(D t-1 ) 5:if n fail ≤ N fail then 6:n fail ← 0 7:Train q ϕ and p θ with L CoBO , D t 8: Guidelines:D t ← x i , z i inv , y i | x i , z i , y i ∈ D t , z i inv ← Inversion x i , q ϕ , p θ , η, d X |D t | i=1 ▷ AlgorithmD new ← p θ (z i ), z i , f (p θ (z i )) |z i ∈ Z next• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.Justification: The paper conform the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).Copyright (c) 2016-Facebook, Inc (Adam Paszke) [51], Licensed under BSD-style licenseCopyright (c) Meta Platforms, Inc. and affiliates. Licensed under MIT LicenseCopyright (c) 2017 Jake Gardner. Licensed under MIT LicenseCopyright (c) 2020 The Apache Software Foundation, Licensed under the Apache License, Version 2.0. 


Decision: This paper overall is quite strong, and the reviewers are nearly unanimous about this. The authors have clearly identified a subtle but very important and, in hindsight obvious, problem with existing state of the art methods in latent space BO and their approach to solving it "feels" like the right one. This is clearly borne out in empirical evaluation, and the authors' method is clearly quite strong. It feels like essentially every software implementation of LOL-BO should probably adopt InvBO as an on-by-default feature. The fact that so many oracle calls can be saved by "in model" changes since they aren't being done directly in service of optimization is great.

Reviews: review: paper_summary: paper_summary: Latent Bayesian optimization has been tackled in this work. In order to solve an optimization problem on a continuous latent space, it utilizes auto-encoder-based neural networks. In particular, this work attempts to solve a misalignment problem in the latent Bayesian optimization. Some experimental results are demonstrated to validate the method proposed in this work.
strength_weakness: strength_weakness: - Motivation of this work is weak.
- Thorough analysis on the misalignment problem is not provided.
- Experiments are domain-specific.
questions: questions: - Does the misalignment problem certainly degrade the performance of Bayesian optimization? Is there any particular evidence?
- I think that the proposed method lets each decision focus on exploitation. What do you think about this issue?
- Equation (4) doesn't seem to be inversion. It just finds the nearest output of the decoder.
- How did you choose the dimensionality of the latent space?
- The proposed method seems to require theoretical analysis.
- Considering the nature of Bayesian optimization, which is to solve black-box optimization, the benchmarks used in this work are too domain-specific to show algorithm's performance. Under the assumption of optimizing a black-box function, the proposed method actively utilizes the information of objective functions. Do you expect that your algorithm works well in more general optimization problems?
- I cannot find the details of the neural architectures used. How did you design such networks?
- Could you elaborate on the description of Figure 3?
limitations: limitations: There are no specific societal limitations of this work.
paper_summary: paper_summary: This paper identifies and addresses an overlooked issue in several latent space Bayesian optimization methods and proposes a new trust region anchor selection method (PAS) that incorporates the "potential" of a trust region to improve optimization. Specifically, the paper proposes an inversion method to correct the misalignment between the encoding that generated a sample and the resultant encoding of the generated sample. Aligning these representations allows for a better estimate by the surrogate and improves optimization. Further, the trust region anchor selection incorporates both the observed objective value at a given point and the potential for improvement within that trust region (evaluated through Thompson sampling), which similarly improves the optimization.
strength_weakness: strength_weakness: - Methodologically the contribution is a bit weak due the fact the proposals here are largely extensions of LOLBO and CoBO.
- The Lipschitz assumption for the VAE decoder and objective function doesn't seem well motivated. This isn't a significant issue with respect to the method, but I do question the relevance of Proposition 1.
questions: questions: PAS is not specific to LSBO and can be employed wherever TuRBO-derived methods are used. Was this tried on one of the standard BO benchmarks (Robot, Lunar, etc)?
limitations: limitations: The noted limitation of InvBO being sensitive to the quality of the generative model is fair and to be expected.
paper_summary: paper_summary: The authors propose Inversion-based Latent Bayesian Optimization (InvBO), a novel approach to improve latent space Bayesian optimization (LBO) by introducing two novel components. First, to fix the misalignment problem that typically plagues LBO methods that rely on encoder-decoder models, InvBO introduces an inversion method that can be used to recover the latent code that decodes to a given data point. This allows the misalignment problem to be largely circumvented without the need for additional black-box function evaluations. Second, InvBO proposes a new strategy for the selection of the center of the trust region for trust-region based LBO. In existing LBO methods that use trust regions, the trust region centers are usually chosen to be the latent data point associated with the best objective value observed thus far. InvBO proposes a new method for trust region center selection that encourages selection of trust region centers that give the trust region the highest potential to improve local optimization performance. The authors refer to this method as “potential-aware trust region anchor selection”. By combining these two components (the inversion method and the potential-aware trust region anchor selection method), the authors show that InvBO can be applied to substantially outperform current state-of-the-art LBO methods across nine high-dimensional, discrete optimization tasks. These include some of the most difficult tasks from the GaucaMol benchmark suite of molecule optimization tasks.
strength_weakness: strength_weakness: Figure 1 colors should be changed to be more friendly to red-green color blind folks. 

It would be interesting to see an additional comparison to the Genetic Expert-Guided Learning (GEGL) method for the molecular design tasks in the results section. GEGL is an reinforcement learning (RL) method that obtains state-of-the-art performance across tasks in the GuacaMol benchmark suite (see GuacaMol results in Table 2 of their paper here https://arxiv.org/pdf/2007.04897). While I do think that this additional comparison to GEGL would strengthen the paper, I do not think that it is strictly necessary for this paper to be accepted because RL is an orthogonal method and the results currently in the paper compare to all relevant LBO baselines.
questions: questions: I am interested in your ideas for future work. Do you have plans for how you might build upon this work and continue to improve methods for LBO?
limitations: limitations: Yes.
paper_summary: paper_summary: &nbsp;

The authors propose two empirical improvements to VAE-based Bayesian optimization methodology. First, the authors propose a correction for the eponymous misalignment problem which they characterize in the paper, the idea being that the latent z corresponded to an encoded x may not correspond to the x' decoded from the same latent z. The correction proposed by the authors is based on inversion. The authors demonstrate that inversion systematically improves a suite of VAE-based Bayesian optimization methods and furthermore, include diagnostic experiments that demonstrate how the Gaussian process surrogate model fit is improved following inversion. Secondly, the authors propose an improvement to trust region-based optimization in the VAE latent space based on using "potential-aware" scoring. Given that the method is broadly applicable and the experimental validation of the approach is rigorous, I recommend acceptance with the following points for the authors to consider. I am ready to increase my score if these points are addressed.

&nbsp;
strength_weakness: strength_weakness: &nbsp;

I highlight below some points of concern with the paper. Of particular note is the justification for the potential-aware method. In terms of the sample-efficiency matters benchmark, I believe this would greatly increase the impact of the paper by showcasing the potential for VAE-based BO methods against other molecule generation approaches in the ML literature. If the authors can address these points I will increase my score.

&nbsp;

__MAJOR POINTS__

&nbsp;

1. In Proposition 1, how important is the assumption that the distance function d_\Chi is bounded on [0, 1]? 

2. The Levenshtein distance may not be the most appropriate distance for objects such as molecules which are defined using the SMILES and/or SELFIES syntax. Other distance metrics such as Tanimoto similarity [8, 9] would be more chemically meaningful.

3. I think it would be a great idea for the authors to assess their method on the sample efficiency matters benchmark [13]. While this is not a weakness of the current work per se, I list this here as seeing the performance of this method on the benchmark in comparison to other molecule generation approaches would be very interesting to see and would highly encourage me to increase my score.

4. It would be great if the authors could include the T-LBO method from [14] as a baseline as it would be interesting to understand the effect of metric learning on performance. In particular, metric learning is hypothesized to smoothen the latent function and hence make it easier to fit the GP on the latent points. It would be very interesting to understand the interplay between inversion and metric learning. In other words, does inversion directly solve the problem that metric learning is trying to address? For the arithmetic task, the results from this paper should be directly comparable if the experiment was run under the same settings.

5. For the experiments reported in Section C of the appendix, it would be great if the authors could report the Tanimoto similarity between the molecules that correspond to the SELFIES strings as this would be a much more chemically meaningful measure of similarity relative to Levenshtein distance.

6. For the "potential-aware" method of anchor selection the authors use the acquisition function value is scaled because of the effect of using multiple local GPs to model each trust region?. Do the authors have a motivation for using the sum of the objective function value of the anchor together with the scaled maximum of the Thomson sample as the scoring criterion? Presumably the \alpha term accounts for the local region and the objective function accounts for the known quality of the anchor. Why should this objective be substantially different than taking the maximum of a Thompson sample directly i.e. just keeping the \alpha term? 

7. Could the authors provide clear instructions for reproducing the experimental results in the supplied code by means of a README? Currently it is not clear how to reproduce the results.

&nbsp;

__MINOR POINTS__

&nbsp;

1. There are some missing capitalizations in the references section e.g. "Bayesian" and "Gaussian".

2. When mentioning variational autoencoders it would be worth citing the original paper [1].

3. On line 68, it may be worth mentioning that the goal of LBO is to learn a latent space to enable optimization over a continuous space from a discrete or structured input space where "structured" refers to objects such as graphs and images.

4. In the related work section it would be worth mentioning the following works on VAE-based Bayesian optimization [2-7].

5. In Proposition 1, f, m, and the composition function of f and p_\theta are assumed to be L_1, L_2, and L_3 Lipschitz continuous respectively. It may be beneficial to clarify that these functions are not 1-Lipschitz, 2-Lipschitz continuous etc. but rather the Lipschitz constants can be arbitrary.

6. The PyTorch, BoTorch, and GPyTorch papers [10-12] should be cited given that the packages are used.

7. In Section K of the appendix the citation for LS-BO is not given (presumably Tripp et al., NeurIPS 2020).

8. In Figures 5 and 6 the x-axis label, "Number of Oracle" is somewhat confusing. Perhaps "queries" would be more appropriate? Additionally, it would be great if the authors could give the number of random trials for which the error bars are reported?

9. Although it is mentioned in the main text, it might be worth adding the number of trials and the fact that the uncertainty bars are standard errors in the captions for Figures 4, 5, and 6. 

10. For Table 1, it would be great if the caption appeared above the table rather than below it.

11. The diagnostic experiment on the misalignment problem in Section H of the appendix is interesting. It would be great to report a quantitative R^2 value in addition to the plots.

12. Line 323, "This indicates that both the uncertainty of the surrogate model and objective function value need to be considered for exploration and exploitation". The same can be said about the acquisition function itself?

13. In Algorithm 2 of Section M of the appendix it would be great to explicitly provide the definition of the Calculate subroutine.

14. In Table 3 of Section I of the Appendix it would be worth stating that the rows are fixed wall clock time and fixed oracle calls respectively.

&nbsp;

__REFERENCES__

&nbsp;

[1] Kingma and Welling, Auto-Encoding Variational Bayes, ICLR, 2014.

[2] Stanton, S., Maddox, W., Gruver, N., Maffettone, P., Delaney, E., Greenside, P. and Wilson, A.G., 2022, June. Accelerating Bayesian optimization for biological sequence design with denoising autoencoders. In International Conference on Machine Learning (pp. 20459-20478). PMLR.

[3] Notin, P., Hernández-Lobato, J.M. and Gal, Y., 2021. Improving black-box optimization in VAE latent space using decoder uncertainty. Advances in Neural Information Processing Systems, 34, pp.802-814.

[4] Lu, X., Gonzalez, J., Dai, Z. and Lawrence, N.D., 2018, Structured variationally auto-encoded optimization. In International conference on machine learning (pp. 3267-3275). PMLR.

[5] Siivola, E., Paleyes, A., González, J., & Vehtari, A. (2021). Good practices for Bayesian optimization of high dimensional structured spaces. Applied AI Letters, 2(2), e24.

[6] Maus, N., Wu, K., Eriksson, D. and Gardner, J., 2023, Discovering Many Diverse Solutions with Bayesian Optimization. In International Conference on Artificial Intelligence and Statistics (pp. 1779-1798). PMLR.

[7] Verma, E., Chakraborty, S. and Griffiths, R.R., 2022. High-Dimensional Bayesian optimization with invariance. In ICML Workshop on Adaptive Experimental Design and Active Learning.

[8] Tanimoto TT (17 Nov 1958). "An Elementary Mathematical theory of Classification and Prediction". Internal IBM Technical Report. 1957 (8?).

[9] Bajusz, D., Rácz, A. and Héberger, K., 2015. Why is Tanimoto index an appropriate choice for fingerprint-based similarity calculations?. Journal of cheminformatics, 7, pp.1-13.

[10] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L. and Desmaison, A., 2019. PyTorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32.

[11] Balandat, M., Karrer, B., Jiang, D., Daulton, S., Letham, B., Wilson, A.G. and Bakshy, E., 2020. BoTorch: A framework for efficient Monte-Carlo Bayesian optimization. Advances in neural information processing systems, 33, pp.21524-21538.

[12] Gardner, J., Pleiss, G., Weinberger, K.Q., Bindel, D. and Wilson, A.G., 2018. Gpytorch: Blackbox matrix-matrix Gaussian process inference with GPU acceleration. Advances in neural information processing systems, 31.

[13] Gao, W., Fu, T., Sun, J. and Coley, C., 2022. Sample efficiency matters: a benchmark for practical molecular optimization. Advances in Neural Information Processing Systems, 35, pp.21342-21357.

[14] Grosnit, A., Tutunov, R., Maraval, A.M., Griffiths, R.R., Cowen-Rivers, A.I., Yang, L., Zhu, L., Lyu, W., Chen, Z., Wang, J. and Peters, J., 2021. High-dimensional Bayesian optimisation with variational autoencoders and deep metric learning. arXiv preprint arXiv:2106.03609.

&nbsp;
questions: questions: &nbsp;

1. In Section L, the authors state that they pretrained the SELFIES VAE with 1.27M molecules from the Guacamol benchmark. Do the authors assess whether the generated molecules are contained within the pre-training dataset?

2. What do the authors think the interplay between metric learning and inversion is?

3. On line 248 what is the particular type of approximation that the authors use for their sparse GP implementation? I note that this is also not mentioned in Section L of the appendix. From the code it would appear to be the Sparse Variational Gaussian Process (SVGP) model.

4. Why are Figures 7 and 16 different?


&nbsp;
limitations: limitations: &nbsp;

1. The use of the Levenshtein distance for diagnosing the effect of inversion on the misalignment problem may not be the best criteria for the molecule experiments. Tanimoto similarity would be more indicative of chemically meaningful differences since it is a distance metric between molecules directly and not their string representation. 

2. The aforementioned lack of justification for the potential-aware scoring criterion for the trust regions is an additional limitation for the work in its current form.

3. It would be great if instructions could be included in the codebase to reproduce the results.


&nbsp;
paper_summary: paper_summary: This paper proposes Inversion-based Latent Bayesian Optimization (InvBO), a plug-and-play module for latent Bayesian optimization (LBO) methods. The key components of InvBO are: 1) An inversion method to find latent codes z that exactly reconstruct input samples x, addressing misalignment between the latent space and input space. 2) A potential-aware trust region anchor selection method that considers both observed function values and the potential of the trust region to improve optimization. Empirically, InvBO boosts the performance of several existing LBO methods on nine tasks spanning molecule design and arithmetic expression fitting. Theoretically, the authors prove InvBO reduces an upper bound on the surrogate model's prediction error within trust regions.
strength_weakness: strength_weakness: The paper could include more discussion on the limitations and practical considerations of LBO in general. For example, what are the trade-offs compared to standard BO? How much data is needed to train an effective VAE? Some discussion is given but more would help ground the work. What are the real applications of LBO?

Proposition 1 relies on several key assumptions, like Lipschitz continuity of f and small reconstruction error d(x, p(z)). While these enable a clean result, it would help to discuss how realistic the assumptions are and what can go wrong if they are violated.

To validate INV and PAS, it is also important to implement them on LBOs, TuRBO-L, LOL-BO.

The potential-aware anchor selection mainly seems motivated by intuition. A theoretical grounding, even if loose, could strengthen this contribution. For example, can we say anything about the quality of the local optima or the regret?

The improvement on DRD3 and the arithmetic expression tasks (Figure 5a) appears smaller than on Guacamol. It would be good to comment on what makes certain tasks more or less challenging for the method.
questions: questions: Can you comment more on the limitations of LBO and InvBO for practical applications? For example, what size dataset is needed to train an effective VAE model? What happens if the dataset is small?

Proposition 1 assumes the inversion mapping exists and has a small error. Can you discuss what happens if these do not hold, for example, if the inverse mapping is ill-posed?

Can you provide any theoretical insight into why considering the acquisition function in anchor selection (PAS) helps? For example, can it be related to the quality of local optima or regret bounds?

Why is the improvement on DRD3 and the symbolic regression task smaller than on Guacamol? What properties of these tasks make them challenging?

Can you discuss Validating INV and PAS, how about implementing them on LBOs, TuRBO-L, LOL-BO.
limitations: limitations: Limitations of needing a large dataset to train the VAE

Limitations of the theoretical assumptions, and what happens if they are violated in practice

Including this discussion, even if speculative, would help contextualize the contributions for practitioners. The paper already acknowledges several mathematical assumptions. Expanding this to real-world considerations would be a positive addition.

Comments: title: Rebuttal Acknowledgement
Prepared to Champion the Paper for Acceptance Following the Rebuttal
comment: Thank you for the thorough review. 

- **[Q1 in Comment] Clarification of zero dissimilarity in Figure 12 and test data leakage.**
    
    We do not rely on traditional train or test data splits during the inversion process. Inversion is fundamentally a search algorithm designed to find an **“optimal”** latent code $\mathbf z_{\text{inv}}$ that reconstructs the target data $\mathbf x$, rather than training the model and evaluating on test data. In Figure 12, we measure the dissimilarity between $\mathbf x$ and $p_\theta(\mathbf z)$, denoted as $d_{\cal X}(\mathbf x, p_\theta(\mathbf z))$, with and without our inversion method for all observed data. Our Inversion method is designed to find a latent code $\mathbf z_{\text{inv}}$ that reconstructs the target data $\mathbf x$, by minimizing $d_{\cal X}(\mathbf x, p_\theta(\mathbf z))$. The zero dissimilarity of blue line in Figure 12 demonstrates that the inversion method consistently finds an “**optimal”** latent code $\mathbf z_{\text{inv}}$ that satisfies $d_{\cal X}(\mathbf x, p_\theta(\mathbf z_{\text{inv}}))=0$.
    
- **[Q2 in Comment] Applicability of InvBO in unseen domains.**
    
    While a pre-trained VAE cannot be directly applied to an unseen domain, this limitation pertains to most LBOs rather than our proposed InvBO. InvBO, on the other hand, is designed as a plug-and-play module applicable to any VAE-based LBO. Notably, LBO methods employ a fully unsupervised approach for pre-training VAEs, wherein the objective function is not utilized during the pre-training process. In LBO, we assume the availability of sufficient unlabeled data to pre-train a VAE for any given domain.
    
- **[Q3 in Comment] Clarification on the appropriateness of the term 'Inversion'.**
    
    - **Comparing with GAN inversion.**
    
    Both the inversion process in GANs and in our InvBO aim to find a latent code $\mathbf z_{\text{inv}}$ that reconstructs the target data $\mathbf x$. The distinction between whether a generative model defines the data distribution explicitly or implicitly is irrelevant to the appropriateness of the term `Inversion' in InvBO. In both cases, the fundamental goal remains the same: to find the latent code that generates the target data.
    
    - **Consistency between Equation (4) and Figure 2(b).**
    
    Other reviewers provided positive responses regarding the Figure as follows:
    
    > Reviewer wVbQ: Figure 2 does a nice job of illustrating the author’s inversion method.
    > 
    
    > Reviewer tYNa: Figures, tables and algorithms complement the main text well.
    > 
    
    Figure 2(b) shows we can find $\mathbf z_{\text{inv}}$ that generates the target evaluated data $\mathbf x$ by inversion method $i.e.,d_{\cal X}(\mathbf x, p_\theta(\mathbf z_{\text{inv}})) = 0$, illustrating the Equation (4). 
    
- **[Q4 in Comment] Dimensionality of latent space.**
    
    We did not adjust the dimensionality of latent space. We follow the dimensionality of latent space used in prior works [1].
    
    - Reference
        
        [1] Lee, Seunghun, et al. "Advancing Bayesian optimization via learning correlated latent space." *Advances in Neural Information Processing Systems* 37 (2023).
Thank you for your response.

> [W2, Q1] Evidence and analysis of misalignment problem

In Figure 12, how does it achieve zero dissimilarity? It seems strange. Is there any test data leakage?

Figures 7 and 8 show that the trained model of the proposed method only works for a single specific domain. Please see the concern below.

> [W3, Q6] Diversity of experimental domains and general optimization ability of InvBO

It is the most serious concern. I think that the authors misunderstood my concern. The proposed algorithm is trained on each domain, which implies that the configuration used in this work cannot be used for unseen domains. Bayesian optimization is black-box optimization, so that an optimization method can solve any problem with a small amount of inductive bias.  The authors tackled known domains accessing a true function.

If the proposed method can be applied in unseen tasks without re-training and with the same configuration, I can say that the proposed method is not domain-specific.

> [Q3] Clarification on the appropriateness of the term 'Inversion'

I don't think your answer resolves my concern. First off, "inversion" in GAN inversion is not matched to "inversion" in this work. While GAN models an implicit distribution, the proposed method is defined on an explicit representation. Moreover, Equation (4) does not align with Figure 2(b).

> [Q4, Q7] Details of the dimensionality of latent space and the Variational Autoencoder (VAE) used in experiments

Did you try to adjust the dimensionality of latent space? How does it impact on performance?

Most of my concerns haven't been resolved. I believe that the current manuscript is not ready to be published in NeurIPS.
We appreciate your thorough review. Considering the feedback, we will update the final version if our paper gets accepted.
Thank you for the constructive review. We will include your valuable feedback in the final version.
I would like to thank authors for their response and addressing the points I raised in my review. I am happy to keep my assessment of their work the same.
&nbsp;

Many thanks to the authors for their rebuttal. I believe the rebuttal has greatly strengthened the paper on two fronts:

1. The results on the sample efficiency matters benchmark are very impressive. Not only does the inversion scheme systematically improve the performance of the SELFIES VAE-BO approach, but the authors achieve a new SOTA on the zaleplon_mpo problem. This is notable because the SOTA is achieved in competition with many general black-box optimization techniques.

2. The Tanimoto similarity results provide further evidence of the efficacy of the inversion mechanism. Specifically, a Tanimoto similarity of 1 showcases that the inversion method is able to recover the same molecule. In contrast without the inversion mechanism it is not possible to recover the same molecule. This feature of the inversion mechanism is very important from a scientific standpoint whereby chemists may be interests in guarantees on recovery of the same molecule.

Given the points above, I am prepared to champion this paper for acceptance. The inversion method highlights an important pathology across all VAE-BO architectures, demonstrates systematic empirical improvement by addressing the pathology, demonstrates the mechanism of the pathology and proposed solution, and lastly, produces a new SOTA on a challenging benchmark featuring many general black-box optimizers.

&nbsp;
Thank you for your thoughtful consideration and revisiting your initial concerns. We will incorporate your valuable feedback in our final version.
Thanks for the response and the application of PAS to Rover and Lunar. With those results and the rebuttal, I think my critique of methodological simplicity is not an appropriate weakness.

With respect to W2, I still have doubts that we can safely make the Lipschitz assumption for an arbitrary BB objective that we would be interested in optimizing. Nevertheless, CoBO attempts to optimize the Lipschitz continuity of the decoder-objective composition during end-to-end retraining and whenever the conditions of Eqn. 5 are met within a trust region, Prop. 1 will hold. Its likely that Prop. 1 is more reasonable than I first believed. 

Given the results of the PAS experiment on Rover and Lunar further thought on W2 I'll update my score accordingly.