Generative Adversarial Model-Based Optimization via Source Critic Regularization
Authors: Michael S Yao
Yimeng Zeng
Hamsa Bastani
Jacob R. Gardner
James Gee
Osbert Bastani
Venue: NeurIPS 2024 poster (2024)
Keywords: Offline Optimization
Bayesian Optimization
Surrogate Objectives
Primary area: probabilistic_methods
TL;DR: We show that source critic adversarial networks can effectively regularize offline optimizers for generative tasks in medicine and the sciences.

Abstract: Offline model-based optimization seeks to optimize against a learned surrogate model without querying the true oracle objective function during optimization. Such tasks are commonly encountered in protein design, robotics, and clinical medicine where evaluating the oracle function is prohibitively expensive. However, inaccurate surrogate model predictions are frequently encountered along offline optimization trajectories. To address this limitation, we propose *generative adversarial model-based optimization* using **adaptive source critic regularization (aSCR)**—a task- and optimizer- agnostic framework for constraining the optimization trajectory to regions of the design space where the surrogate function is reliable. We propose a computationally tractable algorithm to dynamically adjust the strength of this constraint, and show how leveraging aSCR with standard Bayesian optimization outperforms existing methods on a suite of offline generative design tasks. Our code is available at https://github.com/michael-s-yao/gabo.

Generative Adversarial Model-Based Optimization via Source Critic Regularization
Michael S. Yao" Department of Bioengineering Perelman School of Medicine University of Pennsylvanie
Yimeng Zeng
Hamsa Bastani
Department of Computer and Information Science University of Pennsylvania
The Wharton School University of Pennsylvania
Jacob Gardner
James C. Gee Department of Radiology University of Pennsylvania
Osbert Bastani
Department of Computer and Information Science University of Pennsylvania
Department of Computer and Information Science University of Pennsylvania
Abstract
Offline model-based optimization seeks to optimize against Iearned surrogate model without querying the true oracle objective function during optimization. Such tasks are commonly encountered in protein design, robotics, and clinical medicine where evaluating the oracle function is prohibitively expensive. However, inaccurate surrogate model predictions are frequently encountered along offline optimization trajectories. To address this limitation, we propose generative ad- versarial model-based optimization using adaptive source critic regularization (aSCR) task- and optimizer- agnostic framework for constraining the opti- mization trajectory to regions of the design space where the surrogate function is reliable. We propose computationally tractable algorithm to dynamically ad- just the strength of this constraint, and show how leveraging aSCR with standard Bayesian optimization outperforms existing methods on suite of offline generative design tasks. Our code is available at htttts///githubbbcmm wwt.///caaaae-ssssooooooooo/////
Introduction
In many real-world tasks, often seek optimize the value of an objective function some search space inputs. Such optimization problems span across a wide variety of domains, including
molecule and protein design (Guimaraes al. 2017 Brown et al., 2019 Maus et al. (022)) patient
treatment effect estimation (Kim & Bastani 2021 Berrevoets et (002) Xu & Bastani 2023).
and resource allocation in public policy (Bastani et al. 2021 Ramchandan al. (021). A number
of algorithms have been explored for online optimization in these domains, including first-order
methods, quasi-Newton methods, and Bayesian optimization (Sun al. 2020).
However, many situations it may prove difficult or costly to estimate the objective function for any arbitrary input configuration. Evaluating newly proposed molecules requires expensive experimental laboratory setups, and testing multiple drug doses for a single patient can potentially be dangerous. In these scenarios, the allowable budget for objective function queries is prohibitive, thereby limiting the utility of out-of-the-box online policy optimization methods.
To overcome this limitation, recent work has investigated the utility of optimization methods in the offliee setting, where we are unable to query the objective function during the optimization
CCorrespondence to: mi chael, yaoeeeeooooommmmiiiiiieee upenn. edu, obastani@seas upenn. edu
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

process and instead only have access to set of prior observations of imputs and associated objective values; this problem can often be referred to as offline model-based optimization (MBO) (Trabucco) et al. 2021 Mashkaria et al., 2023 While one may naively attempt to learn a surrogate black-box model from the prior observations that approximates the true oracle objective function, such models can suffer from overestimation errors, yielding falsely promising objective estimates for inputs not contained in the offline dataset. As a result, offline optimization against the surrogate objective may yield low-scoring candidate designs according to the true oracle objective function-a key limitation of traditional policy optimization techniques in the offline setting (Fig.
In this work, we propose a novel offline MBO algorithm that leverages source critic models to optimize a surrogate objective while simultane- ously remaining in-distribution when compared against reference offline dataset. In this set- ting, an optimizer is rewarded for proposing op- tima that are "similar" to reference data points, thereby minimizing overestimation error and al- lowing for more robust oracle function optimiza- tion in the offline setting. Inspired by recent work on generative adversarial networks (Good. fellow et al. 2014), we quantify design similar- ity by proposing novel method that regularizes a surrogate objective model using a source critic actor, which we call adaptive source critic regu- larization (aSCR). We show how our algorithm can be readily leveraged with optimization meth- ods such as Bayesian optimization (BO) and first-order methods.
Oracle Objective
Fitted Surrogate Model
Dr = ((xx. (xX6)80)2-1
Design Space
Figure I: Naïve offline model-based optimization (MBO) TTabucco et al., [02I], which optimizes against a learned surrogate model fe trained on a fixed dataset D,, {(x,, y:) (shaded re- gion) without access to the true oracle f, often yields candidate designs score poorly using the true oracle (i.e., cross). Our method (aSCR) constrains optimization trajecto- ries to avoid these extrapolated points, instead proposing iin-iitributtin' designs (i.e., star).
(i.e., diamond) that
Contributions: Our contributions are as fol- lows: (1) We propose novel approach for MBO that formulates the task as a constrained primal optimization problem, and we show how this framework can be used to solve for the optimal tradeoff between naïvely optimizing against the surrogate model and staying in-distribution relative to the offline dataset. (2) We introduce a computationally tractable method-which we call adaptive source critic regularization (aSCR) implement this framework with two popular optimization methods: Bayesian optimization and gradient ascent. (3) We show that compared to prior methods, our proposed algorithm with Bayesian optimization empirically achieves the highest rank of 3.8 (second best is 5.5) on top-1 design evaluation, and highest rank of 3.0 (second best is 4.6) on top- 128 design evaluation across variety of tasks spanning multiple scientific domains.
2 Related Work
Leveraging source critic model feedback for adversarial training of neural networks was popularized by works such as Goodfellow et al. (2014), where a generator and adversarial discriminator "play" zero-sum minimax game to train generative model. However, such discriminators often suffer from mode collapse and training instability in practice. To overcome these limitations, AArovssy et al. (2017) introduced the Wasserstein generative adversarial net (WGAN), which instead utilizes source critic that learns to approximate the I-Wasserstein distance between the generated and training distributions. However, WGANs and similar networks primarily aim to generate samples that look in-distribution from a latent space prior, rather than optimize against an objective function. In our work, we adapt WGAN-inspiree source critic models for Wasserstein distance estimation.
Separately in the field of optimization, Brookes et al. (2019) introduced method for conditioning by adaptive sampling (CbAS) that learns a density model of the input space that is gradually adapted towards the optimal solution. However, such prior works have focused on solving low-dinensional. online optimization tasks (Hansen & Ostermeier] 1996. Brookes et al. 2019). More recently, Trabucco jet al. (222)) introduced conservative objective models (COM) specifically for offline optimization tasks; however, their method requires directly modifying the parameters of the surrogate function
2

---

during optimization, which is not always feasible for any general task. Mashkaria et al. (2023) pro- posed Black-box Optimization Networks (BONET) to learn the dynamics of optimization trajectories using a causally masked transformer model, and Krishnamoorthy et al. (2023) introduced Denoising Diffusion Optimization Models (DDOM) to learn the generative process via a diffusion model. Furthermore. Yu et al. (202T) and Chen et al. (2022) describe Robust Model Adaptation (RoMA) and Bidirectional learning via infinite-width networks (BDI), respectively. RoMA regularizes the gradient of surrogate objective models by enforcing local smoothness prior at the observed inputs, and BDI learns bidirectional mappings between low- and high- scoring candidates. Finally, Nguyen et al. (023) introduce Experiment Pretrained Transformers (ExPT) to learn general model for optimization using unsupervised methods. While these recent works and others propose promising algorithms for offline optimization tasks, they are often evaluated using expensive oracle query budgets that are often not achievable in practice- especialy for potentially dangerous tasks such as patient care and other high-stakes applications.
3 Background
3.1 Offline Model-Based Optimization
In many real-world domains, we often seek to optimize an oracle objective function fxx) over a space of design candidates X' to solve for x* x Examples of such problems include optimizing certain desirable properties of molecules in molecular design (Guimaraes et al.) 2017]
Brown et al.) 2019; Maus al.) ]022], and estimating the optimal therapeutic intervention for patient care in clinical medicine (Kim & Bastani 2021 Berrevocts et al. 2022 Xu & Bastani 2023) In practice, however, the true objective function f may be costly to compute or even entirely unknown, making it difficult to query in optimizing fxx). Instead, it is often more feasible to obtain access to a reference labeled dataset of observations from nature D, = {(x1. y1)................. .... (Xn: Fn)) where y: fx;). Optimization methods may use a variety of different strategies to leverage D,, in the offline setting (Mashkaria et al.] 2023] Krishnamoorthy et al.) 2023) Chen et al.] [2022); one common approach used by Trabucco al. 2021 and others is to learn a regressor model So parametrized by
g" argming ((x..)))D,,lll
will
(1)
as surrogate model for the true oracle objective / (x). Rather than querying the oracle I as in the online setting, we can instead solve the related optimization problem
xxg
with the hope that optimizing fo will also lead to desirable oracle values of f as well. Solving 2 is one instantiation of offline model- optimization (MBO) for which number of techniques have been developed, such as gradient ascent and Bayesian optimization (BO) (Sun et al. 2020).
Of note, it is difficult to guarantee the reliability of the model's predictions for x & D,, that are almost certainly encountered in the optimization trajectory. Thus, naïvely optimizing the surrogate objective fo can result in "optima" that are low-scoring according to the oracle objective f.
3.2 Optimization Over Latent Spaces
In certain cases, the search space X' for an optimization task may be discretized over finite set of structured inputs, such as amino acids for protein sequences or atomic building blocks for molecules. However, many historical optimization algorithms do not generalize well to these settings for number of different reasons, such as the lack of gradients with respect to the input designs to guide the optimization trajectory. Instead of directly optimizing over X, recent work leverages deep variational autoencoders (VAEs) to first map the input space into a continuous, (often) lower dimensional latent space Z and then performing optimization over instead (Tripp et al. 2020 Deshwal & Doppa 2021 Maus et al. 2022). A VAE is composed of a two components: (1) an encoder with parameters that learns an approximated posterior distribution (2(x)) for x E X, 2 E Z; and (2) decoder with parameters that learns the conditional likelihood distribution (x 2) (Kingma & Welling
2013). The encoder and decoder are co- te-trained to maximize the evidence lower bound (ELBO)
ELBO- Ez-q, [logr: p. (x|=)] DKI. [=)() || [v)]
where DKL. the Kullback-Leibh (KI.) divergence and p> 3) the prior distribution. common
choice is to set PVAE (0, I) (i.c., the standard normal distribution.. Optimization can then be
3

---

performed over the continuous latent space 2 of the VAE to propose 'latent space designs' that can be readily decoded using the decoder yp back into the original input space.
One such optimization method over VAE latent spaces is Bayesian optimization (BO), a sample- efficient framework for solving expensive black-box optimization problems (Mockus T982 Osboree cet al. 2009) Snoek et al. 2012). While the utility of BO has primarily been explored for expensive- lo-evaluate black-box functions in prior literature, recent work has shown that BO also outperforms baseline optimization methods in offline tasks involving models that are relatively inexpensive to evaluate, such as the neural network surrogates used in model-basee optimization (MBO). Multiple prior works have shown that BO and related methods consistently outperform both first-order gradient- based and stochastic evolutionary methods (Eriksson al.) 2019b] Maus et al. [022] Hvarlner et al.)
2024
Eriksson & Jankowiak 2021 Astudillo & Frazier 2019).
3.3
Wasserstein Metric Between Probability Distributions
The Wasserstein distance is distance metric between any probability distributions, and is closely related to problems in optimal transport. We define the = Wasserstein distance between reference distribution P and generated distribution Q using distance metric d(:,-)
Q) = inf TEr(P.Q)
d(2
(4)
where I is the set of all couplings between P and Q. For empirical distributions where pn, (g.) is based on observations { ((=))=1). (4) can be simplified
Wi (P (n) inf
11 =/l
where the infimum is over all permutations of n elements. Leveraging the Kantorovich-Rubinsthi can be equivalently written as
duality theorem (Kantorovich & Rubinstein 1958), (5)
1 W1 (P (n) = K ||ell
sup E. Be-r(e(2)) - [Z==][(()]]] [c(=)]
(6)
where c(z) is a source critic and ||c||z is the Lipschitz norm of c(=). In the Wasserstein GAN (WGAN) model proposed by Arjovsky al. (2017), generative network and source critic are co- trained in a minimax game where the generator (critic) seeks to minimize (maximize) the Wasserstein distance W2 between the training and generated distributions. Such an optimization schema enables the generator policy to learn the distribution of training samples from nature.
Framework for Generative Adversarial Optimization
In this section we deseribe our proposed framework for generative adversarial model-based optimiza- tion using adaptive source critic regularization (aSCR). Our method uses a K K-Lipschitz source critic model to dynamically regularize the optimization objective avoid extrapolation against the proxy surrogate model fo in offline MBO.
4.1 Constrained Optimization Formulation
In offline generative optimization, aim to optimize against surrogate objective function fo- In order to ensure that we are achieving reliable estimates of the true, unknown oracle objective, we can add regularization penalty to keep generated samples "similar" to those from the training dataset of fo according to an adversarial source critic trained to discriminate between generated and offline samples. That is, in contrast 2
aSCR instead considers a closely related constrained problem
minimize subject to
- fe(zz)
Egrer(c - c² <
over some configuration space 2 C RRd, and where we define o² as source critic model that maximizes (=')) [zeol("(<)) over K-Lipschitzz functions as in (6). We can think of
Eveplc' c²) as the contribution of particular generated datum to the overall p 1

---

Wasserstein distance between the generated candidate (Q) and reference (P) distributions of designs as in (6). In practice, we model c* as fully connected neural net. Intuitively, the imposed constraint restricts the feasible search space to designs that score at least as in-distribution as the average sample in the offline dataset according to the source critic. Therefore, c* acts as an adversarial model to regularize the optimization policy. Of note, this additional constraint in 7 may be highly non-convex for general c², and so it is often impractical to directly apply 77 to any arbitrary MBO policy.
4.2 Dual Formulation
To solve this implementation problem, we instead look to reformulate in its dual space by first considering the Lagrangian L of our constrained problem:
C (2: (z + [Exeplc" -2') c²()
(8)
where À 1 is the Lagrange multiplier associated with the constraint in 77). We can equivalently
think of À as hyperparameter that controls the relative strength of the source critic-penalty term: = equates to naïvely optimizing the surrogate objective, while > 1 asymptotically approaches WGAN-like optimization policy. Minimizing L thus minimizes relative sum of fe and the Wasserstein distance contribution from any particular generated datum z with relative weighting dictated by the hyperparameter A. From duality, minimizing L over and simultaneously maximizing over A E R_ is equivalent to the original constrained problem in (7).
The challenge now is in determining this optimal value of X: if À is too small, then the objective estimates may be unreliable; if À too large. then the optimization trajectory may be unable to adequately explore the input space. Prior work by [rabucco et al. (202)) has previously explored the idea of formulating offline optimization problems as a similarly regularized Lagrangian (albeit with a separate regularization constraint), although their method tunes static hyperparameter by hand. In contrast, aSCR treats À as a dynamic parameter that adapts to the optimization trajectory in real time.
4.3 Computing the Lagrange Multiplier À
Continuing with our dual formulation of (7). the Lagrange dual function g(A) is defined as g()) = infzer= L (2; A). The = å that minimizes the Lagrangian in the definition of g is evidently function of X. To show this, we use the first-order condition that V,f- = 0 at z = 2. Per we have
V A(; = - - c = 0
(9)
In general, solving 99) for 2 is computationally intractable especially in high-dimensional problems. Instead, we can approximate é by relaxing the condition in p according to
; fo(z) XV c (2)]]²
a(A) argmin =ER"
(10)
Our key insight is that although minimizing the loss term (T) is not practical when the feasible set is naïvely uniform over R", we can instead choose to focus our attention on latent space coordinates with high associated probability according to the VAE prior distribution PvAe(2). This is because in optimization problems acting over the latent space of any variational autoencoder, the majority of the encoded information content is embedded according to PVAE(2) due to the Kullback-Leibe (KL) divergence contribution to VAE training. Put simply, the encoder distribution go (2(2 is trained so that ( (go [x)x))lPvEE (2))) ] is optimized as a regularization term in (3). We argue that it is thus sufficient enough to approximate a(A) using a Monte Carlo sampling schema with random samples ZN (Z), 72, 2N) ~ pvAE(=):
((A) R argmin ] - V - cV (2)]
(11)
Z (
2
We can now concretely write an approximation of the Lagrange dual problem of (7):
maximize g(A) = - (2) + > [EErer (=")) - c (2))
(12)
subject to A x 0
where 2 is in
Defining the surrogate variable a such that À = we rewrite [2 as
maximize
- - ()(() a (x')
(13)
subject to 05a<1
5

---

In practice, we discretize the search space for a to 200 evenly spaced points between 0 and inclusive. From weak duality, finding the optimal solution to (12 provides a lower bound on the optimal solution to the primal problem in 7 Algorithm[] | can now be used to choose the optimal a (and hence X) adaptively during offline optimization: we refer to our method as Adaptive SCR (aSCR).
4.4 Overall Algorithm
Using Adaptive SCR, we now have a proposed method for dynamically computing a (and hence the Lagrange multiplier A) of the constrained optimization problem in 77. Importantly, aSCR can be integrated with any standard function optimization method by optimizing the Lagrangian objective in (8) over the candidate design space as opposed to the original unconstrained objective fe. We refer this algorithm as Generative Adversarial Model-Based Optimization (GAMBO). To evaluate aSCR empirically, we instantiate two flavors of GAMBO: (1) Generative Adversarial Bayesian Optimization (GABO, Algorithm 2 and (2) Generative Adversarial Gradient Ascent (GAGA)
We implement GABO using a quasi-expected improvement (qEI) acquisition function, iterative sampling budget of T = 32, sampling batch size of b 64, and GAGA using a step size of f = 0.05, T= 128, and b = 16. Of note, the optimization objective using aSCR is time-varying and causally linked to past observations made during the optimization process via intermittent training of the source critic c. Prior works from Nyikosa et al. (2018) and [Aglittti et al. (2022) have examined optimization against dynamic objective functions, although have either entirely disregarded causal relationships between variables or only examined causality between inputs as opposed to inputs and the objective. We leave such methods for future work given that aSCR works well in practice.
5
Experimental Evaluation
5.1 Datasets and Tasks
To evaluate our proposed algorithm, we focus on a set of eight tasks spanning multiple domains with publicly available datasets in the field of offline model-based optimization. (1) The Branin function is well-known synthetic benchmark function where the task is to maximize the wwo-dimensional
Branin function for : |-5, 10 x [0, 15 + R. (2) The LogP task is a well-studied optimization
2021 lam--Sheherrd et al.) 2022) where search over candidate molecules to maximize the penalized water-octanol partition coefficient (logP) score, which is an approximate measure of molecule's 's hydrophobicity (Ertl & Schuffenhauer 2009) that also rewards structures that can be synthesized easily and feature minimal ring structures. We use the publicly available Guacamol benchmarking dataset from Brown al. (2019) to implement this task.
problem (Zhou et al. 2019 Chen et al.
Tasks (3) - (7) are derived from Design-Bench, a publicly available set of MBO benchmarking tasks (Trabucco et al. 2022): (3) TF-Bind-8 aims to maximize the transcription factor binding efficiency of an 8-base-pan pair DNA sequence (Barrera et al. 2016); (4) GFP the green fluorescence of 237-amino-acii protein sequence (Brookes et al. [0199] Rao et al. 2019); ; (5) UTR the gene expression from a 50-base-pain pair 5'UTR DNA sequence (Sample et al., 2019 Angermueller et al.,
2020); (6) ChEMBL the mean corpuscular hemoglobin concentration (MCHC) biological response of a molecule using an offline dataset collected from the ChEMBL assay CHEMBL3885882 (Gaulton y and (7) D'Kitty the morphological structure of the D' Kitty robot (Ahn al. 2020).
et al.
Finally, (8) the Warfarin task uses the dataset of patients on warfarin medication from Consortium (2009) to estimate the optimal dose of warfarin given clinical and pharmacogenetic patient data. Of note, in contrast to tasks (1) (7) and other traditional MBO tasks in prior literature (Trabucco) et al.) 2022), the Warfarin task is novel in that only subset of the input design dimensions may be optimized over (i.c., warfarin dose) while the others remain fixed as conditioning variables (i.c., patient covariates). Such task can therefore be thought of conditional model-based optimization.
5.2 Policy Optimization and Evaluation
For all experiments, the surrogate objective model fo is fully connected net with two hidden layers of size 2048 and LeakyReLU activations. fo takes as input a VAE-encoded latent space datum and
We detail the explicit algorithmic formulation for GAGA in Supplementary Algorithm

---

returns the predicted objective function value as output. The VAE encoder and decoder backbone architectures vary by MBO task and are detailed in Supplementary Table AT Following [ Bombarelli et al. (2018) and Maus et al. (2022), we co-train the VAE and surrogate objective models together using an Adam optimizer with learning rate of x 10-1 4 for all tasks. For the optimization tasks over continuous design spaces (i.e., Branin, Warfarin, and D Kitty), we fix the VAE encoder and decoders as the identity functions, such that the latent and input spaces are equivalent.
The source critic agent с in 7 is imple- mented as a fully connected net with two hidden layers with sizes equal to four (one) times the number of input dimensions for the first (second) layer. To constrain the I. Lipschitz norm of e as in (6). we clamp the weights of the model between [-0.01, 0.01] after each optimization step as done by [Ar- jovsky et al. (0017). The model is trained using gradient descent with a learning rate of 0.001 to maximize the Wasserstein dis- tance between the dataset and generated can- didates in the VAE latent space.
Algorithm 1 Adaptive Source Critic Regularization (SCR)
Input: differentiable surrogate objective fo : R - R, differentiable source critic e R + IR, reference dataset D. = [2 a step size Acx search budget B, norm threshold T
Sample candidates 2B t- { ~ (0, Ia)
Initialize a +- None and g" -
for in range(start = 0, end = 1, stepsize = Aa) do +
=" argmin 2. [I if - fol + continue end if
aVc(=")) > then
// Discard a if best norm exceeds T
g < (1 (1=) + a [ c(=)]
During optimization, both GABO and GAGA alternate between sampling new de- signs and training the souree critic actor c(z) until there is no improvement to the Wasser- stein distance Wi according to e after 100 consecutive weight updates. We find that training e every Vegenerator = 4 sampling steps is good choice across all tasks assessed, similar to prior work (rrovssky et al. (2017).
if g 2>gg g" then
a' + a and 9" +
I/ Implements 13
end if end for return o
Algorithm 2 Generative Adversarial BayesOpt (GABO)
Input: surrogate objective fe I - R. offline dataset Dr = {ez Le acquisition function a, iterative sampling budget T, sampling batch size b, number of generator steps per source critic training Agemmestre, oracle query budget k AdaptiveSCR Input: step size Ac, search budget B, norm threshold T
All MBO methods were evaluated using fixed surrogate query budget of 2048. We
focus on two evaluation metries: 100th per- centile (1) top à = 1; and (2) top k = 128 oracle score. The top k = 128 evaluation metric is commonly reported in prior offline
Define: Differentiable source critic e: Rdd - R
Define: Lagrangian L(2; Rd x R + R
Eq.
8
[(z;a) = ff(())
[ B lo( c(:)
MBO literature (Mashkaria et al. [2033 Tra- bucco et al. | Yu et al.] (02 }); the top k ==1 metric better accounts for the limited oracle query budget of the real-world tasks in which offline MBO would be of use. In both settings, an optimizer selects the top k design that minimize the Lagrangian fune- tion value in (8) from the 2048 assessed de- signs to evaluate using the true oracle func- tion, and the maximum score of those ks de- signs is reported across 10 random seeds.
Sample candidates 2 { R- SobolSequence
// Train the source critic per Eq.
66) to optimality:
c 4 argmax Wi (D... 2t)
= aggmax|iell, [E [c(=') - Ee [c{=)]]
a t AdaptiveSCR c, Dn, Aa, B, T)
Alg.
Evaluate candidates ) +- [v: ) = {- C(2]; (x);;)))))))) {))
Place Gaussian Process (GP) prior on fo for t 2, ....., do
Update posterior on fa with D.. = 112Th )"
Compute acquisition function a using fitted posterior
Sample candidates 2' <- {2 according to
o AdaptiveSCR Da; Aa, B, 7)
We evaluate both GABO and GAGA against number of pre-existing baseline algorithms on one internal cluster with NVIDIA RTX A6000 GPUs We include vanilla Bayesian Optimization (BO-qEI) and gradient ascent (Grad.) in our evaluation to assess the utility of our proposed aSCR algorithm. Furthermore, we evaluate limited-memory BEGS (L-BFGS) Liu & Nocedal (1989). AMA-ESHannsen & Ostermeier (1996), and simulated annealing (Anneal) Kirkpatrick et al. (188) We also compare our method against the
Evaluate samples y + {u/ )
(-((x); a))
if l mod Vigeneraee equals 0 then
// Train the source critic per Eq. 6 to optimality:
c + arggmax ||cl (1
= =argmax |el BRADDA [c(2')] E.- [c()]]
end if end for return the top samples from the T b observations
Dr = [[(24". yy""" "/"") ) 1 according to y
more recently introduced algorithms TuRBO-qEL (Eriksson et al. 2019a). COM (Trabucco et al.
7

---

2021). RoMA ( Yu et al. 2021 BDI (Chen et al. 2022), DDOM Krishnamoorthy et al. (023). BONET (Mashaaria et a.. 2023), ExPT Nuyen et al.
2023). ROMO (Chen et al. (023). and BootGen (Kim et al. 2023). Because BootGen is proposed by Kim et al. (2023) as an optimization method specifically for biological sequence design, we only assess this baseline method on the five
relevant tasks in our evaluation suite.
Conditional MBO Tasks. To our knowledge, prior work in conditional model-based optimization is limited, and so previously reported algorithms are not equipped to solve such tasks out-of-the-box. Chen al. (203) explore such tasks in their work, but primarily focus on conditional tasks that are built by arbitrarily fixing certain design dimensions from unconstrained problems, which are not representative of true conditional optimization problems in the real world. In our work, we introduce the Warfarin task to assess methods on their ability to design an optimal therapeutic drug regiment conditioned on a fixed patient state and lab values. To assess existing methods on this task, we implement conditional proxies of all baselines employing first-order optimization schema via partial gradient ascent to only update the warfarin dose dimension while leaving the patient attribute conditional dimensions unchanged. Conditional BO-based methods are implemented by fitting separate Gaussian processes for each patient. In conditional DDOM, we exchange the algorithm's 's diffusion model- -based backbone with conditional score-based diffusion model (Gu et al. (023).
Of note, the BONET algorithm (Mashkaria al., 2023 requires multiple observations for any given patient to construct synthetic optimization trajectories. However, the key challenge in conditional MBO that each condition (i.e., patient) has no past observations (i.e., warfarin doses). and instead relies on learning from offlin datasets constructed from different permutations of condition values As result, the BONET algorithm is unable to evaluated on conditional MBO tasks.
5.3 Main Results
Scoring of one-shot optimization candidates is shown in Table I Across all eight assessed tasks spanning a wide range of scientific domains, GABO with our aSCR algorithm achieved the best average rank of 3.8 when compared to other existing methods (next best is 5.5). Furthermore, GABO was able to propose top h = candidate designs that outperform the best design in the pre-existing offline dataset for 6 of the tasks greater than any of the other methods assessed. If larger oracle evaluation budget is available (i.c., = 128). GABO with aSCR performs even better, achieving the best average rank of 3.0 (next best is 4.6). GABO is also the best algorithm on 3 of the 8 tasks and second best on 2 tasks according to this evaluation metric. Altogether, our results suggest that GABO is a promising method for proposing optimal design candidates in offline MBO.
Importantly, our aSCR algorithm improves upon both the naive BO-qEI and Grad. Ascent parent optimizers assessed. GABO outperforms both baseline BO-based optimization methods in our evaluation suite: BO (TuRBO) only achieves a rank of 8.8 (9.0) on the top and rank of 6.6 (7.4) on the top l 128 metric. Similarly, GAGA scores an average rank of 7 7.4 (7.6) on the top k = (k = 128) evaluation metric; by leveraging aSCR, GAGA outperforms its base parent optimizer (Grad. Ascent), which only achieves average rank of 9.0 and I 11.0 the same two evaluation metrics, respectively. Our results show that using aSCR to adaptively penalize the objective of two popular optimization methods can improve their offline performance.
evaluation metric
Qualitative Evaluation: Penalized LogP Task. We evaluate GABO against naive BO-qEI for the LogP task by inspecting the three-dimensional chemical structures of the top-scoring candidate molecules. As a general principle, molecules that are associated with high Penalized LogP scores are hydrophobic with minimal ring structures and therefore often feature long hydrocarbon backbones (Ertl & Schuffenhauer 2009). In Figure 2 we see that BO-qEI using the unconstrained surrogate objective generates candidate molecule of hydrogen and carbon atoms. However, the proposed candidate includes two rings in its structure, resulting in a suboptimal oracle Penalized LogP score.
We hypothesize that this may be due to a lack of ring-containing example molecules in the offline dataset, as only 6.7% (2.7%) of observed molecules contain at least one (two) carbon ring(s). As a result, the surrogate objective model estimator returns more inaccurate Penalized LogP estimates for input ring-containing structures (surrogate model root mean squared error (RMSE) = 25.5 for offline dataset molecules with at least 2 rings; RMSE = 16.5 for those with at least 1 ring; and RMSE = 4.6 for those with at least 0 rings), leading to sub-par BO-qEI optimization performance as the unconstrained algorithm extrapolates against the surrogate to find "optimal" molecules that
8

---

Table 1: Constrained Budget (k = 1) Oracle Evaluation Each method proposes a single design that is evaluated using the oracle function to report the final score (higher is better) across 10 random seeds reported as mean + standard deviation. DD (best) reports the top oracle value in the task dataset. Each of the MBO methods are ranked by their mean one-shot oracle score, and the average rank (lower is better) across all eight tasks is reported in the final table column. Bold (Underlined) entries indicate the best (second best) entry in the column. "Denotes the life sciences-relate discrete MBO tasks from Design- Bench TTabucco et al. 2022).
Method
Branin
Log!
TF-Bind-8
GFF"
UTR'
CLEMBL
D'Kitty
Warfarin
Rank
(hest)
-13.0
11.3
0.439
3,53
7.12
0.61
0.88
-0.19 1.96
Gma..
-24..1
81.3
-5.77 +: 1.44
0.429 + 0.023
3.18 +: 0.88
6.82 +: 0.21
-1.95 + 0.00
0.57 4: 0.19
0.86 = 1.09
9.0
L-BRGS
-29.6
22.6=00.00
3.22
32.6
0.527 + D. 0.140
3:51 + 0.70
6.48 +: 1.20
-1.95 + 0.00
0:31 + 0.00
0.73 = 1,83
8.5
CMA-ES
3.66
0.131
Anneal
.9.5 1.5
8.76 4 0.15
0.807 +: 0.094
3.64 4: 0.03
5.01 1: 0.31
-1.95 4: 0.00
0.55 4: 0.18
0.91 = 0.08
688
BO
-11.0
-52.5 +: 88.8
0,586 1: 0.193
1.43 1: 0.00
5.65
1: 1.30
0.59 = 0.10
0.61
4: 0.15
0.16 = 1.67
8.8
TuRBO
-21.0 5.11
-45.1 + 93.8
0.564 +: 0.194
1.43 + 0.00
6.53
1.19
0.68 0
0.44 4: 0.18
0.05 0.11
BONET
-26.1 0.9
10.8
0.33
0,282 +: 0.000
3.74 +: 0.00
9.12 1: 0.87
0.55 0.13
D.78 +: 0.00
5.7
DDOM
-6677
1: 6360
-1.23
1: 1.28
0.160 +: 0.030
1.43 1: 0.00
5.56
0.00
0.51= 0.15
0.51 1: 0.20
-0.32 0.40
11.1
COM
.3099
32.6
30.8 19.5
0,439 +: 0,000
3.62 +: 0.00
6.65 0.43
0.63
0.01
0.90 1: 0.02
0.72 =: 0.97
RoMA
-32.7 +: 18.4
6.37
1.39
0A33 + 0.040
3.37 + 0.27
0,6
0,50 0.14
0.30 0.27
-0.70 0X02
BDI
-1050 0.0
-0.20 1: 0.00
0,311 + 0.000
3.26 : 0.82
5.61 0.00
0.48 0.00
0.67 1: 0.00
-21.8 1: 253
10.8
EXPT
-57.2 38.6
-15,9 +: 24.1
0.571 1: 0.076
1.43 1: 0.00
6.77
+: 1.38
0,56 0.06
0.66 1: 0.20
-34.6 61.44
BontCen
-13.0
15.1
0,942 0.022
3.10 + 0.73
8.30 0.93
0.59 0.07
ROMO
-2614 + 739.9
-20.5 + 19.2
0,382 + 0,203
3,55 :: 0.13
3.75 2: 1,42
0.65
0.00
0.64 + 0.27
-0.71 + 2.10
9.6
GAGA
2.2
-68.6 109.8
0.571
+: 0.
3.74 0.00
122
-1.95 +: 0.00
0.89 0.00
0.14
GABO
-2.6 1: 111
21.3
0.570
4: 0.131
3.60 1: 0.40
7.51 1: 0.39
0.60 0.07
0.71 1: 0.01
0.50 =: 1.50
3.8
Table 2: Relaxed Budget (k- = 128) Oracle Evaluation Each method now proposes 128 designs that are evaluated using the oracle function, and maximum score out of these 128 designs is reported below (averaged across 10 random seeds and reported as mean + standard deviation). D (best) reports the top oracle value in the task dataset. Each of the MBO methods are ranked by their mean k 128-shot oracle score, and the average rank (lower is better) across all eight tasks is reported in the final table column. Bold (Underlined) entries indicate the best (second best) entry in the column.
Denotes the life sciences-relatee discrete MBO tasks from Design- TTabucco et al. 2022).
Method
Branin
LogP
TF-Bind-8'
GFF
UTR'
CBEMBLE"
D'Kitty
Warfarin
Rank
D (best)
-13.0
11.3
01439
3:53
7.12
0.61
0.88
-0.19 1.96
Grud.
-11..3 20.8
-5.14 :: 1.70
0,977 0.025
3:49 0.69
7,38 0.15
-1.95 1 0.00
0.87 : 0.02
0.86 +: 1.08
11.0
L.BGGS
4.0
0.0
42.8 à: 9.44
0,633 1: 0.140
3,74 2: 0.00
7.51
0.39
-1.95 4: 0.00
0.31 = 0.00
0.75 + 1.67
10.1
CMA-ES
47.6
0,810 0.235
3.74 0.00
0.55
+
0.00
-8.22 63.8
Anneal
-7.422 2.8
11.3 0.00
0,890 0.035
3,72 0 0.00
7,96
0.22
-1.95 1: 0.00
0.88 0.00
0.97 : 0.08
9.3
BO
-0.4 0.0
138.3 :: 16.0
0.942 1: 0.025
2.26 2: 1.03
8.26
11611:000 0.09
0.57 = 0.00
0.72 2 0.00
0.93 4: 0.11
6.6
TuRRO
.0.7 + 0.4
59.7
0.895 0.049
0.11
0.67 0.01
0.72 0.00
0.99 0..1
BONET
-.6.0:1:099
11.7 + 0.38
0,951 0.035
3.74 0.00
2.13+:0000
0.67 0.01
0.95 = 0.01
5.6
DDOM
-18.4
29.8
-2.16 1: 0.60
0.936 1: 0.051
1.44 0.00
8.30 0.33
0.56 = 0.01
0.89 2 0.01
1.00 1: 0.00
8.4
COM
-1981
224.5
42.0
0.902 0.056
0.00
8.18
0.54 0.01
0.95 0.02
0.77 + 0.88
RoMA
-18 183:33000
10.8 : 0.78
0,760 0.113
3.74 1:74=000 0.00
8.12
0.69 0.03
1,02
0.04
0.67 + 0.05
7.8
BDI
-5.00
1.52 2: 5.79
0.735 1 0.086
3.61 2: 0.05
6,31 0:000
0,50 0.12
0.94
2 0.01
-5.07 2: 21.0
11.8
EXPT
-1.7 1.0
46.88 4.58
0.927 0.095
3.74 0.00
8.13 0.09
0.68 0.04
0.97 0.01
0.96 +: 0.05
BootGen
8.10 +: 3.31
0.979 0.002
3.74 0.00
10.5 + 0.95
0.68 0.00
ROMO
-2367 = 787.5
-6.05 :: 14.5
0.572 1 0.202
3.67 0.03
6.94 1 1.07
0.65
0.00
0.90 = 0.02
0.76 +: 1.91
12.1
GAGA
-1.0 0.2
14.1
0.722
0.091
7.96
0.01
0.95 1: 0.07
GABO
4.5
122.1 d: 20.6
0.954 0.025
3.74 0.00
8.36 J: 0.08
0.70 0.01
0.72 A 0.00
1.00 1: 0.03
3.0
are ouito--diitribution. In contrast, GABO generates a candidate molecule with a long hydrocarbon backbone and no rings, resulting in a penalized logP score of 22. than the best observed value in the offline dataset for the task.
Ablation Experiments. Taking inspiration from (Trabucco et al. 2021), it is possible to utilize our SCR algorithm in GABO without dynamically computing a (and hence the Lagrange multiplier A). To better characterize the utility of aSCR, we ablate Algorithm I by treating X instead as a hand-tunable constant hyperparameter, and test our method using different values of À = a/(1 a) (T.ble Setting a = 0 (i.e., x = 0) corresponds to naive BO against the unconstrained surrogate + oc) is equivalent to a WGAN-like policy. Evaluating constant values of a ranging from 0 to 1, we find that there is no consistently optimal constant value for all eight optimization tasks. In contrast, our method achieves an average rank of 1.9 (2.4) on the top-1 (2op-128) evaluation metric, and is one of the top two methods when compared to the ablations for at least five of the eight tasks. These results suggest that the 'adaptive" nature of aSCR is an important component in solving the constrained optimization problem in 77.
model, while a = (i.e..

---

Offlnne Dutmset
BO-qEI
GABO
Penalized LogP
A
Strueture
RCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC CCC.CCCCCCCCCCCCCCCCCCCCCCCCCCCC CCCCCCCCCCC (-)CCCCCCCC(())CCC((C --
c (CC G-44335))66C5-44) CCC(C)CAC[C)CCC) CCCCC (ECCCC (SC) (CCCCCC)EEEECEECCCCCCCCEEEEEEEEE
(XCCCCCCC)=<CCCSS
CB(C)C72
SMILES
Figure 2: Penalized LogP Score Maximization Sample Candidate Designs (Left) The molecule with the highest penalized LogP score of 11.3 in the offline dataset. Separately, we show the 100th percentile candidate molecules according to the surrogate objective generated from (Middle) vanilla BO-qEI and (Right) GABO. Teal- (white-) colored atoms are carbon (hydrogen). Non-hydrocarbon atoms are underlined the SMILES (Weininger) 1988) string representations of the molecules.
Table 3: GABO Adaptive SCR Ablation Study One-shot (k = and few-shot (k- 128) oracle evaluations averaged across 10 random seeds reported as mean standard deviation. D (best) reports the top oracle value in the task dataset.
Top-1
Branin
LogP
TF-Bind-8'
GFP
UTR'
CЫEMBL
D'Kitty
Warfarin
Rank
D (best)
-13.0
11.3
0,439
3.53
7.12
0.61
0.88
-0.19
1.96
= 0.0
7.8
-52.5 - 88.8
0.586 +: 0.193
1.43 = 0.00
5.65 + 1,30
0.59 + 0.10
0.61 = 0.15
0.16 : 1.67
4.5
.98
0:8+:00 3.9
4.39
0.535
1.43 0.00
4.69
+ 1.44
0.63 0.08
0.61 0.15
1.79
a 0.5
-7.9 6.6
33.9 166.3
0.601 + 0,212
1.43 0.00
5.69
+ 1.51
0.63 +: 0.04
0.12
0.16 :: 1.79
0.8
5.2 3.1
43.3 170.0
0.654 1. 0.218
1.66 4: 0.69
6.49
1 1,20
0.64 1. 0.02
0.71 1: 0.01
0,16 1 1,80
24
61.2
114.3
0.454
3,26
2.33
0.52 0.16
0.62
57.3
aSCR
-2.6 +: 1.1
21.3 +. 33.2
0,570 4: 0.131
3.60 : 0.40
7.51 + 0.39
0.00 +: 0.07
0.71 + 0.01
0.60 1.80
1.9
Top-128
Branin
LogP
TF- TF-Bind-8'
GFF'
UTR"
CLEMBL'
D'Kitty
Warfarin
Rank
D (bes)
-13.0
11.3
0.439
3.53
7.12
0.61
0.88
-0.19 + 1.96
-0.4
135.3
0.942 0.025
2.26
1.03
8.26
0.09
0.67
0.72
0.11
= 0.2
-0.4 :: 0.1
121.8 20.6
0.925 + 0.029
3.01 = 1.04
8.20
0.67 0.01
0.22=00000
1.00 :: 0.00
4.8
0.5
-0.4 1: 0.0
127.7 23.1
0.944
: 0.040
3.49
:: 0.69
8.29 1: 0.08
0.67 0.01
0.72 0.00
1.00 0.00
2.9
.0.4
0.933
3.74
8.38
0.11
0.67
0.72
0.00
1.0
-2.2 :: 1.4
142.3 2.41
0.906
1 0.061
3.74 = 0.00
8,54 1 0.08
0.68 0.01
0.72 =: 0.00
0.99 :: 0.04
3.4
.0.5
122.1 20.6 0.954
3.74
8.36 +
0.70
+: 0.00
0.03
Of note, the top designs found across different constant values of can be very similar for certain tasks. This reflects the inherent challenge in developing task-agnostic methods for policy regularization the magnitudes of the unconstrained objective and regularization function vastly differ, then constant values of a may over- or under- constrain the objective. Adaptive SCR overcomes this problem by dynamically setting as an implicit function of prior observations.
6 Conclusion
We propose adaptive source critic regularization (aSCR) to solve the problem of off-distribution objective evaluation in offline MBO. When leveraged with vanilla Bayesian optimization, aSCR outperforms baseline methods to achieve an average rank of 3.8 (3.0) in one-shot k = 1 (few-shot k 128) oracle evaluation, and most consistently proposes designs better than the offline dataset.
Limitations. One limitation of aSCR is that our algorithm requires preexisting knowledge of the prior distribution over the input space in order to be computationally tractable. While we have focused our experimental evaluation on tasks amenable to imposed latent space priors, further work is needed to adapt aSCR to any arbitrary configuration space. Future work may also extend aSCR to improve parent optimization methods more sophisticated than BO-qEI and Gradient Ascent explored herein.
Impact Statement. Offline policy optimization methods, such as those discussed in this work, have the potential to benefit society. Such examples may include helping develop more effective drugs and individualizing patient therapies. However, as with any real- wollw algorithm, these methods can also be leveraged to generate potentially harmful design candidates. Careful oversight by domain experts and researchers is required to ensure that the contributions proposed herein are used for social good.
10

---

Funding Disclosure and Acknowledgements
The authors thank Pratik Chaudhari the University of Pennsylvania and the anonymous NeurIPS peer reviewers for their thoughtful comments, feedback, and discussion regarding this work. MSY is supported by NIH F30 MD020264. YZ and JRG are supported by NSF award IIS-2145644. JCG is supported by NIH RO1 EB031722. OB is supported by NSF Award CCF-1917852.
References
Aglietti, V., Dhir, N., González, J., and Damoulas, T. Dynamic causal Bayesian optimization. In Proc NeurIPS, 2022. doi: 10.48500/aXxv.2210013888
Ahn, M., Zhu, H., Hartikainen, K., Ponte, H., Gupta, A., Levine, S., and Kumar, V. ROBEL: Robotics benchmarks for learning with low-cost robots. In Proc Conf Robot Learn, volume 100 of CoRL' '20 pp. 1300-1313. PMLR, 2020.
Angermueller, C., Belanger, D., Gane, A., Mariet, Z., Dohan, D., Murphy, K., Colwell, I.., and Sculley, D. Population-basel black-box optimization for biological sequence design. In Proc Int Conf Mach Learn, volume 119 ICML'20, pp. 324-34. PMLR, 2020.
Arjovsky, M., Chintala, S., and Bottou, L. Wasserstein generative adversarial networks. In Proc Int Conf Mach Learn, volume 70 of ICMI.'1, pp- 214-23. PMLR, 2017.
Astudillo, R. and Frazier, I. Bayesian optimization of composite functions. In Proc Int Conf Mach Learn, volume 97 of (CM'17, pp. 354-63, PMLR, 2019.
Barrera, L. A., Vedenko, A., Kurland, J. V., Rogers, J. M., Gisselbrecht, S. S., Rossin, E. J., Woodard,
J., Mariani, L.. Kock, K. H., Inukai, S., Siggers, T., Shokri, L., Gordân, R., Sahni, N., Cotsapas. C., Hao, T., Yi, S., Kellis, M., Daly, M. J., Vidal, M., Hill. D. E., and Bulyk, M. L. Survey of variation in human transcription factors reveals prevalent DNA binding changes. Science, 351 (6280))1450-4 2016. doi: 10. 126/science..ad2257.
Bastani, H., Drakopoulos, K., Gupta, V., Vlachogiannis, I., Hadjichristoodoulu., C., Lagiou, P., Magiorkinis, G., Paraskevis, D., and Tsiodras, S. Efficient and targeted COVID-19 border testing via reinforcement learning. Nature, 599: 1999:08-111 103-100 2021. doi: 10.1038/441186-021-44000
Berrevoets, J., Verboven, S., and Verbeke, W. Treatment effect optimisation in dynamic environments. Journal of Causal Inference, 10(1):106 22, 2022. doi: 10. (0.1515/jji-2022-0000.
Branin, F. H. Widely convergent method for finding multiple solutions of simultaneous nonlinear equations. IBM Journal of Research and Development, 16(5):504-22, 1972. doi: 165/ 0504.
Brookes, D., Park, H.. and Listgarten, J. Conditioning by adaptive sampling for robust design. In Proc Int Conf Mach Learn, volume 97 of ICML.'19, pp. 773-82. PMLR, 2019.
Brown, N., Fiscato, M., Segler, M. S., and Vaucher, A. C. GuacaMol: Benchmarking models for de novo molecular design. Journal of Chemical Information and Modeling, 19:1096-108, 2019. doi: wtt2l/aass.ci.SSbO0.
Chen, C., Zhang, Y., Fu, J.. Liu, X., and Coates, M. Bidirectional learning for offline width model-based optimization. In Proc NeurlPS, 2022. doi: 10.48550aaXxiv2220..75000
Chen, M., Zhao, H., Zhao, Y., Fan, H., Gao, H., Yu, Y., and Tian, Z. Romo: Retrieval-enhanced offline model-based optimization. In Proc International Conf Dist Artif Intell, number 10 in DAI'23, pp. 1-9, ACM, 2023.
Chen, Z., Min, M. R., Parthasarathy, S., and Ning, X. deep generative model for molecule optimization via one fragment modification. Nat Mach Intell, 3:1040-9, 2021. doi: 10. 1038/
Consortium, T. I. Estimation the warfarin dose with clinical and pharmacogenetic data. New England Journal of Medicine, 360(8):753-64 2009. doi: 10. 10.1056/NE/Moa0009339.

---

Deshwal, A. and Doppa, J. Combining latent space and structured kernels for Bayesian optimization over combinatorial spaces. Proc NeurIPS, volume 34, pp. 8185-200, 2021.
Eriksson, D. and Jankowiak, M. High dimensional Bayesian optimization with sparse axis axis-aligned subspaces. In Proc Unc Arti Intel, volume 161 of UAI'21, pp. 493-503. PMLR, 2021.
Eriksson, D.,, Pearce, M., Gardner, J., Turner, R. D., and Poloczek, M. Scalable global optimization via local Bayesian optimization. In Proc NeurlPS, pp- 5496-507, 2019a. doi: 10.4 1910.01739.
Eriksson. D.. Pearce, M.. Gardner, J. R., Turner, R., and Poloczek, M. Scalable global optimization via local Bayesian optimization. Proc NeurIPS, 2019b. doi: 10.48550/arXiv. (ttp:/00aa..i.....011.. 1910.01739.
Ertl, and Schuffenhauer, Estimation of synthetic accessibility score of drug- like molecules based on molecular complexity and fragment contributions. Cheminform, 1(8), 2009. doi: 10.1186/1758 01886/178-294-111--
Flam-Shepherd, D., Zhu, K., and Aspuru-Guzin Language models can learn complex molecule distributions. Nature Communications, 13(3293), 2022. doi: 10.1038/s41467-022 30039-x.
Gaulton, A., Bellis, I.. J., Bento, A. P., Chambers, J., Davies, M., Hersey, A., Light, Y., McGilinche, S., Michalovich, D., Al-Lazikani, B., and Overington, J. ChEMBL: large-scale bioactivity database for drug discovery. Nucleic Acids Res, 40:D1100-7, 2012. doi: 10. 10.1093/na//kk7777
R., Wei, J. N., Duvenaud, D., Hernández-Lobato J. M., Sánchez-Lengeling, B., Sheberla, D., gquilera-Iparraguiree J., Hirzel, D., Adams, P., and Aspuru-Guzik, A. Automatic chemical design using a data-driven continuous representation of molecules. ACS Central Science, 4:268 76, 2018. doi: 10. 12. /ac/acccentscc.760005222
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial nets. In Proc NeurIPS, pp. 2672-80, 2014. doi: 10.48550/arXiv. 140..2661...61
Gu, X., Yang, L., Sun, J., and Xu, Optimal transport-guided conditional score-based diffusion models. In Proc NeurlPS, 2023, doi: (0448550arXiv.2311..111
Guimaraes, G. L.,, Sanchez-Lengeling. B., Outeiral, C., Farias, P. L. C., and Aspuru-Guzik, A. Objective-reinforeed generative adversarial networks (ORGAN) for sequence generation models. arXiv Preprint, 2017. doi: 10.48550/arXiv. 1ttxXXvv..7700
Hansen, N. and Ostermeier, A. Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix adaptation. In Proc IEEE Int Conf on Evolutionary Comp, pp. 312-7, 1996. doi: D0.100//CEC.1119.552233
Hvarfner, C., Hellsten, E. O., and Nardi, L. Vanilla Bayesian optimization performs great in high dimensions. arXiv Preprint, 2024. doi: (0-48500/axx.2240202229
Kantorovich, L. and Rubinstein, G. On a space of totally additive functions. Vestnik Leningrad. Univ, 13:52-9, 1958.
Kim, C. and Bastani, O. Learning interpretable models with causal guarantees. arXiv Preprint, 2021. doi: 10.48550/arXiv. 1ww..i.....11076666666
Kim, M., Berto, F., Ahn, S., and Park, J. Bootstrapped training of score-conditioned generator for offline design of biological sequences. In Proc NeurIPS, 2023. doi: 10,48550/axX1.223660033
Kingma, D. P. and Welling, M. Auto-encoding variational bayes. arXiv Preprint, 2013, doi: 10.48550/arXiv. X11 1312.6114.
Kirkpatrick, S., Gelatt, C. D., and Vecchi, Optimization by simulated annealing. Science, 220 (4598):671-80,, 1983, doi: 10.1126lsieenee220.455..
Krenn, M., Häse, F., Nigam, A., Friederick, P., and Aspuru-Guzik, A. Self-referencing embedded strings (selfies): 100% robust molecular string representation. Machine Learning: Science and Technology, 1(4):045024, 2020, doi: 10.10882662221533bbbab.
12

---

Krishnamoorthy, S., Mashkaria, S., and Grover, A. Diffusion models for black-bor optimization. In Proc Int Conf Mach Learn, volume 202 of ICML 23, pp. 17842-57. JMLR, 2023.
Liu, D. C. and Nocedal, J. On the limited memory BFGS method for large scale optimization. Mathematical Programming. 45:503-225 28, 1989. doi: 10.1007/BF0158911
Mashkaria, S., Krishnamoorthy, S., and Grover, A. Generative pretraining for black-box optimization. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of (CML''23, pp. 24173 97. JMLR, 2023.
Maus, N. T., Jones, H. T., Moore, J. S., Kusner, M. J., Bradshaw, J., and Gardner, J. R. Local latent space Bayesian optimization over structured inputs. In Proc NeurIPS, 2022. doi: 10.48550/arXiv. 2201.11872.
Mockus, J. The Bayesian approach to global optimization. In System Modeling and Optimization, pp- 473-481. Springer, 1982.
Nguyen, T., Agrawal, S., and Grover, A. Expt: Synthetic pretraining for few-shot experimental design. In Proc NeurlPS, 2023. doi: 10.48550/arXiv.23100.1
Nyikosa, F. M., Osborne, M. A., and Roberts, S. T. Bayesian optimization for dynamic problems. arXiv Preprint, 2018. doi: 21.48550/arxi..1833..00 180.00333..
Osborne, M. A., Garnett, R., and Roberts, S. Gaussian processes for global optimization. Inr Conf Learn Intell Opt, pp. 1-15, 2009.
Ramchandani, P., Bastani, H., and Wyatt, E. Unmasking human trafficking risk in commercial sex supply chains with machine learning. Social Science Research Network, 2021, doi: 10.2139/ssrn. 3866259.
Rao, R., Bhattacharya, N., Thomas, N., Duan, Y., Chen, X., Canny, J., Abbeel, P., and Song, Y. S. Evaluating protein transfer learning with TAPE. In Proc NeurlPS, 2019. doi: 10.48550/arXiv. 1906.08230.
Sample, P. J., Wang, B., Reid, D. W., Presnyak, V., McFadyen, I. J., Morris, D. R., and Seelig, G. Human 5'UTR design and variant effect prediction from a massively parallel translation study. Nature Biotechnology. 37:803-9, 2019. doi: (0.10088/411587--19-01666
Snoek, J., Larochelle, H., and Adams, R. P. Practical Bayesian optimization of machine learning algorithms. In Proc NeurIPS, volume 25, pp. 2951-9, 2012, doi: 10.48550/arXiv. 120..20662222 2944.994.
Sobol, I. M. On the distribution of points Mathematical Physics, 7:86-112, 1967. doi: 10. 10110160041-55556679991444
eube and the approximate evaluation of integrals.
Sun, S., Cao, 7., Zhu, H., and Zhao, J. A survey of optimization methods from machine learning perspective. IEEE Transactions on Cybernetics, 50(8):3668 81, 2020. doi: 10.1 2019 2950779.
Székely, G. J., Rizzo, M. L.,, and Bakirov, N. K. Measuring and testing dependence by correlation of distances. Annals of Statistics, 35(6):2769-9) 2007. doi: 10. 00.1214/09903330000000055
Todorov, E.,, Erez, T., and Tassa, Y. MuJoCo: physics engine for model-based control. In Proc Int Conf Intell Rob Sys, pp. 5026-33, 2012. doi: 10. 109//0OS 201 2.986669..
Trabucco, B., Kumar, A., Geng, X., and Levine, S. Conservative objective models for effective offline model-based optimization. In Proc Int Conf Mach Learn, volume 139 of ICML 21, pp. 10358-6 68. PMLR, 2021.
Trabucco, B., Geng, X., Kumar, A., and Levine, S. Design-Bench: Benchmarks for data-driven offline model-bbsed optimization. In Proc Int Conf Mach Learn, volume 162 of ICMI. 22, pp. 21658-76. PMLR, 2022.
Tripp, A., Daxberger, E., and Hernández-Lobato. J. M. Sample-efficien optimization in the latent space of deep generative models via weights retraining. In Proc NeurIPS, volume 33, pp. 11259-72. Curran Associates, Inc., 2020.
13

---

Truda, G. and Marais, P. Evaluating warfarin dosing models on multiple datasets with novel software framework and evoluationary optimisation. J Biomedical Informatics, 113:103634, 2021. doi: 10.1016/j.jbi 20.00..b..200033333333
Weininger, D. Smiles, A chemical language and information system. J Chem Inf Comput Sci, 28(1): 31-6, 1988. doi: 10.1021/:100055:000.
Wildman, S. A. and Crippen, G. M. Prediction of physicochemical parameters by atomic contributions. J Chem Inf Comput Sci, 39:868-7, 73, 1999.
Xu, K. and Bastani, H. Multitask learning and bandits via robust statistics, arXiv Preprint, 2023. doi: 10:48500urxiv:2112..2222
Yu, S., Ahn, S., Song. L., and Shin, J. RoMA: Robust model adaptation for offline model based optimization. In Proc NeurIPS, 2021. doi: 10.48550/arXiv.21 10.
Zhou, Z., Kearnes, S., Li, L., Zare, R. N., and Riley, P. Optimization of molecules via deep reinforcement learning. Scientific Reports, 9(17722), 2019. doi: (0988//4199-0194477111
14

---

A Additional Implementation Details
Oracle Functions. All oracle functions for the tasks assessed are either exact functions or approximate oracles developed by domain experts. Specifically, the Branin and TF-Bind-8 tasks utilize exact oracles described in detail by [Branin (1972) and and (Brrrera al. (2016), respectively. The oracle for the Penalized LogP task is approximate oracle from Wildman & Crippen 1999) that is the same oracle used by domain experts in the Guacamol benchmarking study (Brown et al. (2019). The GFP, UTR, and ChEMBL tasks feature approximate oracles from [Snoek et (2012), Angermueller et al. 22000) and Trabucco et al. (2022), respectively, that were trained on a larger. hidden datasets inaccessible to us for the respective tasks. The D' Kitty morphology task uses MuJoCo (Todorov et al. 2012)
simulation environment and learned control policy from Trabucco et al. (2022) to evaluate proposed designs. Finally, the Warfarin task uses linear model (Consortium 2009) to estimate a patient's
optimal warfarin dose given their pharmacogenetic attributes.
Data Preprocessing. (1) For the Branin task, we sample 1000 points from the square input domain [-5, 10) x [o, 15) to construct the offline dataset, and remove the top 20%-ile according to the oracle function to make the task more challenging in line with prior work (Mashkaria et al. 2023). In this continuous task (along with the D'Kitty and Warfarin tasks), we treat input designs as their own latent space mappings, such that the VAE encoder and decoder for this task are both the identity function with zero trainable parameters. (2) The offline dataset of the Penalized LogP task is the validation partition of the Guacamol dataset from Brown et al. (2019), which consists of 79,564 unigue molecules and their corresponding penalized LogP scores. The input molecules are represented as SMILES strings /Weininger| 1988), which is molecule representation format shown to frequently yield invalid molecules in prior work (Krenn et al.] (020). Therefore, we encode the molecules instead as SELFIES strings, an alternative molecule representation from Krenn et al. (2020) with 100% robustness.
(3)- (5) The TF-Bind-8, GFP, and UTR tasks are assessed as-released by Design-Bench from Trabucco et al. (2022) please refer to their work for task-specific descriptions. (6) - (7) In the normalize all objective values y in the offline dataset to (y ymin)/ (ymax Vmmnn) as done in prior work (Mashkaria et al. 2023). 3, where ý is the corresponding normalized objective value and Ymin (ymax) is the minimum (maximum) observed objective value in the full, unobserved dataset. Because only the bottom 60%- (40%-ile) from the full dataset is used in the available offline dataset for the ChEMBL (D' 'itttty) task, the respective maximum values are less than 1.0 (Supplementary Table AII We also translate the original SMILES string representations in the ChEMBL task into SELFIES strings (Krenn et al. 2020) as in the LogP task.
ChEMBL and D'Kitty tasks,
(8) Finally, the Warfarin task uses the dataset of pharmacogenetic patient covariates published by Consortium (2009). We split the original dataset of 3,936 unique patient observations into training (validation) partitions with 3,736 (200) datums. The patient attributes in the Warfarin datasct consist of combination of discrete and continuous values. All discrete attributes are one-hot encoded into binarized dimensions, and continuous values are normalized to zero mean and unit variance using the training dataset. Missing patient values were imputed following prior work (Truda & Marais 2021). We define the cost c(2(1) accrued by a patient with attributes x E 32 as a function of the input dose E R is c(2[2) (z doracte (x)) )² where doracte 32 - R is the domain-exper. oracle warfarin dose estimator from Consortium| (2009). The observed objective values associated with each of the training datums is calculated
y = [c(E|z)) c(>[=)]/ / /((((ar) where 2 is the mean warfarin dose over the training dataset and 2 is the true dose given to the patient. Using this constructed offline dataset, our task is then assign optimal doses to the 200 validation patients to maximize y with no prior warfarin dosing observations.
B Additional Experimental Results
In this section, we provide additional experimental results that help better characterize both the strengths and limitations of GABO and GAGA.
B.1 How do sub-optimal design candidates proposed by GABO and GAGA perform?
To evaluate the robustness of optimization methods, we report one-shot 90th percentile oracle scores in Supplementary Tables B1 and B2 For each method, all proposed designs are ranked according
15

---

Table A1: MBO Datasets and Tasks Implementation details for each of the seven MBO tasks assessed in our work. "Denotes the life sciences- related discrete MBO tasks offered by the Design- Bench benchmarking repository (Trabucco al.
2022
Property
Branin
LogP
TF-Bind-8"
GFP
UTR'
ChEMBI,"
D'Kitty
Warfarin
Dataset Size
800
79,564
32,898
5,000
140,000
441
10,004
200
Input Shape
2
108
8
237
50
32
56
(33)
Vocab Size
97
4
20
4
40
VAE Backbone
Identity
Transformer 256
ResNet 16
ResNet
ResNet 32
Transformer 128
Identity 56
Identity 33
VAE Latent Shape
2
32
Oracle
Exact -3.00
Linear 11.3
Exact 0.439
Transforner 3.53
RosNet 7.12
Random Forest 0.61
Exact 0.88
Lincar
D (best)
-0.19 1.96
to the surrogate forward model (8) for Generative Adversarial Bayesian Optimization (GABO) and Generative Adversarial Gradient Ascent (GAGA)), and the single 90th percentile design according to this ranking is selected and evaluated using the oracle function. We report the oracle score of this suboptimal design averaged over 10 seeds.
Our results show that GABO and GAGA do not propose suboptimal designs that are better than those proposed by other methods, such BONET (Mashkaria al. (023). Simulated Annealing (Kirkpatrick et al., 1983). L-BFGS (Liu & Nocedal 1989). and ExPT (Nguyen et al. (023). This not surprising, as aSCR is not designed target this metric (and it is not our primary metric of interest). Separately for GABO, we also hypothesize that the algorithm's 's performance according to this metric may partially be explained by the limitations of the underlying Bayesian optimization (BO) optimization algorithm. Because BO is not an iterative first-order algorithm, the designs proposed by any BO-based algorithm often have high variance practice-thi this is indeed what we observe across all of our experiments, including in Table 1 and Supplementary Tables B1 and B2
Finally, we note that in most applications of offline optimization, the 90th percentile metric- or any metric that does not use the best proposed design(s) not as useful as the other metrics assessed where GABO does perform well. This is because in offline optimization tasks with a restricted budget to query the hidden, expensive-to-evaluel limited budget on subpar design candidates. While the 90th percentile and similar metries can be helpful to understand the limitations of algorithms (as in this case), we believe that the alternative evaluation metries reported in the main natt-amell. the 100th percentile top-1 and top- 128 oracle score metrics- are more useful and practical in assessing each of the optimization algorithms.
oracle function, we are not interested in "wasting" this
Table B1: Constrained Budget (k 1) Suboptimal (90%-ile Oracle Evaluation The oracle score of the 90th percentile design candidate according to the surrogate across 10 random seeds is reported as mean + standard deviation. D (best) reports the top oracle value in the task dataset. The average rank across all seven tasks is reported in the final table column. Bolded (Underlined) entries indicate the best (second best) entry in the column. "Denotes the life criences-related discrete MBO tasks from Design-Benche (Trabucco et al. 2022).
Method
Branin
LogP
TF-Bind-8*
GFP
UTR'
CLEMBL."
D' D'Kitty
Warfurin
Runk
(best)
0.439
-0.19
Grad.
-4,4 20.9
-5.47
1.32
0.429
0.023
3.43 1:3-2:00 0.67
7.16
0.21
-1.95 + 0.00
0.53 = 0.30
0.87 + 1.0K
9.1
L-BFGS
-.00
4,96
0.547
7.36
0.00
0.75 1: 1.66
CMA-ES
4.35 2 6.18
0.448
0.068
3.74 0.00
1.13 6.95
-1.95 1: 0.00
0.60
0.29
4.02
21.8
7.1
Anneal
-13.2 D.0
9.57 + 0.66
0.439
0.000
3.65
1.65+00.0
0.04
7.41
111++000 0.22
-1.95 + 0.00
0.56 0.00
0.96 + 0.08
6.0
BO
-11.5
91.9
0.552
0.01
0.18
TuRBO
-16.3 =: 10.2
-2.3 66.3
0.563
0.087
1.42 0.00
6.79
1.25
0.65 0.00
0.71
0.01
-32.3 1: 94.9
7.9
BONET
-29,2 + 2.2
10.8 + 0.43
0.324
0.041
3.74 0.00
8.70
0.32
0.56 0.11
0.78 0.00
6.0
-1870
7.91
0.29
-56.6
COM
-3468
679
-37.4 1 23.0
0.346
0.093
3.62 0.00
5.26
1.01
0.60 0:001:000 0.04
0 0.01
0.80 1: 0.93
9.6
RoMA
-18.5 = 8.2
5.21 + 1.39
0.500
0.153
3.58 0.11
6.94
1.11
0.43 + 0.18
0.41
0.21
-2.44 +: 2.16
8.1
BDI
-109
0.49::0000
0.00
-24.8
EXPT
-2311 = 11.3
-16.7 1 25.1
0
0.091
3.74 0.00
4.70
0.62 1 0.04
0.75 0.07
-0.10 1: 1.61
6,9
BootGen
-116.8 85.7
0.388
0.007
3.60 0.04
7.74 0.56
0.61 0.03
8.8
-3142
-25.6
0.04
0.17
-2.77 5:21
GAGA -14.2=15.2
16.7+881.1
0.546
0.148
3.22 0.86
6.40 + 1.13
-1.95 + 0.00
0.89 = 0.01
0.24 + 0.20
8.5
GABO
-12.7=100
46.1
0.467
0.066
3.56 1.66
6.12 1.22
0.61 0.08
0.57 0.17
0.02 2: 5:77
7.9
16

---

Table B2: GABO Adaptive SCR Ablation
Budget (k = 1) Suboptimal (90%- -ile) Oracle Evaluation The oracle score of the 90th percentile design candidate according to the surrogate across 10 random seeds reported as mean + standard deviation. D (best) reports the
top oracle value in the task dataset. Average method rank across all seven tasks reported in the final column. Denotes the life sciences-related discrete MBO tasks
Bench (Trbucco et al. (022).
GABO
Branin
LogP
TV-Bind-8'
GFF
UTR'
CBEMBL'
D'Killy
Warfarin
Rank
D (best)
-13.0
11.3
0.439
3.53
7.12
0.61
0.88
.0.9 4: 1.96
0.0
-11.5
-6.2 91,9
0.552
0 052
1.42 4: 0.00
5.80
+: 1.71
1.64 = 0.01
0.46
4: 0.18
.369 + 205
3.3
-90
2.6 2.6
27.4
0.612
0.114
1.42 0.00
5.81
1.83
0.59-0.15
0.49
0.18
31.74 + 265
a 0.5
386:44
90.1 1 107.2
0.501
0.109
1.65 1: 0.69
6.64
1: 1.42
0.52
0.15
0.11
1 0.16
63.5 1: 336
3.9
0.8
-109 2.1
-41. 41.9:49. 82.5
0.433
0.158
1.97
2: 0.88
4.89
+ 1.23
0.36
205-0.33 0.15
0.38
+: 0.15
-48.
+ 265
4.4
1.0
-104.6
68.9
-77.1 + 146.1
0.452
0.179
2.05
+ 0.98
5.15
+ 1.51
0.60
0.08
0.41
+ 0.16
.82.1 + 552
4.5
aSCR
-12.7 10.0
-12.2 46.1
0.467 0.066
3.56
1.66
1.22
0.57 0.17
To further characterize the distribution of designs and their associated oracle scores proposed by GABO, Figure [1 plots a histogram of the oracle scores of (1) all 2,048 oracle scores, and (2) the oracle scores of the top 256 designs according to the penalized surrogate objective in (8) for the LogP task. Compared with the other optimization methods assessed, we notice that the range of oracle scores is larger for BO-based optimization methods compared with the baseline methods assessed. This helps motivate our design choice to leverage aSCR and Algorithm| I with BO-qEI, as BO is able to explore larger region of the design space and is an effective parent optimizer for complex design spaces. Secondly, we also find that the distribution of scores is similar between BO-qEI and GABO, even though the performance of these two methods is remarkably different in Tables [ and 2. This is likely due to the fact that while BO enables us to explore a larger effective region of the design space (compared with first-order iterative methods), aSCR more accurately ranks proposed designs using the penalized surrogate so that we can identify promising candidates even in the low-budget oracle evaluation regime.
B.2 Are offline objectives and oracle function values correlated
A key component of GABO with Adapative SCR critical to the above discussion in Section B.1 is that generated designs score similarly according to the hidden oracle function and the regularized Lagrangian objective as in (8) in order to solve the problem of surrogate objective overestimation encountered in traditional offline optimization settings (Fig. []. To assess this quantitatively, we computed the distance covariance I X*) --= (xxx) ) between the oracle scores (xx)) and the constrained Lagrangian scores L(x;; X*) with = (L) computed using our Adaptive SCR algorithm. The empirical distance covariance metric is computed over the n. 2048 design candidates generated using our GABO algorithm. Briefly, the distance covariance is a nonnegative measure of dependence between two vectors which may be related nonlincarly: greater distance covariance implies a greater degree of association between observations (Szčkely et al. 2007). We focus our subsequent discussion the Penalized LogP task.
Across five random seeds, GABO with Adaptive SCR achieves a distance covariance score of 0.535 + 0.067 (mean + standard deviation). In contrast, naive BO-qEI (i.e., À = 0) only achieves a distance covariance score of 0.392 +: 0.040. Using p < 0.05 as a cutoff for statistical significance, the distance covariance scores are significantly different between these two methods (p R 0.006, unpaired two-tailed (-test). These results help support our conclusion that GABO with Adaptive SCR is able to provide better estimates of design candidate performance according to the hidden oracle function when compared to the corresponding unconstrained BO policy.
B.3 adaptively computing a in aSCR important for the performance of GAGA?
In our ablation experiments presented in Table we showed how that 'adaptive" nature of aSCR is an important component in solving the constrained optimization problem in 7 for GABO, and outperforms alternative approaches that manually hand-tune a (and hence A) constant hyperparameter. We explore whether this conclusion also applies for GAGA as well here.
For clarity, we first offer the explicit formulation of GAGA in Supplementary Algorithm 3 We ablate Algorithm [ in GAGA by instead evaluating our method using different values of = a/(1 - aa). As reminder, setting a 0 (i.c., A = 0) corresponds to naïvely performing gradient
17

---

Histogram Density
0.4
Grad.
Grad. Mean
0.3
0.2
0.2
0.1
8.1
0.0
0.0
Histogram Density
Grad, Min
L-BFGS
0.06
0.2
0.04
0.1
0.02
0.0
0.00
Histogram Density
0.15
BOBYQA
CMA-ES
0.06
0.10
0.05
0.02
0.00
0.00
Histogram Density
Sim. Anneal
0.100
BO-qEI
0.4
0.3
0.2
0.1
0.025
0.0
0.000
Histogram Density
0.15
TuRBO-qEI
BONET
0.4
0.10
0.00
0.0
Histogram Density
0.20
DDOM
COM
0.15
0.2
0.00
0.0
Histogram Density
RoMA
0.100
GABO
0.2
0.075
0.050
0.1
0.025
0.0
-50
25
50
75
100
0.000
-50
-25
25
30
75
100
Oracle Penalized LogP Score
Oracle Penalized LogP Score
Figure B1: Distribution of Oracle Penalized LogP Scores We plot the distribution of oracle scores for the top 128 surrogate model-ranked designs in black, and the distribution for all 2,048 generated designs in light gray for each of the offline model-based optimization methods assessed in our work across 10 random seeds. While GABO and BO-qEI have similar distributions, GABO is able to more reliably rank top-performing designs higher, such that these designs can be identified even under limited oracle query budgets.
18

---

ascent against the unconstrained surrogate model; setting a 1 (i.e., À + oo) is equivalent to WGAN-like generative policy.
Algorithm 3 Generative Adversarial Gradient Ascent (GAGA)
Input: surrogate objective fe 1e + IR, offline dataset D, イ iterative sampling budget T' sampling batch size number of generator steps per source critic training Nyenerator. oracle query budget k. step size 7
AdaptiveSCR Imput: a step size Ao, search budget B, norm threshold т
Define: Differentiable source critic c: R" + IR
Define: Lagrangian [(z; ar) IK' x R - R fo(=) LE [c(=']] c(=)]
// Eq. 8
Sample Zz
{2
p 1 the top 6 designs in D., according to their previously observed oracle scores
// Train source critic per Eq. 6 to optimality:
c - argmax ell., 1 (D,, ; 2
argmax Se E. [e(=')] - 22 [c(2)]]
a f AdaptiveSCRi fe, c, P,, Ao, B,
Alg.
Evaluate candidates } < {1 Ji=1 a2)
for in 2,3. T do
3 12
[2
2,- 2 (0:001-100000 a)
a 6 AdaptiveSCR( (fossCRR(oC.c C, D., Do, B,
Evaluate samples ) <- {vf 1==1 { [=::
if mod Regmeratur equals 0 then
// Train the source critic per Eq.
6 to optimality:
c t argmax |ell Wi (D., Z" argmax |le IaSK lc(=)] E. [==2)(((=)]]
end end for return the top l samples from the T x observations Dr = [[(==""(""))))""""""
"[ according to y²"
similar to the analogous ablation results for GABO T dynamically adjusting the strength of source critic regularization using our aSCR algorithm outperforms manually setting the value of to constant in both the one-shot k = 1 and few-shot 128 evaluation settings.
Our results are shown in Supplementary Table 13
B.4 What is the impact of dynamic updates to the source critic over the optimization trajectory?
In Algorithm 2 and Supplementary Algorithm 3 we describe how generative adversarial opti- mization alternates between batched acquisition steps according to the optimizer and re-training the source critic on the newly sampled trajectory points. To better interrogate the significance of dynamically re-training the source critic during optimization, we compare the performance of the default GABO and GAGA algorithms (with Rgeeerraorr = 4 as the number of acquisition steps per critic retraining step) against the respective methods without source critic re-training (i.e., Ngenerator 0o) in Supplementary Table B4. Across all three evaluation metrics and all eight tasks, dynamically retraining the source-critt improves upon the performance of the GABO when Nggnerator oo by 67.4% in the top-1 evaluation metric; 0.0% in the top-1 128 evaluation metric; and 33.5% in the 90%-il evaluation metric. Intuitively, these results align with the value of the source critic being able to implicitly set the value of the regularization strength in (8) according to the sampled trajectory ceinss-sspecillly in the constrained budget oracle evaluation setting.
Interestingly, we do not observe similar performance improvements with dynamic re-training of the source critic in GAGA. Qualitatively, we find that this is because the iterative first-order nature of the parent gradient ascent algorithm because the sampled designs are clustered in the same regions of the design space over the course of optimization, the energy landscape of the penalized surrogate (i.e.. the negative of the Lagrangian expression in (8) does not change significantly during source critic re-training. This further reinforces the optimizer to stay roughly the same regions of the design space. As result, it is likely that no major updates are often made to the source critic when aSCR is used in conjunction with a first-order optimization method, and so the benefit of using finite Reguserator hyperparameter value is largely reduced when compared to its utility in GABO.
19

---

Table B3: GAGA Adaptive ACR Ablation Study We ablate the dynamic computation of a (and hence À in (8) by instead choosing to manually fix a to a constant value. A value of a = 0.0 corresponds naive gradient ascent, and a value of a 1.0 corresponds to WGAN-like generative policy. Oracle values are averaged across 10 random seeds and reported as mean + standard deviation. In each evaluation setring, we rank all 2,048 proposed designs according the penalized surrogate forward model in (8) and evaluate the top k designs using the oracle function, reporting the maximum out of the & oracle values. In the suboptimal evaluation setting, we report the oracle score of the single 90th percentile design according to the penalized surrogate ranking. Bold (Underlined) entries indicate the best (second best) entry in the column for the particular evaluation metric. *Denotes the life sciences MBO tasks offered by Design-Bench (Trabucco et al. 2022).
Brauin
LogP
TF-Bind-8'
GFP
UTR'
CbEMBL'
D'Kitty
Warfaria
Rank
D (best)
-3300
11.3
0.439
3,53
7.12
0.61
0.88
-0,19 + 1.96
Construined Budget (k:
1) Orade Evaluation
0.0
-245.1 81.
-5.37 + 1.44
0.429 + 0,023
3,18 + 0.88
6,82 + 0.21
-1.95 0.00
0.57 + 0.19
(.86 + 1.09
3.6
=13.7:000
115.8
3,74 1: 0.00
7.73 0.46
-1.95 0.000
0.00
a 0.5
0.0
-70.3 +: 114.7
0.439 1: 0.000
3.74 1: 0.00
6.75 -: 0.72
-1.954.0.00
0.88 0.00
0.44 4: 0.00
2.4
0.8
-13.7
-84.6
115.8
0.439
+ 0.000
3.74 + 0.00
6,75 + 0.72
-1.95++0.00
0.88 0.00
0.44 + 0.00
2.6
-14,4
99.8
0,439
0.000
3.74
0.00
0.00
6.15
aSCR
-2,9 1: 2.2
-68.6 4 109.8
0.571 1: 0.120
3.74 1: 0.00
5.89 = L.42
-1.95 0.00
0.89
0.00
0.01 1: 0.14
2.3
Relaxed Budget (A 128)
Oracle Evaluation
0.0
-115.3 = 20.8
-5.14 1,70
0,977 0.025
3.49 :: 0.69
7.38 = 0.15
-1.95 +: 0.90
0.87 0.02
0.86 + 1.08
0.2
-13.2 0.0
4.70 + 10.3
0.439 + 0.000
3,74 + 0,00
7.92 + 0.24
+1.05++0..0
0.95 0.00
1.00 + 0.000
2.4
5.07
0.000
7,77 0.21
-1.95
0.01
a 0.8
1.0
5.13 1 4.28
0.439 1: 0.000
3.74 1: 0.00
7.44 4: 0.30
-1.95 0.00
0.95 0.01
1.00 1: 0.00
2.9
1.0
.13.1
5.11 + 4.11
0.445 + 0.017
3.74 + 0.00
7.40 + 0.28
-1.95
+ 0.00
0.90
1:90++10 0.01
0.96 + 0.05
3.0
aSCR
-1.0 = 0.2
14.1 : 25.0
0.722 + 0.091
3.74 + 0.00
7.98 = 0.36
-1.95 + 0.90
0.90 0.01
0.95 1 0.07
Constrained Budget (k
1) Suboptimal (90%-lle) Oraele Evaluation
94.4 20.9
-5.47
1.32
0.429 0.023
3.43 + 0.67
7.16 + 0.21
-1.95 0.00
0.53 1:3-+000 0.20
0.87 +: 1.08
==
0.5
-1099
14,9
0.439 + 0.000
3.74 0.00
6.57 = 0.94
-1.95
+ 0.90
0.89 0.02
0.97 : 0.04
2.5
Q 0.5
16.22
0.6
15.2
14.4
0.445 1: 0.017
3,74 1: 0.00
6.75 = L18
-1.95 0.000
0.90 0.02
0.93 4 0.18
2.4
10
-12.7 13,8
0,439
1
3,74
6.81
-1.95
0.000
a 1.0
-14.6
2 1.4
16.9
1: 13.1
0.439 1: 0.000
3.74 1: 0.00
6.82 = L.1
-1.95 4 0.00
0.89 0.01
-2.71 1: 7.71
3.3
aSCR
-14.2 L 15.2
16.77 1 81,1
0.546 1: 0.148
3.22 1: 0.86
6.40 a 1.1
1..95.00.00
0.89 0.01
0.24 1: 0.20
3.5
B.5 How does initialization affect the performance of GABO?
Per Algorithm 2 GABO is based the BO-qEI baseline optimization policy, which involves initializing the gaussian process (GP) to approximate the offline surrogate model. Consistent with prior work (Eriksson et al., 2019a Maus al.) 2022), we initialize the GP using the pseudo-random Sobol sequence (Sobol) 1967) at the beginning of the optimization procedure. However, an alternative approach is instead initialize the GP using the top Alinit samples from the offline dataset. In particular, this strategy is already employed in both related work describing the baseline first-order optimization methods assessed herein, with the idea that better designs can be generated by initializing from better designs. We compare these two GP initialization strategies in Supplementary Table B5
Interestingly, our results show that initializing the GABO GP from the Sobol sequence consistently outperforms initialization from the top candidates in offline dataset. We hypothesize that this may be due to the fact that top-scoring candidates likely lie in similar regions of the input space, which significantly alters the ability of the optimizer to explore other regions of the design space over the course of the optimization process. Future work may help better interrogate the relationship between GP initialization and offline optimization, which is outside the scope of this work.
B.6 Can the Gaussian process (GP) in GABO be directly used as the surrogate forward model?
In Algorithm 2 we leverage a surrogate forward model fe in model-based optimization and separate GP to acquire samples in the Bayesian optimization framework. However, may be possible to use the GP directly as the surrogate forward model. Our results in Supplementary Table B6 suggest that this is not an effective strategy with which to use GABO- using even the simple neural-network as the surrogate function (as done in our approach in Algorithm 2 outperforms the alternative GP-based
20

---

Table B4: Ablating Dynamic Updates to the Source Critic We study the effect of training the source critic model exactly once (i.e., setting Agenerator = 0o in A and Supplementary Algorithm 3) as opposed to re-training the source critie model every ngeneeattr on the newly sampled designs. Oracle values are averaged across 10 random seeds and reported as mean + standard deviation. In each evaluation setting, we rank all 2,048 proposed designs according to the penalized surrogate forward model in ( and evaluate the top k designs using the oracle function, reporting the maximum out of the k oracle values. In the suboptimal evaluation setting, we report the oracle score of the single 90th percentile design according to the penalized surrogate ranking. Bold entries indicate the best entry in the column for the particular optimizer and evaluation
acquisition steps
metric. Denotees the life sciences MBO tasks offered by Design- Bench (Trabucco et al., 2022)
GABO
Branin
LogP
TF-Bind-8'
GFP
UTR'
CbEMBL'
D'Kitty
Warfarin
D (best)
-13.0
11.3
0.439
3,53
7.12
0.61
0.88
-0.19 + 1.96
Constrained Budget (k = 1) Oracle Evaluation
Plananse 30
-3.5 +
-55.6 52.1
0.577 0.151
3.74 + 0.00
6.73 1.10
0.65 + 0.00
0.46
-0.27 13.7
-2.6 + 1.1
21.3 + 33.2
0,570 0.131
3.60 + 0.40
7,51 = 0.39
0.60 + 0.07
0.71 + 0.01
0.60 + 1.80
Relaxed Budget (K
128) Orucle Evaluation
Ogganese 00
-0,5 0.1
128.0 = 19.5
0.946 = 0.035
3,74 + 0.00
8.38 = 0.11
0.67 + 0.01 0.70 + 0.01
0.72 + 0.00
1.00 + 0.00
-0.5 + 0.1
122.1 20.6
0.954 0.025
3.74 + 0.00
8.36 0.08
0.72 + 0.00
1.00 100++0.00 0.03
Construined Budgel (k 1) Suboptimul (90%-ile) Oracle Evaluution
Oganmas 30
-8.9
-4.1 62.6
0,061
1.04
6.02
1: 0.07
0.62
-5.32 4.59
-12.7 + 10.0
-12.2 = 46.1
0.467 0.066
3,56 + 1.66
6.12 = 1.22
0.61 + 0.08
0.57 : 0.17
0.02 + 5.77
GAGA
Brania
LogP'
TF-Bind-8"
GFP
UTR'
CbEMBL'
D'Kitty
Warfarin
D (best)
-13.0
11.3
0.439
3,53
7.12
0.61
0.88
-0.19 = 1.96
Constralued Budget (k: 1) Oracle Evaluation
14,6 1: 0.8
-1.87 14.9
0,439
0.000
3,74 1. 0.00
6.45
0.54
-1.95 1 0.00
0.88 1 0.00
-0.17 1 0.29
-2.9 +: 2.2
-68.6 + 109.8
0.571
0.120
3.74 + 0.00
5,89 = 1.42
-1.95 + 0.00
0.89 + 0.00
0.01 + 0.14
Relaxed Budget (A 128) Oraele Evaluation
Rgaaeree 00
-13.3 0.2
50.2 + 2.48
0.439
0.000
3.74 + 0.00
7.38 + 0.31
-1.95 + 0.00
0.90
+ 0.01
0.99 + 0.01
-1.0 1. 0.2
14.1 1 25.0
0.722 0.091
3,74 1. 0.00
7,98 0.36
-1.95 0,00
0.90 0.01
0.95 0.07
Constrained Budget (k = 1) Suboptimal (90%-ile Oracle Evaluation
Figraer -17.0 1.6
5.88 4.88
0.439
0.000
3,74 :: 0.00
7,08
0.73
-1.95
0.00
0.89 0.01
-1.38 1.68
Figneraar .14.2 1: 15.2
-16.7 81.1
0.546 0.148
3,22 1: 0.86
6.40 1 1.13
-1.95 1 0.00
0.89 1 0.01
0.24 1: 0.20
approach in six of the eight tasks in the top-1 evaluation setting, and is non-inferior the alternative Ga-based approach in all eight tasks in the top-128 evaluation setting. These results suggest that using more complex neural-network surrogate function for GABO leads to better optimization results than directly using the GP as the surrogate function.
B.7 What is the computational cost of running aSCR (i.e., Algorithm
At first glance, Adaptive SCR may appear to be a computationally expensive algorithm: it requires us to dynamically re-train a source critic neural network and compute the Lagrangian hyperparameter at each step through grid search. However, in the implementation used for our experiments, the grid search to compute is highly vectorized, and the source critic re-training patience and learning rate are such that the computational cost from re-training is not too significant. As a result, we are able to run Adaptive SCR with both Bayesian Optimization (BO) and Gradient Ascent (GA) using an experimental setup with one 24-core Intel Xcon CPU and one NVIDIA RTX A6000 GPU. To benchmark our implementation, we evaluate BO and GA both with and without our Generative Adversarial (GA) source critic regularization algorithm on the Branin and Penalized LogP optimization tasks. As a reminder, the Branin task is standard benchmarking task for offline optimization, and the Penalized LogP task is subjectively the most challenging task assessed in our manuscript with the highest dimensional design space out of the eight assessed tasks.
Our results are shown in Supplementary Table B7 On the Branin toy task, aSCR increases the compute time by 257% for BO and 680% for GA, which is a significant computational cost. However, on the more challenging LogP task more representative of the tasks encountered in the applications of offline optimization, aSCR only introduces 6.9% increase in compute time for GA and 28.9%
21

---

Table B5: GABO GP Initialization Ablation Study We investigate the effect of initializing the Gaussian process (GP) in GABO using the best au points from the offline dataset (i.e., Best initialization strategy) versus our method in Algorithm 2 where the GP is initialized using the first Vinit points from the Sobol sequence from (Sobol 1967) (i.e., Sobol initialization strategy). Oracle values are averaged across 10 random seeds and reported mean + standard deviation. In each evaluation setting, we rank all 2,048 proposed designs according to the penalized surrogate forward model in (8) and evaluate the top designs using the oracle function, reporting the maximum out of the & oracle values. In the suboptimal evaluation setting, we report the oracle score of the single 90th percentile design according to the penalized surrogate ranking. Bold entries indicate the best entry in the column for the particular optimizer and evaluation metric. "Denotes the life sciences MBO tasks
offered by Design- Bench (Trabucco et al. 2022).
Strategy
Brasin
Logl'
-F--Bid-88'
GFF
UTR'
ChEMBL'
D'Kitly
Warfarin
(hest)
13.0
11.3
0.439
3.53
7.12
0.61
0.88
-0.19 + 1.96
Constrained Budgel (k 1) Oracle Evaluation
Best
-3.6
14.0 18.4
0.504 0.117
2.97 = 1.02
5.36
1.24
0.61
0.00
0.50
0.19
-2.97
9.00
Sobol
-2.6 1.1
21.3 33.2
0.570 +: 0.131
3,60 :: 0.40
7,51 +: 0.39
0.60 +: 0.07
0.71
0.01
0.60 1.80
Relaxed Budget (k- = 128) Oracle Evaluation
Best Sobol
118.9 19.5
0.034
0.00
8.37
0.09
0.66 0.01
0.87
0.05
-0.5 0.1
122.1 + 20.6
0.954 + 0.025
3.74 + 0.00
8. + 0.08
0.70 + 0.01
0.72 + 0.00
1.00 0.03
Constrained Budgel (k: 1) Suboptinal (90%--ile Oracle Evaluation
Best
-11.8 + 6.4
.85.9 124
0.382 0.106
3.45 0.77
6.28
1.70
0.60 0.03
0.64 0.23
-0.65 3.97
Sobol
-12.7 10.0
-12.2 46.1
0.467 0.066
3.56 1,66
6.12 = 1,22
0.61 = 0.08
0.57 = 0.17
0.02 = 5.77
Table B6: GABO Neural Network Surrogate Ablation Study Instead of using neural network (NN) as our surrogate forward model, we explore if the Gaussian process (GP) employed by the parent BO optimizer can directly be used as the surrogate model in GABO's framwork. Oracle values are averaged across 10 random seeds and reported as mean + standard deviation. In each evaluation setting, we rank all 2,048 proposed designs according to the penalized surrogate forward model in (8) and evaluate the top k designs using the oracle function, reporting the maximum out of the k oracle values. In the suboptimal evaluation setting, report the oracle score of the single 90th percentile design according to the penalized surrogate ranking. Bold entries indicate the best entry in the column for the particular optimizer and evaluation metrie. *Denotes the life sciences MBO tasks
offered by Design-Bench (Trabucco et al. 2022).
Surrogate
Braniu
LogP
TF-Bind-8
GFP
UTR'
CLEMBL
D'Kitty 0.88
Warfarin
D (best)
-13.0
11.3
0.439
3.53
7.12
0.61
-0.19 = 1.96
Constrained Budget (k 1) Oracle Evaluation
GP
-3744 4.4
-57.9 : 159.2
0.576 = 0.058
3,51 + 0.69
6.84 1.24
0.65 : 0.01
0.42 = 0.17
-0.28 2.13
NN
-2.6 1.1
21.3 33.2
0.570 0.131
3.60 1 0.40
7.51 0.39
0.60 1 0.07
0.71 1. 0.01
0.60 1.80
Relaxed Budget (A: 128) Oracle Evaluation
GP
0.5
119.9 = 20.1
0.755 = 0.071
3.74 + 0.00
8.34 0.07
0.67 0 0.01 0.70 + 0.01
0.72 - 0.00
-0.27 2.13
NN
-0.5 0.1
122.1 20.6
0.954 0.025
3.74 + 0.00
8.36 + 0.08
0.72 - 0.00
1.00 + 0.03
Constrained Budget (k 1) Suhoptimal (10%)-ile Oracle Evaluation
GP
-10.1 10.6
-51.5 + 108.8
0.562 0.091
1.13
6,54 1.56
0.65 0.00
0.50
-0.27 2.13
NN
-12.7 = 10.0
-12.2 + 46.1
0.467 = 0.066
3.56 + 1.66
6.12 + 1.22
0.61 + 0.08
0.57 = 0.17
0.02 + 5.77
increase for BO. Furthermore, while there are evidently additional compute costs associated with running our aSCR algorithm, we note that in most applications of offline optimization, obtaining labeled data is the main bottleneck in many practical applications. Thus, it is often worth spending this extra compute to ensure the best results for a given evaluation budget using aSCR.
B.8
How do the performance of GABO and other optimization methods vary with the allowed oracle query budget k??
To investigate this question, we vary the number of allowed le-shot oracle calls the Penalized LogP task (Supplementary Fig. B2) While the majority of first-order optimization methods we evaluated
22

---

Table B7: Computational Tractability
Runtimes single node using NVIDIA RTX A6000 GPU are averaged across 10 random seeds and reported as mean standard deviation.
Time (sec)
Branin
LogP
Time (sec)
Branin
LogP
Grad.
9.68 0.23
765 6.64
965
GAGA
75.6 +: 25.4
818 + 10.5
GABO
329 + 146
1245 +: 55.2
% Increase
680% 259%
6.9% 1.6%
Incrense
257% 157%
28.9% 4.5%
are able to reach local optima rapidly, the proposed designs from such approaches are suboptimal compared those from GABO (and GAGA) with Adaptive SCR as the oracle query budget size increases. Separately, comparing the curves for GABO and vanilla BO-qEI, we see that GABO with Adaptive SCR is able propose consistently superior design candidates in the small query budget regime often encountered in real-world settings. This is due to the fact that GABO regularizes the surrogate function estimates such that the proposed candidates are both high-scoring according to the surrogate objective and relatively in-distribution. Our results demonstrate that especially for real-world tasks like molecule design with complex objective function landscapes, methods such as GABO with Adaptive SCR are able to explore diverse, high-performing design candidates effectively even in the setting of small oracle query budgets.
150
Best Observed Orncle Penalized LogP Score
Offine MBO Method
L-BEGS
CMA-ES
BO-gEl
50
TuRBO q
BONET
COM
0-
RoMA
BDI
Backfien
-50
GAGA
1500
2000
Oracle Query Budget
Figure B2: 100th Percentile Oracle Scores versus k-Shot Oracle Budget Size We plot the 100th percentile oracle Penalized LogP score averaged across 10 random seeds as a function of the number of allowed oracle calls k.
B.9 the optimization budget sufficient for optimization convergence?
For all of our experimental results, we restrict the surrogate query budget to total of 2048 allowed offline surrogate model queries in order to ensure fair comparison between different optimization methods. To ensure that such budget is sufficient for optimizer convergence across different optimization methods, we plot the best achieved oracle Penalized LogP value (i.e., assuming an unlimited oracle evaluation budget) (Supplementary Fig. B3) for the Penalized LogP task. These results show that our methods are indeed able to converge the course of the optimization trajectory.
function of the number of optimizer surrogate queries
23

---

140
Bext Observed Oracle Penalized LogP
120
L-BPGS
CMA-ES
100
BO-GEI
TuRBO-qEI
60
COM
RoMA
40
Beaten
ROMO
20
1000
Number of Surrogate Queries
Figure B3: Best Oracle Penalized LogP Value versus Optimization Step Count We plot the best Penalized LogP score averaged across 10 random seeds as function of the number of surrogate queries made over the optimization trajectory. All offline model-based optimization (MBO) methods assessed consistently converge within the allowed oracle query budget used in our experimental setup as described in Section 5.1
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made the abstract and introduction are supported by the experimental results presented. The introduction includes the contributions made in the paper, and important assumptions and limitations are included where relevant.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please see Section in the main text for a focused 'Limitations" section and relevant discussion.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
25

---

Answer:
[NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper fully discloses all the relevant information needed to reproduce all reported experimental results of the paper. We have made our code required to reproduce our experimental results available in the Supplementary Material ZIP file.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
26

---

reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: All data and code associated with this paper is open access and includes sufficient instructions to faithfully reproduce the experimental results reported herein. We have made the data and code available in the Supplementary Material ZIP file. The public link to the GitHub repository containing the code will be included in the Abstract in the final version (currently not linked so to comply with double-bline review). All datasets used herein are publicly available without any limitations public accessibility. Our data and code adhere to the NeurIPS code and data submission guidelines
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
www.uammmiiiiooooooioooo.. for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
wwtpc///giiesseee Swtp///oooeSSsssiieeieeeea wwnniisssioioicyoy.. for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The experimental setting is clearly described in Section 5.2 and Appendix A and also reproducible via the code released alongside the paper in the Supplementary Material ZIP file.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: All statistics and results included in the paper are accompanied by confidence intervals. We clearly describe the factors of variability the confidence intervals Section 5.2 and Appendix A All additional details are included in relevant table captions.
27

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the eπror bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Information for the resources required to reproduce the experiments are included in Section the main paper. More specifically, all experiments on a single internal cluster with NVIDIA RTX A6000 GPUs. Any particular experimental configuration required more than 24 hours complete using our setup. The full research project did not require more compute than the experiments reported in the paper.
Justification:
performed
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http:///wwwpp..ccooooo/.ccccccca
Answer: [Yes]
Justification: The authors have reviewed the NeurIPS Code of Ethics and assert that the research described herein conforms with the Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
28

---

10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please see the conclusion (i.e. Section 6 in the main text for focused 'Impact Statement" subsection and relevant discussion on the potential societal impacts our work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairesss considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper does not introduce any assets that have high risk for misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
29

---

Justification: The only existing assets used in the paper are the seven MBO datasets used for experimental evaluation as described Section [. All of the relevant datasets are publicly available, and the references for each of the datasets are cited in the aforementioned section. The licenses associated with each of the seven datasets made available in each of the relevant citations are properly respected. There are no restrictions with respect to accessing any of the datasets used in the paper. No scraped data was used this paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
The only asset introduced and released alongside the paper is the experimen- tal code to reproduce the reported results. The repository containing the code is included in the Supplementary Material ZIP file. All datasets used for the experiments discussed herein are publicly available and available online via the appropriate references in this paper and via online links included in the Supplementary Material ZIP file.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The research described herein does not involve crowdsoureing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
30

---

According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The research described herein does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
31


Reviews:
[{"review_id": "muldQIEMTL", "reviewer": "Reviewer_czFU", "summary": "This paper proposes to use an adversarial regulariser in the BO setting. In particular, the authors propose a systematic way to compute the regularization parameter through a lagrange duality. Overall the regularized method performs on average better than existing methods across a suite of benchmark datasets.", "questions": "see above", "limitations": "yes", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper is the first to propose an adversarial regularization objective for BO.\n- The method is well motivated and a practical way to obtain the hyperparameter is given, albeit through a grid.\n- The proposed methods performs on par with baselines and on average is better across several benchmark datasets.", "weaknesses": "- to clarify, the way you choose alpha is through a grid. Wouldnt that mean your computational cost is significantly larger than baseline methods, please add a section on computational time as well as cost to better understand the tradeoffs of the proposed method over existing methods. Please comment on the above and clarify any misunderstandings please. Especially this part: \"To ensure a fair evaluation schema, all MBO methods were evaluated using a fixed surrogate query budget of 2048\". Can you confirm that no extra data was used to obtain the alpha in your algorithm? There might be a misunderstanding here, in particular what how exactly to pick the best alpha based on the grid of 200 and the associated costs?\n- VAEs as well as GAN are often time hard to train and need very specific architectures to work well. How, were these chosen and is there any ablation study? how does the quality of \"c\" affect the problem setting? My concern is that there are so many moving parts that is becomes hard to understand what the contribution of each part is\n- I dont understand D(best) how come some values in the table are able to be better than the oracle? Am i reading the table correctly?\n\n\nI am more than happy to change my score if the above have been clarified", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer czFU for their thoughtful review of our work, and appreciate that they recognize the technical and empirical contributions of GABO compared with prior work. Please find our responses to their questions below.\n\n**Search for $\\alpha$**\n\nImportantly, we confirm that no extra data is used to compute the value of $\\alpha$ in our algorithm -- because only the source critic $c^*$ is retrained dynamically over the optimization process, we only need to query the easy-to-compute neural network $c^*$ in computing $\\alpha$.\n\nWe also confirm that we compute $\\alpha$ through a grid search. However, in our implementation used for our experiments (which will be made publicly available), the grid search to compute $\\alpha$ is highly vectorized such that the computational time is non-limiting even using a single-GPU setup. To benchmark our implementation, we evaluate both BO and Gradient Ascent (GA) both with and without our Generative Adversarial (GA) source critic regularization algorithm on the Branin and Penalized LogP optimization tasks:\n\n| **Method** | **Branin Task Compute Time (sec)** | **LogP Task Compute Time (sec)** |\n| --- | --- | --- |\n| BO-qEI | 92.1 $\\pm$ 10.2 | 965 $\\pm$ 17.9 |\n| GABO | 328 $\\pm$ 146 | 1245 $\\pm$ 55.2 |\n| GA | 9.68 $\\pm$ 0.23 | 765 $\\pm$ 6.64 |\n| GAGA | 75.6 $\\pm$ 25.4 | 818 $\\pm$ 10.5 |\n\nCompute time is reported as mean $\\pm$ standard deviation over 10 random seeds. As a reminder, the Branin task is a standard benchmarking task for offline optimization, and the Penalized LogP task is subjectively the most challenging task assessed in our manuscript with the highest dimensional design space out of the seven assessed tasks.\n\nWhile there are obviously additional compute costs associated with running our Source Critic Regularization method (i.e., **Algorithm 1**), as made evident by the above results, we note that in most applications of offline optimization, obtaining labeled data is the main bottleneck in many practical applications; thus, it is often worth spending this extra compute to ensure the best results for the given budget.\n\n**Selection of Model Hyperparameters**\n\nThe exact same generic FCNN-based source critic as introduced by the initial WGAN authors [Arjovsky et al. (2017)](https://arxiv.org/abs/1701.07875) is used across all of the different tasks assessed in our work. Similarly, we perform **no** hyperparameter fine-tuning of VAEs and use standard task-agnostic VAE architectures across the different tasks in our work. By using generic model architectures that have *not* been optimally tuned to each task, we can focus on assessing our contributions that are primarily algorithmic in nature. That is, the VAEs and source critic models may perform well on certain tasks and may perform poorly on others that were assessed. By evaluating our source critic regularization algorithm across a wide variety of different benchmarking tasks, we are able to give a good picture of the \"average\" performance of GABO and other optimizers independent of how well VAEs and source critics are tuned to the specific optimization task at hand.\n\n**What is D(best)?**\n\nIn our results, $\\mathcal{D}$(best) refers to the best oracle score achieved by any previously observed design in the offline dataset. For example, in **Table 1**, the value of $\\mathcal{D}_{\\text{best}}=11.3$ for the LogP task means that out of all the 79,564 unique molecules and their corresponding oracle LogP values in the base offline dataset associated with the LogP task, the maximum score achieved by any given molecule is 11.3. Therefore, if an optimization method (such as GABO) proposes a molecule design that achieves an oracle LogP score greater than 11.3, then we have found a design better than any of the designs previously seen in the offline dataset. Our choice of notation is consistent with that used in related work ([Krishnamoorthy et al. Proc ICML 2023](https://arxiv.org/abs/2206.10786), [Krishnamoorthy et al. Proc ICML 2023](https://arxiv.org/abs/2306.07180), [Trabucco et al. Proc ICML 2021](https://arxiv.org/abs/2107.06882), [Chen et al. Proc NeurIPS 2022](https://arxiv.org/abs/2209.07507)).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "K0FIJPuVKx", "reviewer": "Reviewer_U2YG", "summary": "This paper proposes a novel approach for offline MBO that combines Source Critic Regularization and Bayesian Optimization. Offline MBO aims to train a surrogate model from offline data and subsequently extrapolates it to find an optimal design (as opposed to online MBO which collects more data as it trains the surrogate model). One key challenge of offline MBO is knowing when to trust the surrogate model, since it could be highly erroneous at ood inputs. This paper handles said challenge by incorporating a critic constraint that penalizes candidate that are not sufficiently similar to the training samples.", "questions": "- Why run BO on the learned surrogate when it is already differentiable and can be numerically optimized using GA? (I saw the ablation study on GAGA but just want to hear an argument for the benefit of GABO).", "limitations": "No potential negative societal impact", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- Clear description, easy to follow\n- Practical and novel method for MBO. As a regularization method it can be incorporated into many other MBO frameworks.\n- Hyperparameter \\lambda is chosen methodically\n- Perform well on 100th percentile metric", "weaknesses": "- Doesn't seem to be very good on 90th percentile metric (i.e., mean rank ~ 8). Does this mean that the method is very good at picking out high-reward outlier, but the overall quality of the surrogate is about the same as everything else? If that's the case is there any justification for this behavior? I would prefer if the authors plotted out the entire scatter plot of 100 candidates for each method, as that would allow us to compare performance on a distributional level.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer U2YG for their thoughtful feedback on our work, and address their outstanding questions and concerns below.\n\n**Performance of GABO on the 90th percentile metric**\n\nUnlike the other evaluation metrics assessed in our work, GABO indeed does not perform as well on the 90th percentile metric. However, this is not surprising since GABO is not designed to target this metric (and it is not our metric of interest). In particular, in our analysis, we have found that this is largely due to the nature of the underlying Bayesian optimization (BO) optimization algorithm. Because BO is not an iterative first-order algorithm, the designs proposed by any BO-based algorithm often have high variance in practice. This is what we observe across all of our experiments, including in **Table 1** and **Supplementary Tables B1 and B3**.\n\nWe also note that in most offline optimization applications, the 90th percentile metric$\\textemdash$or any metric that does not use for the best proposed design(s)$\\textemdash$is not as useful as the other metrics assessed where GABO *does* perform well. This is because in offline optimization tasks with a restricted budget to query the hidden, expensive-to-evaluate oracle function, we are not interested in \"wasting\" this budget on subpar design candidates. While the 90th percentile and similar metrics can be helpful to understand the limitations of algorithms (as in this case), we believe that the 100th percentile metrics reported in the main text are more useful and practical in assessing each of the optimization algorithms.\n\nFor completeness, we also show the distributions of the proposed designs for across all of the optimization methods for the LogP task to allow interested readers to compare the performance of the different methods on a distributional level. The plot is available here: [Link](https://bashify.io/files/dxnMON). We will include this result in the final manuscript.\n\n**Why run BO on the learned surrogate when it is already differentiable and can be numerically optimized using GA?**\n\nThank you for this question. We choose to use BO for the basis of our work because it is a well established principle in recent Bayesian Optimization (BO) literature that BO is useful even for differentiable objectives, outperforming baselines such as GA [Eriksson et al. (NeurIPS 2019)](https://arxiv.org/abs/1910.01739), [Maus et al. (NeurIPS 2022)](https://arxiv.org/abs/2201.11872), [Hvarfner et al. (2024)](https://arxiv.org/abs/2402.02229), [Eriksson et al. (UAI 2021)](https://proceedings.mlr.press/v161/eriksson21a/eriksson21a.pdf), [Astudillo et al. (ICML 2019)](https://arxiv.org/abs/1906.01537). It is the dimensionality and non-convexity of the search space that makes the optimization problems in our benchmark challenging, leading first order methods like GA to struggle. Based on these results, we believe BO is a natural basis for our work. We discuss this in detail in **Section 3.2** (L110-7).\n\nNevertheless, we emphasize that our proposed source critic regularization (SCR) algorithm can be used with other optimization methods as well, such as Gradient Ascent (GA) as suggested. Indeed, as already noted by the Reviewer, we also evaluated SCR with GA, which we call Generative Adversarial Gradient Ascent (GAGA), and include results in **Supplementary Table B5**. Comparing GABO with GAGA helps motivate our decision to use BO for experimental evaluation of our SCR algorithm. To better emphasize that our main contribution is SCR, we plan to revise the title of our work to the following: **Generative Adversarial Model-Based Optimization via Source Critic Regularization**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "o5c0Rc6FnD", "reviewer": "Reviewer_bMXC", "summary": "The paper considers the problem of offline black-box optimization where only limited (zero-shot or few-shot) online interactions with the objective function is available. Existing approaches commonly train a neural net parametrized surrogate model of the objective using the offline data. The paper proposes to use a source critic model, inspired by discriminator training in generative adversarial networks, to regularize this surrogate. This is accomplished by formulating a constrained optimization problem which constrains the optimization (over the surrogate model) trajectory samples to be similar to the training data using the source critic model. The Lagrangian version of this constrained optimization is used as the objective for a standard batch mode Bayesian optimization algorithm. Experiments are performed on tasks from Design-bench benchmark and a new Warfarin task.", "questions": "Please see weaknesses section.", "limitations": "Please see weaknesses section.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- One thing I really like about the paper is its evaluation which truly mirrors the offline optimization setting. The paper compares the proposed approach and baselines on a single evaluation from the oracle which I believe is the right way to evaluate algorithms for offline model-based optimization.\n\n- The proposed approach shows good performance across all the tasks on both zero-shot and few-shot evaluation. The ablation study in 5.5 also shows that adaptively tuning the lagrangian coefficient is also important for good performance.\n\n- The constrained optimization formulation is well-formulated and principled approach to tackle this problem. The design choice of constraining the surrogate model using the source critic model makes sense and is relevant for the problem.\n\n- Some of the tasks in design-bench benchmark have multiple errors which makes them not so informative for evaluation. I like the paper didn't include superconductor task where the original offline dataset itself has multiple copies of the same inputs but with different outputs. Similarly, I think the ChemBL and UTR tasks are also not useful. In my practical experience, samples searched over ChemBL generates a lot of syntactic errors. I like the fact that the paper introduces new tasks like LogP and Warfarin which will be useful to the broader community.", "weaknesses": "- One question I have is about the broader picture of using Bayesian optimization (BO) for this problem space. Why should we first train the neural network surrogate model and then fit the Gaussian process (GP) on top of the neural surrogate model to do few steps of BO? Why can't we just fit a GP (with a latent space presumably to handle high/structured dimensionality) directly on the offline data and do one step of BO? I am assuming the TurBO-qEI baseline doesn't do that and also fits the Turbo's GP on the neural network surrogate.\n\n- It would be nice to see more justification for the BO hyperparameters (like sampling budget (T) and batch size (b)). Is the method sensitive to the choice of these parameters?\n\n- One thing I like about the paper is that it can work with any choice of the surrogate model. I would also like to point out one very recent related work that also constrains the optimization given any surrogate model by formulating the search as an offline RL problem.\n\n        - Yassine Chemingui, Aryan Deshwal, Nghia Hoang, Janardhan Rao Doppa. Offline Model-based Black-Box Optimization via Policy-guided Gradient Search. Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI), 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer bMXC for their thoughtful comments and insights, and share their enthusiasm for the proposed significance and strengths of our work. We address their outstanding questions in our response below.\n\n**The Role of the Surrogate NN in MBO**\n\nWe thank the reviewer for this comment. We use this setup for generality; in many applications, the surrogate objective is not under our control. For instance, it might be a physics simulator or other domain-specific function that has been optimized by domain experts to give more information about the hidden oracle function across the entire design space than what can be accomplished with GPs alone. (Note that task-specific optimization of the surrogate is *not* done in our work.) By fitting a GP to this surrogate instead of just the observations of the hidden oracle function, we can take advantage of the information encoded in a more complex (and hopefully accurate) surrogate.\n\nTo demonstrate this empirically, we explore this potential strategy of using the single GP from BO directly as the both the surrogate function *and* for BO sampling. We leverage Source Critic Regularization (i.e., **Algorithm 1**) using this framework. We refer to this as Generative Adversarial Bayesian Optimization with a GP Surrogate (GABO GP-Surrogate), and compare this method with GABO:\n\n| One-Shot Oracle Evaluation | Branin          | LogP            | TFBind            | GFP             | UTR             | ChEMBL          | Warfarin | Avg. Rank |\n| -------------------------- | ------ | ---- | ------ | --- | --- | ------ | -------- | --- |\n| **GABO GP-Surrogate**          | -37.4 $\\pm$ 4.4 | -57.9 $\\pm$ 159 | **0.576 $\\pm$ 0.058** | 3.51 $\\pm$ 0.69 | 6.84 $\\pm$ 1.24 | **0.65 $\\pm$ 0.01** | -0.27 $\\pm$ 2.13 | 7.7 |\n| **GABO**                       | **-2.6 $\\pm$ 1.1** | **21.3 $\\pm$ 33.2** | 0.570 $\\pm$ 0.131 | **3.60 $\\pm$ 0.40** | **7.51 $\\pm$ 0.39** | 0.60 $\\pm$ 0.07 | **0.60 $\\pm$ 1.80** | **4.3** |\n\n| $k=64$-Shot Oracle Evaluation | Branin          | LogP            | TFBind            | GFP                | UTR             | ChEMBL          | Warfarin | Rank |\n| ----------------------------- | ------ | ---- | ------ | --- | --- | ------ | -------- | --- |\n| **GABO GP-Surrogate**             | -8.4 $\\pm$ 1.6  | 0.66 $\\pm$ 0.01 | 0.720 $\\pm$ 0.068 | 3.74 $\\pm$ 0.00 | **8.27 $\\pm$ 0.08** | 0.66 $\\pm$ 0.01 | 0.98 $\\pm$ 0.05 | 6.6 |\n| **GABO**                          | **-0.5 $\\pm$ 0.1** | **98.0 $\\pm$ 37.6** | **0.942 $\\pm$ 0.026** | **3.74 $\\pm$ 0.00** | 8.25 $\\pm$ 0.17 | **0.67 $\\pm$ 0.02** | **1.00 $\\pm$ 0.28** | **2.1** |\n\nAs we can see, using a more complex, neural-network surrogate function for GABO leads to better optimization results than directly using the GP as the surrogate function.\n\nOur evaluation strategy of using an neural network surrogate for GABO is also identical to the one used in the existing offline MBO literature to make it easier to compare our work with pre-existing methods: for instance, [Trabucco et al. (ICML 2021)](https://arxiv.org/abs/2107.06882), [Yu et al. (NeurIPS 2021)](https://arxiv.org/abs/2110.14188), [Chen et al. (ICML 2023)](https://arxiv.org/abs/2301.02931), [Krishnamoorthy et al. (ICML 2023)](https://arxiv.org/abs/2306.07180), and [Krishnamoorthy et al. (ICML 2023)](https://arxiv.org/abs/2206.10786) in addition to our work.\n\n**Sensitivity to Sampling Budget and Batch Size**\n\nThank you for this comment. Prior work has shown that BO methods are relatively robust to perturbations in the batch size and other hyperparameters (i.e., Figure 7 in [Eriksson et al. (NeurIPS 2019)](https://arxiv.org/abs/1910.01739)), and we have observed that this similarly applies to GABO. We will include a discussion on the sensitivity to the optimization hyperparameters in an updated version of our manuscript.\n\n**Additional Citation**\n\nThank you for sharing this related work from Chemingui et al. with us. We will include a citation to this work and appropriate associated discussion in the final manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mfmBdDW9ZG", "reviewer": "Reviewer_Yd2B", "summary": "This paper considered an offline optimization problem where a surrogate objective function instead of the oracle objective can be queried. The surrogate objective function is trained using a reference offline dataset and thus may falsely predict the optimum due to overestimation errors. To resolve this issue, this paper proposed a Generative Adversarial Bayesian Optimization (GABO) algorithm which exploits the adaptive Source Critic Regularization (SCR) to achieve robust oracle function optimization. The proposed method is demonstrated to outperform other tested offline optimization baselines on several synthetic and real-world datasets.", "questions": "In Lines 210-213, it is mentioned that the objective of GABO is time-varying, which is an interesting and non-trivial issue of applying BO to this problem. Do you have any idea about how to resolve this issue?", "limitations": "This limitations are clearly discussed.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The idea of adding a regularization penalty to the original offline optimization problem is interesting. The technical issues are clearly introduced.\n\n2. The empirical performance improvement of the proposed GABO is significant compared to the other BO and model-based optimization algorithms.", "weaknesses": "I have reviewed this paper before and had multiple rounds of discussions with the authors in the previous review process. My major concern is still about the motivation and positioning of this work. The key contribution of this work is the design of the surrogate objective in (7) and its dual formulation in (8) for a model-based optimization (MBO) problem. This new objective is independent of BO and can actually be optimized using the zero-order, one-order, or any other optimization algorithm. Even though this paper has highlighted and empirically shown that BO is an effective choice among multiple optimization methods, I still think the combination of SCR and BO is trivial and should not be the focus of this work.\n\nI cannot consider the proposed GABO algorithm as a novel **BO** framework since all the new techniques proposed are not specifically designed for BO. The challenges of applying BO in the generative adversarial optimization problem are unclear. I highly suggest the authors to reconsider the title and motivation of this work such that its contributions can be clearer.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer Yd2B for their thoughtful review of our work, and appreciate that they recognize our proposed source critic regularization (SCR) formulation and strategy as a key contribution of our work.\n\n**Framing of the Manuscript's Narrative**\n\nWe agree that our main contribution is Source Critic Regularization (SCR) in **Algorithm 1**, which is optimizer-agnostic and can be used with any optimization method. Our experimental evaluation of SCR using Bayesian Optimization (BO) is because we find it to be the most effective optimizer to use in conjunction with SCR, likely due to BO's well-documented effectiveness as an offline optimizer as outlined in **Section 3.2** (L110-7). Indeed, as acknowledged by the Reviewer, we have already included evidence of the effectiveness of BO compared to alternatives such as Gradient Ascent (GA)$\\textemdash$in particular, our comparison to Generative Adversarial Gradient Ascent (GAGA) in **Supplementary Table B5**. In addition, GABO also outperforms vanilla BO in all our experiments, demonstrating that SCR has significant added value as a methodological contribution.\n\nIn alignment with the feedback from the Reviewer, we are happy to revise the title of our work to the following: **Generative Adversarial Model-Based Optimization via Source Critic Regularization**. This improvement will help better emphasize that our main contribution is SCR independent of the choice of baseline optimizer. We will also do our best to revise our paper to make it more clear that SCR is our main contribution, and that BO is simply the most effective instantiation of our method. \n\n**Future Work on Time-Varying Optimization**\n\nThank you for this comment; indeed time-varying BO is an interesting and nascent field of research and a complex problem in general. We hypothesize that there may be opportunities to exploit existing methods in time-varying BO for SCR if the updates to the source critic function $c^*$ follow some sort of pattern over time. In practice, we were unable to identify any such patterns in preliminary studies largely due to the limited number of updates to $c^*$ made over the course of the optimization process. Furthermore, factoring in time-varying BO was not necessary to demonstrate that GABO and SCR are effective empirically. Given the complexity of this problem and the empirical success of GABO and SCR already demonstrated in our work, we leave this opportunity to explore time-varying BO for future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9KlwAgwoxa", "reviewer": "Reviewer_AH6U", "summary": "This work tackles offline Bayesian optimization using adaptive source critic regularization. The authors propose generative adversarial Bayesian optimization, which optimizes against a learned surrogate model without querying the true oracle function during optimization. It utilizes a Lipschitz-bounded source critic model to constrain the optimization trajectory. Finally the authors provide experimental results against various baseline methods.", "questions": "- Could you explain how these examples \"Evaluating newly proposed molecules requires expensive experimental laboratory setups, and testing multiple drug doses for a single patient can potentially be dangerous\" agree with your problem formulation?\n- This sentence \"Leveraging source critic model feedback for adversarial training of neural networks was first introduced by Goodfellow et al. (2014)\" is not true. The authors should revise it.\n- I think that this work just proposes a better regression method in the offline setting.\n- Can you provide the details of $f_\\theta$? Why did you choose such parameterization?\n- How does a learned surrogate model provide additional information without querying the true oracle function? Source critic regularization doesn't add anything.\n- When do you evaluate a true objective function? Didn't you evaluate query points when comparing your method to baseline methods?\n- Equation (10): Is argmin correct? Should it be min?\n- Can you explain the rationale behind $\\lambda = \\alpha / (1 - \\alpha)$?\n- Tables 1 and 2: How can the baseline methods show the results over the best in $\\mathcal{D}$? I think that it is bounded in $\\mathcal{D}$.\n- Tables 1 and 2: What is the purpose of the row of the best? How can I interpret the results better than the best?\n- It is a minor thing, but \"bolded\" is not a correct word. It should be just \"bold.\"", "limitations": "There is no particular societal limitations of this work.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "- The setup of offline Bayesian optimization seems interesting.", "weaknesses": "- The rationale behind the proposed method is unclear.\n- Details of experiments are missing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We respond to Reviewer AH6U's comments below:\n\n**Rationale of SCR**\n\nTo summarize our work and as discussed in our manuscript, **the primary rationale behind our proposed source critic regularization (SCR) method is to stop optimization algorithms from extrapolating against offline surrogate models in generative design.** We accomplish this by showing how to optimally and tractably balance the tradeoff between exploring the design space for potentially better designs while ensuring we don't over-extrapolate against the learned surrogate model.\n\n**Experimental Details**\n\nWe have described the details of our experiments in **Section 5** and **Appendix A** of our manuscript, and also have made our code available for easy reproducibility and transparency. If the Reviewer has any remaining questions on experimental details, please feel free to let us know.\n\n**SCR is not a regression method.**\n\nTo clarify, our work is not proposing a regression method; instead, **the primary contribution of SCR is to show how to balance the tradeoff between (1) optimization against a regressor model and (2) staying in-distribution in a computationally tractable way in offline MBO**.\n\nTo validate this primary contribution empirically, we make use of generic, task-agnostic surrogate regression models trained using standard machine learning techniques (see **Section 5.2**, L271-5 for details). We do not optimize the hyperparameters of our learned surrogate regression models in any way, and use the same generic parametrization of $f_{\\theta}$ across all tasks (L271-2). There are no special training methods that were used in constructing the surrogate regressor models. We are happy to clarify further if the Reviewer has any additional questions regarding the primary motivation and contributions of our work.\n\n> ***How does a learned surrogate model provide additional information without querying the true oracle function? Source critic regularization doesn't add anything.***\n\nThe learned surrogate model provides information on the true oracle function because it is trained on historical designs and their corresponding true oracle function scores (represented as the dataset $\\mathcal{D}$ in our manuscript). In regions of the design space that contain many examples of these historical designs in $\\mathcal{D}$, the surrogate model agrees with the true oracle function because it has been trained on past observations of the true oracle function. However, in regions of the design space that do *not* contain many examples of historical designs, the surrogate model and true oracle function likely disagree due to the problem of **extrapolation**. A good illustration of this is shown in **Figure 1** of our manuscript.\n\n**The value added by source critic regularization is in preventing optimization algorithms from taking advantage of these extrapolated regions of the design space that result in falsely \"optimal\" designs.** In offline optimization, we want to avoid such designs that \"look promising\" according to the surrogate but in reality score poorly on the true oracle due to extrapolation of the surrogate model.\n\n> ***When do you evaluate a true objective function?***\n\nFor each of the optimization methods (including GABO), we query the true objective function once (for **Table 1**) as the final step after an offline optimization algorithm is finished running and proposes a single design to evaluate using the hidden oracle objective function. This evaluation strategy is aligned with the motivation for this work; after we perform offline optimization experiments to propose a candidate design(s), we need to evaluate these designs with the actual oracle function to see how good these designs actually are. This evaluation schema is consistent with that used in most (if not all) other related work in offline generative design.\n\n**Reparametrization of $\\lambda**\n\nThe rationale behind re-parameterizing $\\lambda$ in terms of $\\alpha$ is that the feasible search space for $\\lambda$ is all non-negative real numbers, which cannot be tractably searched over in a clear way. Our approach to solve this problem is to instead re-parameterize $\\lambda$ in terms of $\\alpha$ so that searching over a finite $\\alpha\\in[0, 1]$ is \"equivalent\" to searching over $\\lambda\\in[0, \\infty)$.\n\n**Interpretation of $\\mathcal{D}$(best) in Tables 1 and 2**\n\nRecall that $\\mathcal{D}$ refers to the dataset of  historical designs and their corresponding true oracle function scores that was used to train the learn surrogate model. In Tables 1 and 2, $\\mathcal{D}(\\text{best})$ refers to the best oracle function score observed from this set of prior historical designs. For example, in **Table 1**, the value of $\\mathcal{D}(\\text{best})=11.3$ for the LogP task means that out of all the 79,564 unique molecules and their corresponding oracle LogP values in the base offline dataset associated with the LogP task, the maximum score achieved by any given molecule is 11.3.\n\nTherefore, if an optimization algorithm like GABO proposes a design with an oracle score greater than the best in $\\mathcal{D}$ (e.g., a molcule with an oracle LogP score greater than 11.3), this means that we have \"discovered\" a new design not previously seen in the historical dataset that is even better than the best design in the historical dataset. Finding designs better than $\\mathcal{D}(\\text{best})$ is the main goal of offline optimization for generative design.\n\nOur choice of notation and in reporting $\\mathcal{D}(\\text{best})$ is consistent with that used in other related work ([Krishnamoorthy et al. Proc ICML 2023](https://arxiv.org/abs/2206.10786), [Krishnamoorthy et al. Proc ICML 2023](https://arxiv.org/abs/2306.07180), [Trabucco et al. Proc ICML 2021](https://arxiv.org/abs/2107.06882), [Chen et al. Proc NeurIPS 2022](https://arxiv.org/abs/2209.07507)).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eGjqMDEnPM", "reviewer": "Reviewer_yPeb", "summary": "The paper proposed GABO, a novel Bayesian optimization method for offline model-based optimization problems. GABO regulates the surrogate model with source critic actor so that the BO procedure remains in the in-distribution. Experiment results validate that GABO outperforms several baselines in terms of the mean rank.", "questions": "- In the experiment part, there is a new task called Warfarin, which is a conditional offline MBO task. I think the problem setting is practical and important. However, it seems that the proposed method lags behind other baselines in terms of performance. Could authors elaborate on why this phenomenon happens?\n\n- It is also hard for me to accept the results of BONET and DDOM in the Branin task, as both papers deeply analyze the behavior of their methods in the Branin function. Is it due to the different evaluation settings?", "limitations": "As written in the weakness section, authors should specify the scope of the research and compare their method with proper baselines. Furthermore, the authors should explain why the evaluation setting has been changed.\n\nThere are a few minor comments on the manuscript.\n\n- In the related work part, it might be beneficial that authors clearly state the limitations of prior methods rather than just explain the methods.\n\n- In the background part, it might mislead readers if we define offline MBO as solving the optimization problem in Eq (2). There are several methods that formulate the problem as conditional generative modeling. Even for forward approaches, they do not solve the problem in Eq (2) and propose various approaches such as regularization.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- Bayesian optimization is a widely used algorithm for black-box optimization but unexplored in offline MBO settings. As far as I know, it is the first paper that improves BO for offline MBO settings\n\n- Mathematical formulation seems to be valid, and practical algorithm is given", "weaknesses": "- It seems that the tasks used for evaluation from the Design Bench are discrete, **biological sequence design tasks**. There are several continuous tasks, such as Superconductor, Ant, and Dkitty, which are also high-dimensional and challenging problems. I think the research scope is then limited to offline biological sequence design, not general offline optimization tasks. Then, authors should compare their method with papers specialized in biological sequence designs, such as BIB[1] and BootGen[2].\n\n- If my understanding is correct, the **evaluation procedure is a bit different from the offline MBO conventions**. The authors first train a surrogate model and choose the top-1 candidate among 2048 candidates according to the predicted score of the surrogate model. As the surrogate model gives inaccurate predictions on the data points outside of the offline dataset, it is not convincing to choose the best one with the surrogate model for evaluation. The authors should elaborate on the reason why they changed the evaluation setting.\n\n[1] Chen, Can, et al. \"Bidirectional learning for offline model-based biological sequence design.\" International Conference on Machine Learning. PMLR, 2023.\n\n[2] Kim, Minsu, et al. \"Bootstrapped training of score-conditioned generator for offline design of biological sequences.\" Advances in Neural Information Processing Systems 36 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer yPeb for their thoughtful comments and insights. We address their outstanding questions in our response below.\n\n**Research Scope**\n\nWhile we focus on the discrete, biological sequence design tasks from Design Bench, it is important to recognize that GABO and Source Critic Regularization (SCR) can be used for both continuous and discrete tasks (unlike BIB and BootGen). This is why we evaluate SCR and GABO on continuous optimization tasks in our experiments, such as the Branin and Warfarin tasks. In choosing which Design bench tasks to evaluate, we focus on the biological sequence design tasks because these tasks have been the most reproducible across recent work, as highlighted by Reviewer bMXC. The research scope is *not* limited to offline biological sequence design.\n\nNonetheless, we evaluate BootGen on the discrete biological sequence design tasks to compare its performance against GABO for these subset of tasks. Our results are shown here:\n\n| Top $k=1$ Evaluation | LogP | TFBind | GFP | UTR | ChEMBL | Avg. Rank |\n| --- | --- | --- | --- | --- | --- | --- |\n| BootGen | -59.1 $\\pm$ 69.2 | 0.398 $\\pm$ 0.002 | **3.60 $\\pm$ 0.03** | **7.61 $\\pm$ 0.55** | **0.63 $\\pm$ 0.02** | 6.3 |\n| GABO | **21.3 $\\pm$ 33.2** | **0.570 $\\pm$ 0.131** | **3.60 $\\pm$ 0.40** | 7.51 $\\pm$ 0.39 | 0.60 $\\pm$ 0.07 | **4.3** |\n\n| Top $k=64$ Evaluation | LogP | TFBind | GFP | UTR | ChEMBL | Avg. Rank |\n| --- | --- | --- | --- | --- | --- | --- |\n| BootGen | 30.8 $\\pm$ 14.2 | 0.401 $\\pm$ 0.000 | 3.62 $\\pm$ 0.00 | **8.29 $\\pm$ 0.57** | 0.65 $\\pm$ 0.00 | 6.6 |\n| GABO | **98.0 $\\pm$ 37.6** | **0.942 $\\pm$ 0.026** | **3.74 $\\pm$ 0.00** | 8.25 $\\pm$ 0.17 | **0.67 $\\pm$ 0.02** | **2.1** |\n\nThe main conclusions of our manuscript are not changed after the inclusion of BootGen as an additional baseline method. We are unable to compare GABO against BIB due to limitations in the BIB authors' publicly available code implementation.\n\n**Top-1 Experimental Evaluation**\n\nThank you for this comment. As discussed in **Section 1**, of our manuscript, the primary motivation in evaluating the top-1 candidates is that **real-world use cases for offline generative design do not have access to large oracle query budgets**. This evaluation schema is actually studied in other related offline MBO work (e.g., [Kim et al. Proc NeurIPS 2024](https://arxiv.org/abs/2306.03111)). This evaluation schema is important because offline optimization is most helpful when evaluating newly proposed molecules requires expensive experimental laboratory setups, or when we want to optimize a patient's drug regiment without being able to test multiple doses on that patient. In these settings, the evaluation of up a large number of designs (sometimes as much 256) as in prior work is not feasible and not representative of how an algorithm would perform in practice. This change in evaluation setup is also why BONET and DDOM perform worse on the Branin task in this more realistic setting.\n\nIn fact, GABO actually performs ***better*** using the more standard top-64 metric (**Supp. Tables B1 and B2**). In particular, GABO achieves an average rank of 2.1, which is better than any other reported method. Thus, our motivation for showing top-1 results is only because we believe it is more practical and relevant for real-world applications of offline model-based optimization.\n\nWe also cite Reviewer bMXC's review as well for additional insights. In particular,\n  - \"*One thing I really like about the paper is its evaluation which truly mirrors the offline optimization setting. The paper compares the proposed approach and baselines on a single evaluation from the oracle which I believe is the right way to evaluate algorithms for offline model-based optimization.*\"\n\n**Performance on the Warfarin Task**\n\nThank you for raising this point, and we appreciate that the Reviewer recognizes the importance of introducing practical tasks such as the Warfarin task. Compared with the other tasks assessed, the landscape of true oracle function for the Warfarin task (a LASSO model reported previously by domain experts) is uniquely a smooth convex function, and the trained surrogate likely captures similar properties over the design space. As a result, we hypothesized that this task was more conducive to first-order offline optimization methods, which is exactly what is shown in our results.\n\n**Definition of Offline MBO in Eq (2)**\n\nThank you for this comment; we used the definition of offline MBO as in Eq (2) because (1) it is the definition of offline MBO that best frames the problem setup for our proposed approach of SCR; and (2) it is the definition consistent with the original Design-bench publication from [Trabucco et al. CoRR 2022](https://arxiv.org/abs/2202.08450). In related work that study forward approaches to offline MBO (e.g., [Yu et al. Proc NeurIPS 2021](https://arxiv.org/abs/2110.14188); [Trabucco et al. Proc ICML 2021](https://arxiv.org/abs/2107.06882); [Chen et al. Proc NeurIPS 2022](https://arxiv.org/abs/2209.07507)), the authors similarly first consider a motivating problem setup for offline MBO identical to Eq (2), and then extend on it to solve a related problem through their own methodological contributions as pointed out by the Reviewer. This is how we approached framing the problem formulation to motivate SCR as well.\n\nHowever, we also agree with the Reviewer that a few recent related work have used alternative formulations of offline MBO to motivate their work (e.g., [Kim et al. Proc NeurIPS 2023](https://arxiv.org/abs/2306.03111); [Krishnamoorthy et al. Proc ICML 2023](https://arxiv.org/abs/2206.10786); [Krishnamoorthy et al. Proc ICML 2023](https://arxiv.org/abs/2306.07180)). In the final manuscript, we will be sure to clarify that (1) such other methods for offline MBO exist; (2) we consider one possible definition of offline MBO in Eq (2); and (3) how these additional problem formulations relate to Eq (2) considered in our work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# Summary of Revisions Made to the Paper

We thank the Reviewers for their thoughtful comments and consideration of our paper. We are grateful that the Reviewers find our method novel (Reviewer czFU, bMXC, Yd2B, yPeB), well-justified (Reviewer bMXC, Yd2B, yPeB), well-written (Reviewer U2YG), and a useful resource (Reviewer bMXC, yPeB) for the broader scientific community.

We have the following general comments, including several changes we have made to our manuscript to address reviewer comments:
  1. Several reviewers asked about the importance of Bayesian Optimization (BO) in our framework. Indeed, our main contribution is to use Source Critic Regularization (SCR) for Model Based Optimization (MBO), as illustrated in **Algorithm 1**. While SCR can be applied to several different optimization methods, we find BO to be the most effective (**Section 3.2**, **Supplementary Table B5**). This motivates using BO as our vehicle to evaluate SCR. To illustrate that SCR can also be used with other optimization methods, we have also demonstrated how SCR can be used to improve upon Gradient Ascent, resulting in Generative Adversarial Gradient Ascent (GAGA). To better emphasize that our main contribution is SCR, we are happy to change our title to **Generative Adversarial Model-Based Optimization via Source Critic Regularization**.
  2. We have included a plot of the distribution of scores for the Penalized LogP task for different optimizers in the attached PDF file for the interested reader to analyze how the distribution of oracle scores vary across different methods.
  3. We have included further justification for our top-1 evaluation of designs using the oracle, which is different from prior work as noted by some Reviewers. Similar to Reviewer bMXC, we believe that this evaluation metric is "right way to evaluate algorithms for offline model-based optimization." Nevertheless, we emphasize that we have already included top-64 scores in **Supplementary Tables B1 and B2**, which is a "more common" metric in related work. In fact, GABO performs even better in terms of the top-64 metric than it does at our top-1 metric; our focus on top-1 is solely because we believe it to be more relevant to real-world tasks.
  4. We have included additional experimental details regarding the computational time and resources to run our experiments compared with baseline optimization methods, which we discuss in more detail in responses to individual reviewers.

We look forward to discussing further with the Reviewers to answer any outstanding questions or concerns. Thank you!

Meta-review: The reviewers unanimously recommend acceptance (6-6-7-6-5-6). The author-reviewer discussion has been constructive and has led to a number of improvements to the paper, regarding the presentation, the framing of contribution, as well as the empirical evaluation. Proposed changes and new results have been discussed during the rebuttal. The authors are asked to now implement the changes discussed with the reviewers in the final version of the paper.
Decision: Accept (poster)