Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Authors: Michael S Yao
James Gee
Osbert Bastani
Venue: ICML 2025 poster (2025)
Keywords: Offline Optimization
AI4Science
Generative Design
Model-Based Optimization
Primary area: optimization->zeroorder_and_blackbox_optimization
TL;DR: We introduce a method to propose both high-quality and diverse designs in a wide variety of offline model-based optimization tasks.

Abstract: The goal of offline model-based optimization (MBO) is to propose new designs that maximize a reward function given only an offline dataset. However, an important desiderata is to also propose a *diverse* set of final candidates that capture many optimal and near-optimal design configurations. We propose **D**iversit**y** I**n** **A**dversarial **M**odel-based **O**ptimization (**DynAMO**) as a novel method to introduce design diversity as an explicit objective into any MBO problem. Our key insight is to formulate diversity as a *distribution matching problem* where the distribution of generated designs captures the inherent diversity contained within the offline dataset. Extensive experiments spanning multiple scientific domains show that DynAMO can be used with common optimization methods to significantly improve the diversity of proposed designs while still discovering high-quality candidates.

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Michael S. Yao
James C. Gee
Osbert Bastani
Abstract
Baseline BO-qUCB
The goal of offline model-based optimization (MBO) is to propose new designs that maximize a reward function given only an offline dataset. However, an important desiderata is to also pro- pose a diverse set of final candidates that capture many optimal and near-optimal design config- urations. We propose Diversity in Adversarial Model-based Optimization (DynAMO) as novel method to introduce design diversity an explicit objective into any MBO problem. Our key insight is to formulate diversity as a distribu- tion matching problem where the distribution of generated designs captures the inherent diversity contained within the offline dataset. Extensive experiments spanning multiple scientific domains show that DynAMO can be used with common optimization methods to significantly improve the diversity of proposed designs while still discover- ing sigh-qualtty candidates.
DynAMO-Modified BO-QUCB
x
Figure I. Overview of Diversity in Adversarial Model-basee Optimization (DynAMO). Traditional model-based optimiza- tion (MBO) (Trabucco et al., 2021) techniques scoring designs, although often at the expense the diversity of proposed designs. Idcally, the final set of candidates should be of high quality while capturing multiple 'modes of goodness' within the design space. For example, although there are 3 unique global maxima (stars) in the 2D Branin (Branin, 1972) optimization problem, traditional Bayesian optimization (BO-qUCB) proposes designs clustered around only singla optima (diamonds). In con- trast, show how DynAMO can be used to modify the MBO objective to discover diverse and high-quality designs (circles).
generate high-
1. Introduction
Discovering designs that optimize certain desirable proper- ties is ubiquitous task that spans a wide range of scientific and engineering domains. For example, we might seek to design a drug with the most potent therapeutic efficacy (Brown et al., 2019; Kong et al., 2023; Du et al., 2024); build robot that is most capable of navigating complex environ- ments (Ahn al., 2020; Trabucco et al., 2021; Wang al., 2023); or engineer a material with a certain desirable prop- erty (Stanev et al., 2018; Pogue et al., 2023; Gashmard et al., 2024; Ma et al., 2024). However, experimentally validating every proposed design can be expensive, time-intennn iime-intensive, or even impossible in many applications. These limitations can preclude the use of conventional 'online" optimization methods for such generative design tasks.
An alternative approach is instead discover design candi- dates in the offline setting, where we assume that newly proposed designs can be experimentally evaluated during the course of the optimization process. Instead, we only have access to a static dataset of previously observed de- signs and their corresponding reward values. The objective then to propose a (small) of candidate designs to ul- timately evaluate experimentally, with the hope that using the information available in the offline dataset will yield desirable designs in the real-world.
'University of Pennsylvania, Philadelphia, PA, USA. Corre- spondence to: Michael Yao Emya22199 @ew.1199.99ccummmmmmm..mm www..uuen..edu>...
Multiple prior works have proposed a variety of offline opti- mization algorithms (Yu et al., 2021; Trabucco et al., 2021; Fu & Levine, 2021; Chen et al., 2022; Mashkaria et al., 2023;
Proceedings of the 4dnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Krishnamoorthy et al., 2023; Nguyen et al., 2023; Kim et al., 2023). Using static, offline dataset of previously evaluated designs and their corresponding oracle scores, an offline algorithm proposes a small set of final candidate designs that are empirically evaluated using the expensive 'oracle' function. Broadly, these algorithms can be divided into two categories: model-based and model-free, where 'model' refers to a predictive surrogate function trained on the of- fline dataset to approximate the hidden oracle function. We specifically consider model-based optimization (MBO) algorithms (Trabucco et al., 2021) here that explicitly opti- mize against the offline forward surrogate model to discover designs that maximize the final oracle reward.
X [0. 1J of designs generated by the policy maximizes
oracle reward function r(x) : X IR
arg max (x(e(()))()) Tell
(I)
over a design space X. For example, 2 might be a candidate drug, and the reward r(x) the therapeutic efficacy of the drug. However, in many such problems the oracle reward function may be prohibitively expensive to evaluate; in the aforementioned example, we cannot administer arbitrary doses of potentially dangerous candidate molecules into pa- tients to test their therapeutic efficacy. Similarly, experimen- tally evaluating designs in materials science discovery often necessitates many months of intensive laboratory work, and building candidate robots from scratch can be intractable. Instead, is more common to have access to a static dataset of previously evaluated designs D {(z,, yt y:) where yi r(x:)) Such settings where D is readily available but r(x) is hidden are referred to offline optimization.
secondary, often overlooked metric in offline MBO is can- didate diversity (Jain et al., 2022; Kim et al., 2023; Maus et al., 2023): it is often ideal to include diverse array of designs in the final samples proposed by an optimization procedure (Fig. 1). Different designs may achieve promis- ing oracle rewards in different ways, and many real-world optimization tasks seek to capture as many of these 'modes of goodness' as possible (Mullis et al., 2019; Jain et al., 2022). Furthermore, there may be secondary optimization objective(s) (e.g.. manufacturing cost or drug toxicity) that are better explored and evaluated in a diverse sample set. In these settings, it may be more desirable to sample slightly suboptimal designs in addition to the most optimal design to achieve a greater diversity of proposed candidates.
To overcome the limitation that r(z) cannot be queried during the optimization process, one approach is to learn a forward surrogate approximation r0(2)) of the true reward function r rar) Here, rg is parameterized by 0r given by
0" = arg min (o 1lu - r
(2)
In practice, such a surrogate model might be a neural net- work, a physies simulator, or other domain-specifis model. In our work, we do not require any particular surrogate model architecture training paradigm, and consider neural network implementations of re for generalizability across different domains. Rather than solving (1), we can now instead consider the related optimization problem
To this end, we introduce Diversity in Adversarial Model- based Optimization (DynAMO) as a novel approach ex- plicitly control the trade-off between the reward-optimality and diversity of proposed batch of designs in offline MBO, To motivate our contributions, we show how naïve opti- mization algorithms provably suffer from poor candidate diversity. To overcome this limitation, we propose a mod- ified optimization objective in the offline setting that en- courages discovery of designs that encapsulate the diversity of samples in the offline dataset-an approach inspired by recent advancements in imitation learning and offline rein- forcement learning (Ho & Ermon, 2016; Kostrikov et al.,
n = arg max Ex-aq(s)(rr(()))
(3)
*ENl
with the hope that optimizing against ra()) will learn a gen- erative policy that also proposes optimal designs according to r(z). too. Such an approach is often referred to as offline model-based optimization (MBO) (Trabucco et al., 2021). Traditionally, an important limitation of offline MBO is the distribution shift between the forward surrogate re and the oracle reward r: that is, re may incorrectly overestimate the reward associated with proposed designs that are out-of- distribution compared to D, which can often be exploited by traditional optimization algorithms (Trabucco et al., 2021; Yu et al., 2021: Fu & Levine, 2021; Yao et al., 2024).
2020; Ke et al., 2021; Ma et al., 2022; Rafailov et al., 2023; Deka et al., 2023; Huang et al., 2024b). We then derive DynAMO as a provably optimal solution to our modified optimization objective. Finally, we empirically demonstrate how DynAMO can be used with wide variety of different offline optimization methods to propose promising design candidates comparable to the -fate-of-the-art while also achieving significantly better candidate diversity.
Optimization Algorithms. To solve (3) and similar prob- lem formulations, number of optimization algorithms have been reported in prior work. One of the most popular ap- proaches is first-order methods such as gradient ascent, adap- tive moment estimation (Adam) (Kingma & Ba, 2015), and derivative work (Duci et al., 2011; Loshchilov & Hutter,
2. Background and Preliminaries
Offfine Model-Based Optimization. In generative design problems, we seek to learn generative policy x² over space of policies Il such that the distribution 9" (x)
2

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
2019; Zhu al., 1997; Defazio et al., 2024). Broadly, these optimizers leverage the gradient Vzro of the forward sur- rogate to iteratively update a candidate design. However, such techniques have been shown to struggle in optimizing against highly non-convex functions typical of real-world offline optimization problems (Trabucco et al., 2021; 2022).
match the diversity of the non-zero support of p(æ). Distri- bution matching has been used in prior work to learn robotic control policies (Ho & Ermon, 2016; Wang et al., 2020; Kostrikov et al., 2020; Ke et al., 2021; Ma et al., 2022) and align language models (Rafailov et al., 2023; Huang et al., 2024b; Chakraborty et al., 2024); here, we demonstrate how distribution matching can also be leveraged in offline generative design (a non-RL application) by matching the distribution of designs learned by a generative policy with the distribution of designs from the offline dataset.
Evolutionary algorithmss, such as covariance matrix adap- tation evolution strategy (CMA-ES) (Hansen, 2016; 2006) and cooperative synapse neuroevolution (CoSyNE) (Gomez et al., 2008), are an alternative approach to optimization. In- spired by biological evolution, such methods iteratively im- prove a population of candidate solutions using mechanisms like selection and mutation, and do not require gradient information from the forward model.
Generative Adversarial Networks. Generative adversarial networks (GANs) are a method popularized by Goodfellow et al. (2014); Arjovsky et al. (2017); and others to train a generative model. Such approaches train a generative policy using adversarial supervision provided by source critic c(z) X + R. The source critic and generator trained in zero-sum 'game' as the discriminator learns to distinguish between generated and real designs, and the generator simultaneously learns to generate designs that are similar to real examples according to the source critic. One particular GAN architecture introduced by Arjovsky et al. (2017) is the Wasserstein GAN, which learns a source critic
Separately, Bayesian optimization (BO) (Kushner, 1964) is another model-based optimization technique historically used to optimize reward functions that are non-convex, noisy, and/or lack a closed-form expression. Briefly, BO iteratively alternatives between (1) fitting a probabilistic sur- rogate model (c.g.. a Gaussian process) to the acquired data and their scores according to re: and (2) acquiring new can- didate designs according to an acquisition function, such as the expected improvement (EI) or upper confidence bound (UCB) (Ament et al., 2023; Wilson et al., 2018; Zhou et al., 2024a). While BO has traditionally been leveraged for opti- mization problems using expensive-to-evaluee functions, recent work has shown that BO is also a powerful method for offline optimization tasks, too (Maus et al., 2022:
e" (z) = arg max [Ext)p(a)d((') - [x= (4) [ell..11
where ||e||e is the Lipschitz constant of the source critic, p(x) is distribution over real designs (i.e., from an offline dataset D), and g(x) is a distribution over generated designs. Intuitively, we can think of c" (xx) as assigning a real-number score of in-diistriuttionnnss' to an input design x: large (resp., small) values of c* (x) mean that the source critic predicts the input design is in (resp., out of) distribution compared to the reference distribution p(z) over real de- signs. Yao et al. (2024) previously showed how source critics as in (4) can be leveraged in offline generative design tasks to prevent out-o--distr evaluation of the forward surrogate model ro(z); we leverage similar approach in our work.
black-box
Yao et al., 2024; Eriksson et al., 2019; Hvarfner et al., 2024;
Eriksson & Jankowiak, 2021; Astudillo & Frazier, 2019). Prior work from Maus et al. (2023); Hernández-Garehh et al. (2024); and others have investigated how to incorporate di- versity in existing BO frameworks; however, such methods either (1) gate whether to sample candidate designs based on a diversity- bbaed thresholding schema; or (2) have specif- ically been proposed for the BO optimization framework. In contrast, our method explicitly includes diversity via distri- bution matching as an optimization objective, and is readily compatible with standard optimization algorithms.
3. Distribution Matching for Generative Offline Optimization
Distribution Matching. Distribution matching is a tech- nique leveraged in recent work on imitation learning and offline reinforcement learning (RI.) (Kostrikov et al., 2020; Ke et al., 2021). The approach considers an experimental setup where RL agents cannot interact with the environment and instead must learn from static, offline expert demonstra- tions sampled from an unknown state-action-rewart distri- bution. The Kullback-Leibler (KL)- divergence (Matthews et al., 2016) is commonly used to train an agent to minimize the discrepancy between state-action visitations made by the RL agent and the offline expert. Given a sufficiently large and diverse dataset of expert demonstrations, we can also think of the KL divergence as encouraging the agent to
3.1. Motivating Limitation of Naïve MBO
Prior work from Mullis et al. (2019); Jain et al. (2022); Kim et al. (2023) have shown that an important challenge in offline optimization as in (3) is that of reward hacking: learned generative policies can exploit a small region of the design space, resulting in a low diversity of proposed designs. For example, consider the following lemma:
Lemma 3.1. (Diversity Collapse in Reward Optimization) Suppose that there exists a finite set of globally optimal designs 2; such that x; := arg max r(x) and r" == r(a)) is the optimal reward given a finite, non-uniform
3

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
reward function r(z). Given any distribution q", we can decompose it into the form of (x) = E, wz5(x= 2z) + En w/ 1( =x))= + ((x), where w; 0 for all j. and ((x) 2 0 and ((æ3) = for all j. Then, ((x) satisfies [ q(x) 0.
The KL-divergence strength hyperparameter 3. Sepa- rately, the hyperparameter | 2 0 controls the relative im- portance of the distribution matching objective. As 8 + oo, it becomes increasingly important for the generator to learn a distribution of designs that match pip (2); setting 3 = 0 reduces (T) to the original MBO objective in (3).
The proof for Lemma 3.1 is included in Appendix A. Note that this result holds for both the oracle function r(z) and the forward surrogate (a)). Intuitively, this lemma states that an optimal policy that maximizes (1) (resp-, (3) in the offline setting) can only have measurable nonzero support at the global optimizers in the design space X' However, many real-world reward functions do not have a large numher of globally optimal designs (Trabucco et al., 2021), leading to a low diversity of generated designs seen in practice (Kim et al., 2023). Furthermore, there is no guaranice that the set of optimal si cover a large region of the design space; in practice, we might be interested in trading optimality of subset of designs to achieve a greater diversity.
3.3. A Adversarial Source Critic as a Constraint
Separately, to address the problem of forward surrogate model overestimation of candidate design fitness according to ra (zr), we constrain the optimization problem to ensure that expected source critic scores over q" (x) and p (b(()) differ by more than constant Wo E RR,, similar the approach to offline MBO used by Yao et al. (2024). That is,
max ((T) = = [ro()]][ - 2 (g (Pb) TEll
(7)
s.t.
lp (x)( Egy [c" (x)) s W.
where the source critic c² : :X' - R is a neural network as in (4) that maximizes Ep [x]]] EEy= [c* (=)] subject the constraint [|c (2))12 S 1, where 11-11z is the Lipschitz norm. Intuitively, this constraint prevents the evaluation of the forward surrogate model rx())) on wildly disttriiist inputs encountered in the offline setting.
3.2. An Alternative MBO Problem Formulation
To reward generative policies in proposing diverse designs, we modify the original MBO objective in (3) according to
3 DKL (4 lIbb
J(T) == Exr
(5)
where (-1(- is the Kullback-Leible divergence (KL- divergence) and 8 E RR, are hyperparameters. In sub- sequent steps, we abbreviate the expectation value over probability distributions (x) [+] as Eg [ for brevity.
We are now interested in finding a generative policy x² that solves this optimization problem in (7); in our work below, we demonstrate how this approach can yield a policy that generates high-scoring candidate designs that also better capture the diversity of possible designs in X.
The temperature hyperparameter т. Equation (5) im- plicitly introduces hyperparameter E R4, to control the trade-off' between diversity and optimality. Note that the KL- divergence in (5) is computed with respect to a distribution PB (x) defined as the r-weighted probability distribution: Definition 3.2 (eighett Probability Distribution). Sup- pose that we are given a reward function r(æ)= X + IR over a space of possible designs X' and access to an static, offline dataset D of real designs. We define the T-weighted probability distribution over X (for T 2 0) as
3.4. Constrained Optimization via Lagrangian Duality
Our problem in (7) is ostensibly challenging to solve: both the objective ((z) and the constraint imposed by the source critic can be arbitrarily non-convex, making traditional con- strained optimization techniques intractable in solving the optimization problem out-of-the-box In this section, we de- rive an explicit solution to (7) to make the problem tractably solvable using any standard optimization algorithm.
p" (x) exp(Tr(x)) Zr
Recall from Lagrangian duality that solving (7) is equivalent to the min-max problem
(6)
where the partition function Z" := fx dx exp(rr(x)) is a normalizing constant. We use the dataset of prior obser- vations D = {a r( to empirically approximate p (x), and refer to this approximation as (T( = (x). For 7 >> near-optimal designs that are associated with high reward scores are weighted more heavily in pb; con- versely, 7 = 0 weights all designs equally to achieve the greatest diversity in designs. The penalized objective in (5) thereby encourages the learned policy to capture the diver- sity of designs in the w-weighted distribution Pb(x). We show empirical 7-weighted distributions in Appendix D.7.
min max L(T;; X)
(8)
well XER_
where the Lagrangian [(TT;)) II R4 R is given by
L(T; A) = - -((r) + +3A [Epp (x)] - [c (x)) - Wo]
(9)
introducing À E R,, such that B3 E R_ is the Lagrange multiplier associated with the constraint in (7). From weak duality, the Lagrange dual problem provides us with a tight
4

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Basce Optimization
lower bound on the primal problem in (7):
duality as in (10), it follows that maximizing (e(A) bounds the maxima over g()) from below. In subsequent steps, we therefore optimize over this explicit function ge()).
max min [(T;; X)
max (A) min max ((T;;) A)
AER
Fell
AER
Tell XER
(10)
The utility of Lemma 3,4 is in solving for the optimal À that maximizes the dual function lower bound in (13). Prior work has explored approximating via a grid search (Yao et al., 2024) or using iterative implicit solvers; these methods cannot provide any formal guarantee in arriving at reasonable solution for X. In contrast, maximizing against ge (A) is easy because the function is guaranteed to be concave for any 3, r, Wo and source critic c² (z). We can therefore derive an exact solution for À using any convex optimization problem solver. We now have a method to write an explicit expression for the Lagrangian [(T; A) by exactly specifying the optimal X, and then leverage any out-of-the- policy optimization method to solve (7) via solving the easier, ostensibly unconstrained problem in (8).
where g()) = min [(;; A) is the Lagrange dual func-
tion. In general, computing g(A) is challenging for an arbi- trary offline optimization problem; in prior work, Trabucco et al. (2021) bypassed this dual problem entirely by treating À as hyperparameter tuned by hand (albeit for different constraint); and Yao et al. (2024) approximated the dual function under certain assumptions about the input space by performing a grid search over possible values. In our approach, we look to rewrite the problem into an equivalent representation that admits a closed-form, computationally tractable expression for g()).
Lemma 3.3 Senntroy-Divergeeeee Formulation). Define (TT) as in (5). An equivalent representation of J(T) is
[T) ~ (z) (l)
(11)
3.5. Overall Algorithm
where H(-) is the Shannon entropy. Maximizing (II) is equivalent to maximizing (5) in the sense that both objectives admit the same optimal policy.
To summarize, our work aims to solve two separate but related problems in offline MBO in (3): traditional model- based optimization approaches can yield candidate designs that are [1] of low diversity; and [2] not optimal due to exploiting out-of-distribuiio errors of the forward surro- gate ry (z)). We introduce a -I.-divergence-bae based distribu- tion matching Sojective-with input hyperparameters 7 and 8 to solve the diversity problem; and build off prior work (Yao et al., 2024) to constrain the search space using source critic feedback to solve the diito-f-distibution evaluation problem. We then show that there exists a provable, explicit solution to our modified offline MBO problem (i.e., Lemma 3.4 and (8)). In contrast with prior work imposing specific constraints on the forward model (Trabucco et al., 2021; Yu et al., 2021) or design space (Yao al., 2024), or requiring the use of model-free optimization methods (Krishnamoor- thy et al., 2023; Mashkaria et al., 2023), our approach only modifies the MBO objective and is therefore both optimizer- and task- agnostic. We refer to our method as Diversity in Adversarial Model-based Optimization (DynAMO).
The proof of this result is in Appendix A. To build intuition about how (11) is equivalent to (5), we can consider the behavior of the objective in the limit of + +oo: the refer- ence distribution Pb approaches the sun-of---distribution formulation in Lemma 3.1. In this setting, the entropy and KL-divergence terms are equivalent, and the optimal policy nt admits a distribution qº' with nonzero support only at the globally optimal designs in Pb- Alternatively in the limit that f + and B + +00, pb approaches a uniform distri- bution and both (5) and (11) simplify to a state-mate mate-matthiin objective according to the KL-divergence loss term, without any explicit optimization against the surrogate model rx)).
The utility of Lemma 3.3 is to enable us to write an exact formulation for the Lagrangian dual function g(A):
Lemma 3.4 (Explicit Dual Function of (7)). Consider the primal problem
max J(x) = -²(=-)a)= - (1 (a REl
4. Experimental Evaluation
(12)
S.t. (pg[c((x)) Egg ie (x) < Wo
Datasets and Offfine Optimization Tasks. We evaluate Dy- nAMO on a set of six real-world offline MBO tasks spanning multiple scientific domains and both discrete and continuous search spaces. Five of the tasks are from Design- Bench, a publicly available set of offline optimization benchmarking tasks from Trabucco et al. (2022): (1) TFBind8 aims to max- imize the transcription factor binding efficiency of short DNA sequence (Barrera et al., 2016); (2) UTR the gene expression from a 5' UTR DNA sequence (Sample et al., 2019; Angermüeller et al., 2020); (3) ChEMBL the mean corpuscular hemoglobin concentration (MCHC) biological
The Lagrangian dual function g()) is bounded from below by the function ge(A) given by
(x)] - Wo) - -Epg to (1))1] (13) go(A): = : (xCEpg(oo
The proof of this result is included in Appendix A. Lemma 3.4 admits an explicit concave function (A) such that g(A) 2 ge (A) for all A E R4; because we are interested in maximizing the dual function leveraging Lagrangian
5

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Algorithm 1 (DynAMO). Diversity in Adversarial Model- based Optimization
previously observed designs and their corresponding objec- tive values. We first use DD train task-specific forward surrogate model re with parameters 0* according to (2). We parameterize each forward surrogate model (=(2) as a fully connected neural network with two hidden layers of size 2048 and LeakyReLU activations, trained using an Adam optimizer with a learning rate of rj = 0.0003 for 100 epochs.
Inputs:
rg X R pre-trained forward surrogate model
c² X + R initialized source critic model
D = [ j. r( (1))))1 reference dataset
3 2 O- - KI. regularization strength
T 2 0 - temperature
Importantly, optimization problems over discrete search spaces generally NP-hard and often involve heuristic- based solutions (Papalexopoulos et al., 2022; Xiong, 2022). Instead, we use the standard approach of learning a vari- ational autoencoder (VAE) (Kingma & Welling, 2014) to encode and decode discrete designs to and from a contin- uous latent space, and optimize over the continuous VAE latent space instead-see Appendix B for additional details.
62 1 - batch size
ah: : X x R + x - optimizer algorithm
Meriticc > 0 source critic learning rate
TIx 0 - À dual step size
k 22 1 - oracle evaluation budget
Initialize sampled candidates D yen Ø C X' IR while a² has not converged do // Solve for the globally optimal À using (13)
DynAMO also involves training and implementing source critic model c² (a:) as in (4); we implement c* as a fully con- nected neural network with two hidden layers each with size 512. We implement the constraint on the model's Lipschitz norm by clamping the weights of the model such that the le- of the parameters is no greater than 0.01 after each optimization step, consistent with Arjovsky et al. (2017). We train the critic using gradient descent with a learning rate of 0.01 according to (4). Separately to solve for the globally optimal À using Lemma 3.4, we perform gra- dient ascent on À until the algorithm converges. Finally, we fix the KL-divergence weighting 3 = 1.0, temperature hyperparameter = 1.0, and constraint bound W,, 0 for all experiments avoid overfitting DynAMO to any particular task or optimizer. All experiments were run for 10 random seeds on single internal cluster with 8 NVIDIA RTX A6000 GPUs Of note, all DynAMO experiments were run using only a single GPU.
X + do (Ao = 1.0 in our experiments)
while À has not converged do
À - X + 1/A (A)
end while
I/ Given previously sampled candidates Dgen input, I/ sample new candidates using the optimizer {a)e0)] + a² (Dgen)
// Re-train the source critic parameters 0g.
while SW has not converged do
8W + Ve
- E (ato 3. [c)
0- f min (max(Oe + eeitic oW, 0.001)
end while
// Evaluate and cache the candidates according to (9)
Dgen + Dgen U {<>?ew - -(x {))
end while return top-k candidates from Dgen according to their penalized MBO objective values
Baseline Methods. Our proposed work, DynAMO, specifi- cally looks to modify an offline MBO optimization problem as in (3) where we assume access to forward surrogate model re(x)) to rank proposed design candidates and offer potential information about the design space. We com- pare DynAMO against other objective modifying MBO approaches: (1) Conservative Objective Models (COMs; Trabucco et al. (2021)) penalizes the objective at a 'look- ahead' gradient-ascent iterate to prevent falsely promising gradient ascent steps: (2) Robust Model Adaptation (RoMA; Yu et al. (2021)) modifies the objective re(x) to enforce local smoothness prior; (3) Retrieval-enhanced Offline Model- -osel- Optimization (ROMO; Chen et al. (2023c)) retrieves relevant samples from the offline dataset for more trustworthy gradient updates; and (4) Generative Adversar- ial Model-Based Optimization (GAMBO; Yao et al. (2024)) introduces framework for initially leveraging source critic feedback to regularize an MBO objective. We evaluate each of these MBO objective transformation methods alongside
response of molecule using an offline dataset collected from the ChEMBL assay CHEMBL3385882 (Gaulton et al., 2011); (4) Superconductor the critical temperature of superconductor material specified by its chemical formula design (Hamidieh, 2018); and (5) D'Kitty the morphologi- cal structure of quadrupedal robot (Ahn et al., 2020). Tasks (1)- (3) (i.e., TFBind8, UTR, and ChEMBL.) are discrete optimization tasks, where tasks (4) and (5) (i.e., Supercon- ductor and D' D'Kitty) are continuous optimization tasks. We also evaluate our method on the discrete (6) Molecule task described in Brown et al. (2019); Chen et al. (2021); Flam- Shepherd et al. (2022); Yao et al. (2024), where the goal is to design a maximally hydrophobic molecule. Additional implementation details are detailed in Appendix B.
Experiment Implementation. All our optimization tasks include an offline, static dataset D {( (2x)))] of
6

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Basch Optimization
Table Quality and Diversity of Designs Under MBO Objective Transforms. We evaluate DynAMO against other MBO objective- modifying methods using six different backbone optimizers. Each cell consists of @Best@ 128 (Best)/Pairwise Diversity (PD) Rank and Optimality Gap scores separated by a forward slash. Bolded (resp., Underlined) entries indicate the best (resp., second best) performing algorithm for a given optimizer (i.e., within each column). See Supplementary Table A1 for detailed results broken down by MBO task.
Best/PD
Rank
Optimality Gap t
Grad.
Adam
CMA-ES
CoSyNE
BO-qEI
BO-qUCB
Grad.
Adam
CMA-ES
CoSyNE
BO-qEI
BO-qUCB
Baseline
5.0/5.5
4.5/6.0
3.7/3.8
5.3/4.5
5.8/5.2
3.7/3.0
6.8/-53.2
0.5/-52.4
14,4/9.5
-0.6/-52.1
18.7/47.1
19.444..5
COMs
7.3/6.5
6.0/5.3
5.7/5.2
5.7/5.7
4.3/4.0
4.5/4.5
-3.04-3..5
-3.0/ 52.4
7.6/
1.1/-51.6
19.2/51.4
19.0/45.5
COMs'
2.5/2.8
3.2/3.0
7.3/7.8
5.7/3.3
6.0/5.7
5.2/5.7
12.34-699
8.1/-12.3
7.1/38.22
3.5/-10.4
17,9/40.3
18.6/51.3
ReMA
6.7/5.7
4.5/6.3
3.7/3.5
5.3/4.5
2.7/3.3
3,8/3.0
-1,2/-53.3
0.5/-524
14,4/9.5
-0.6/-52.2
21.0/48.4
19.2/43.6
RoMA'
3.8/5.8
2.8/5.2
5.04.8
2.8/5.0
5.2/6.3
4,7/6.5
9.2/-46.8
14.5
14.178.0
6.21-52.0
18.
18.5/39.9
ROMO
4.2/2.8
4.8/2.8
4,2/4,3
4.2/5.2
5.0/6.0
4.7/6.2
10.9/-12.7
6.4/-20.5
15.7/-3.1
3,1/-50.8
19.2/34,9
19.9/33.2
GAMBO
3,2/5.3
5.3/588
2,2/4,3
3,7/6.3
2.2/4.0
4.7/5.0
10.5//52.1
8.6V-5199
16.7/16.8
5.0153.6
20.8/30.0
20.2/30.3
DynAMO
2.8/1.2
2.8/1.2
3.3/1.8
2.3/1.2
3.0/1.3
3.5/1.8
14.2/27.8
14.5/35.7
17.5/55.2
12.3/-20.7
20.7/74.2
20.5/59.4
DynAMO and naïve, unmodified Baseline MBO using rep- resentative first-order methods (1) Grad. (Gradient Ascent) and (2) Adam (Adaptive Moment Estimation (Kingma & Ba, 2015)); evaluationary algorithms (3) CMA-ES (Co- variance Matrix Adaptation Evolution Strategy (Hansen, 2016)) and (4) CoSyNE (Cooperative Synapse Neuroevo- lution (Gomez et al., 2008)); and Bayesian optimization with (5) Expected Improvement (BO-qEI) and (6) Upper Confidence Bound (BO-qUCB) acquisition functions.
sure that diversity does not come at the expense of finding optimal design candidates, report the @ oracle score obtained by evaluating k 128 candidate designs { ) proposed in an experiment. Consistent with prior work (Trabucco et al., 2021; Yao et al., 2024), we define
Best@ (eet@@k((=)e)) F (( )==) = max r(a))
(15)
1sisk
Crucially, the Best@k A: metric is computed with respect to the oracle function r(z)) that was hidden during optimization; we only use r(x:) in (15) to report the true reward associated with each candidate design.
Notably, the baseline methods COMs and RoMA impose specific constraints on the training process for the forward surrogate model rz)),, and/or also assume that the forward model can be updated during the sampling process (Yu et al., 2021; Trabucco et al., 2021). These constraints are not gen- erally satisfied for any arbitrary offline MBO problem; for example, re may be non-differentiable black-box simu- lator with fixed parameters. In contrast, both our method (DynAMO) and baseline methods GAMBO and ROMO are compatible with this more general experimental setting; to ensure a fair experimental comparison, we evaluate both RoMA and COMs using a baseline forward surrogate (i.e., RoMA-, COMs)) and using a specialized forward surro- gate model trained and updated according to the methods described by the respective authors (i.e., RoMA* COMs*).
Finally, we rank each method for a given optimizer and task and report the method's Rank averaged over the six tasks according to the Best 128 (15) and PD (14) metrics. We also report the Optimality Gap (Opt. Gap) (averaged over the six tasks), defined as as difference between the score achieved by MBO optimization method and the score in the offline dataset, for both the Best @ 128 and PD metrics.
5. Results
Main Results. DynAMO consistently proposes the most diverse set of designs and achieves an Optimality Gap as high 74.2 (DynAMO-BO-qUCCH) and an average Rank as low 1.2 compared to baseline methods (Table 1). We find that DynAMO offers the largest improvements in diversity for first-order methods, although also improves upon the evolutionary algorithms and Bayesian optimization methods evaluated. This makes sense, as both Grad. and Adam are only local optimizers that often end up exploring a much smaller region of the design space (without using DynAMO) compared to gradient-free methods. For exam- ple, DynaMO-Grad. (resp., DynAMO-CMA-ESS; resp., DynAMO-BO-qEI achieves Pairwise Diversity Optimal- ity Gap of 35.7 (resp., 55.2; resp., 74.2); in contrast, no other baseline method achieves a diversity score greater than -6.9 (resp., 16.8; resp., $1.4) within the same optimizer class.
Evaluating the Diversity of Candidate Designs. To em- pirically evaluate the diversity of a final set of h- = 128 candidate designs {xf" y proposed by an offline MBO experiment, we report the Pairwise Diversity (PD) of a batch of k candidate designs, defined by Jain et al. (2022);
Kim et al. (2023); and Maus et al. (2023)
PD( 1im) = E.,r [Eafes [d(af =f]]]]
(14)
where d(.,.) is the normalized Levenshtein edit distance (Haldar& & Mukhopadhyay, 2011) (resp., Euclidean distance)
for discrete (resp., continuous) tasks. We detail alternative definitions of diversity in Appendix D.2.
Evaluating the Quality of Candidate Designs. To en-
7

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
These results do not come at the cost of the quality of de- signs; for example, for all 3 optimizers where DynAMO scores average Rank of 1.2 (i.e., Grad., Adam, and CoSyNE backbone optimizers), DynAMO is also within the top methods in proposing high-quality designs according to both Rank and Optimality Gap. In fact, DynAMO pro- poses the best designs for out of the backbone optimizers according to the @est@ 128 Optimality Gap. These results suggest that DynAMO can be used to improve both the quality and diversity of designs in a variety of experimental settings for both discrete and continuous search spaces.
formulated as a multi-fidelity optimization problem.
Reinforcement learning. Prior work has explored how to formulate offline generative design tasks as reinforce- ment learning (RL) problems. Trabucco et al. (2022) used REINFORCE-style methods similar to Williams (1992) to learn a myopic sampling policy, although do not use RL for offline generative design. Angermueller et al. (2020); Korshunova et al. (2022); and Jang et al. (2022) leverage RL for offline optimization in the active learning setting described above, which is outside the scope of our work.
Ablation Studies. DynAMO consists of two important but separate algorithmic components: (I) a -i-devergence- based distribution matching objective; and (2) a constraint dependent on an adversarial source critic. We show both components are important for DynAMO to generate both di- verse and high-quality designs (Appendix E.2). DynAMO also takes as input two important hyperparameters-22 and a-as introduced in (5). We empirically ablate the values of these hyperparameters in Appendix E.3. Additional results and discussion are included in Appendix E.
7. Discussion and Conclusion
We introduce DynAMO, a novel task- and optimizer- agnos- tic approach to MBO that improves the diversity of proposed designs in offline optimization tasks. By framing diversity as a distribution matioing problem, we show how DynAMO can enable generative policies to sample both high-qualing and diverse sets of designs. Our experiments reveal that DynAMO significantly improves the diversity of proposed designs while also discovering high-quality candidates.
Limitations and Future Work. There are also important limitations of our method. Firstly, we note that while Dy- nAMO can significantly improve the diversity of proposed designs in offline MBO while preserving @est@ 128 per- formance, our method is not as competitive with existing baselines according to the median score obtained by the 128 designs (Supp. Fig. A2, Appendix B). The suboptimal performance of DynAMO according to this Median @ 128 metric is unsurprising given that our primary motivation of DynAMO is to obtain a diverse sample of designs while simultaneously ensuring that a nonzero subset of them are (near-) optimal. Furthermore, we empirically observe that no method is state-of-the-an on both Best@ 128 and Me- dian @ 128 metries. While it would be ideal for DynAMO (or any method) to be state-of-the the-art for all quality and diversity metries, we argue that obtaining a good Best@ 128 score is more important than good Median @ 128 score, as the the principle real-world goal of offline MBO is to find design that maximizes the oracle function.
6. Related Work
Model-free offline optimization. In our work, we specifi- cally look at model-based optimization methods that explic- itly optimize against a forward surrogate model rx( that acts as a proxy for the hidden oracle function r(z). How- ever, related work have also proposed offline optimization methods that do not require access to a model re(2r) and instead impose constraints on the backbone optimization method-we refer to such work as model-free offline op- timization. For example, Mashkaria et al. (2023) frame generative design tasks as a 'next-sample' prediction prob- lem and learn a transformer to roll out sample predictions; and Krishnamoorthy et al. (2023); Yun et al. (2024) learn a diffusion model to sample candidate designs conditioned on reward values. Because DynAMO operates MBO for- ward surrogate models r=()), we cannot leverage DynAMO with these model-free methods. However, we compare them against DynAMO in Appendix D.5.
Secondly, we also limit our study of DynAMO to offline MBO tasks that are well-described and studied in prior work. In principle, real-world optimization problems may be complicated by noisy and/or sparse objective functions, ultra-high high dimensional search spaces, small offline datasets, and other practical limitations. We leave more rigorous interrogation of how such offline MBO methods perform in such settings for future work.
Active learning in optimization. In our work, we specifi- cally consider the experimental setup of one-shot, batched oracle evaluation: that is, the final candidate designs that are scored by the oracle function at the end of optimization are not used to subsequently update the prior over the design space to better inform subsequent optimization steps. In con- trast, a separate body of recent work has investigated gener- ative design in the setting of active learning where there can be multiple rounds of offline optimization to inform subse- quent online acquisitions (Hernández-Garce et al., 2024; Li et al., 2022b;a; Wu et al., 2023; Palizhati et al., 2022). For example, Li et al. (2024) show how active learning can be
Finally, our work focuses on evaluating DynAMO and base- line methods in a one-shot, batched oracle evaluation set- ting-future work might explore how extend our method to the active learning setting. Separately, recent domain-
8

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
specific foundation models (Lin al., 2023; Ohana al., 2024; Nguyen al., 2024; Zeni al., 2025) may also give rise to more sophisticated and accurate forward surrogate models ræ that can be leveraged with DynAMO and other MBO methods in future work.
2020. URLhttpsss/prrceeddnng mlr. press// 01000/ ahn20a html.
Ament, S., Daulton, S., Eriksson, D., Balandat, M., and Bakshy, E. Unexpected improvements expected im- provement for Bayesian optimization. In Proc NeurIPS, 2023. URL https:: /tt.ttpss//pppeeee/e///// /ttp///ppppee.eeee//eep/eee/eeep net forum? id-1vyAG6j9PE.
Acknowledgments
We thank Shuo Li the University of Pennsylvania for their helpful comments and feedback on an earlier draft of this work, and the anonymous referees whose peer review helped significantly improve the quality of our manuscript. MSY is supported by NIH Award F30 MD020264. JCG is supported by NIH Award RO1 EB031722. OB supported by NSF Award CCF-1917852.
Amos, B. On amortizing convex conjugates for opti- mal transport. In Proc ICLR, 2023. URL https: // /openreview. hww.ne///ee.. forum?
d=TQ5wUwS_4ad ttt.:fffrmmmmd--ss
Angermüieller, C., Belanger. D., Gane, A., Mariet, Z., Dohan, D., Murphy, K., Colwell, I.,, and Sculley, D. Population-based black-box optimization for biological sequence design. In Proc ICML, volume 119, pp. 324-34. PMLR, 2020. URL https: https::/ press v1 9 press/v119/rrgermmnn wwaaannrrueeleeeeeeee
mllr.
Impact Statement
html.
Offline generative methods such as DynAMO have signifi- cant potential to help design more effective drugs; engineer new materials with desirable properties; and solve other sei- entific tasks. However, like any real-world algorithm, these methods can also be misused create potentially harmful dangerous designs. Careful oversight by domain experts and researchers is essential to ensure our contributions are used for social good.
Angermueller, C., Dohan, D., Belanger, D., Deshpande, R., Murphy, K., and Colwell, I.. Model-based reinforcement learning for biological sequence design. In Proc ICLR, 2020. URL https:/ woweneeviewwwveeeeee net forum? [d=Hk1xbgBBK
Arjovsky, M., Chintala, S., and Bottou, L. Wasser- stein generative adversarial networks. In Proc ICMI,, volume 70, pp. 214-23. PMIR, 2017. URL https://proceediing morg.mmr.ssg www....rrs///1.0 arjovsky17a. html.
Code and Data Availability
All datasets used in our experiments are publicly avail- able; offline datasets associated with Design- Bench tasks made available by Trabucco et al. (2022). The of- fline dataset for the Molecule task is made available by Brown et al. (2019). Our custom code implemen- tation for our experiments is made publicly available at github. com/miimmii..... http//icchhhhheeeeeeee ssyao DDyAAM..
Astudillo, R. and Frazier, P. Bayesian optimization of com- posite functions. In Proc ICML, volume 97, pp. 354-63. PMLR, 2019. URL https: pptp/eeecoceeenssseoeeee//e .mlr. press/ htt.99//ssuuillllo 9a. html.
Barrera, L. A., Vedenko, A., Kurland, J. V., Rogers, J. M., Gisselbrecht, S., Rossin, E. J., Woodard, J., Mariani, L., Kock, K. H., Inukai, S., Siggers, T., Shokri, L., Gordân, R., Sahni, N., Cotsapas, C., Hao, T., Yi, S., Kellis, M., Daly, M. J., Vidal, M., Hill, D. E., and Bulyk, M. L. Survey of variation in human transcription factors reveals prevalent DNA binding changes. Science, 352(6)))): 1450-54, 2016. doi: 10. 1116/science.aad225.
References
Agarwal, S., Durugkar, I., Stone, P., and Zhang, A. f-Policy gradients: general framework for goal conditioned RI. using f-divergences. In Proc NeurIPS, pp. 12100-23, 2024. doi:
Benita, Y., Oosting, R. S., Lok, M. C., Wise, M. J., and Humphery-Smith, Regionalized GC content of template DNA as a predictor of PCR success. Nucleic Acids Res, 31 31(16):e99, 2003, doi: 10. 00.10///aa//
Agrawal, R. and Horel, T. Optimal bounds between divergences and integral probability metrics. J Mach Learn Res, 22(128):1-59, 2021. doi: 10.48550/arXiv. 2006.05973.
Blank, J. and Deb, K. pymoo: Multi-objective optimization in Python. IEEE Access, 8:89497-509, 2020. doi: 10. 1 1109/ACCESS.2220.29000
Ahn, M., Zhu, H., Hartikainen, K., Ponte, H., Gupta, A.. Levine, S., and Kumar, V. Ro- bel: Robotics benchmarks for learning with low-cost robots. In Proc CoRL, volume 100 of Proceedings of Machine Learning Research, pp. 1300-13. PMLR,
Borwein, J. and Lewis, A. Convex analysis and non- linear optimization: Theory and examples. Springer, 2006. ISBN 978-0-387-295700
doi: 10.1007/
978-0-387-31155-9

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Boyd, S. and Vandenberghe, L. Convex optimization. Cambridge university press, 2004. doi: 10.1017/ CBO978051 180444.
Chen, Z., Min, M. R., Parthasarathy, S., and Ning, X. A deep generative model for molecule optimization via fragment modification. Nat Mach Intell, 3:1040-9, 2021. doi: 10. 00588/42225-021-0041101
Branin, F. H. Widely convergent method for finding multiple solutions of simultaneous nonlinear equations. IBM Jour- nal of Research and Development, 16(5):00422,, 1972. doi: 10. 10.11477 147//d. 165.0504.
Cortes, C., Mansour, Y., and Mohri, M. Learning bounds for importance weighting. In Proc NeurIPS, volume 23, pp. 442-50, 2010. doi: 10.5555/2997 09..9997999999999599
Brown, N., Fiscato, M., Segler, M. H. S., and Vaucher, A. C. Guacamol: Benchmarking models for de novo molecular design. J Chem Inf Model, 1993):1066-10S 2019, doi: 1a.1021/acs.jcim.8o000 8b00899.
Cuturi, M. Sinkhorn distances: Lightspeed computation of optimal transportation distances. In Proc NeurlPS, pp. 2292-300, 2013. doi: 10.48550aarXiv. 1306.0895.
Dao, M. C., Nguyen, P. L., Truong. T. N., and Hoang,
Bubeck, S. Convex optimization: Algorithms and complex- ity. Found Trends Mach Learn, 8(3-4):231-357, 2015. doi: 10.1561/22000000000
T. N. Boosting offline optimizers with surrogate sensitiv- ity. In Proc ICML, volume 235, pp. 10072-90. PMLR, 2024. URL https: http///ppooceeec/eeeee/ee/e/ssse mlr. p..resss//
v235 /ww//daaaa.. 4b html.
Chakraborty, S., Qiu, J., Yuan, H., Koppel, A., Manocha, D., Huang, F., Bedi, A., and Wang, M. MaxMin- Alignment with diverse human pref- erences. In Proc ICML, volume 235 of Proceedings of Machine Learning Research, pp. 61 16-35. PMLR, 2024. UtLhttps: http::/::///cooeddnngg/ www.m.rrrsss... v235/ 2235/chakkabortyy2e html.
Deh, K., Sunda, J., Rao, U. B. N., and Chaudhuri, S. Ref- erence point based multi-objectivv optimization using evolutionary algorithms. Int J Comp Intell Res. 2(3): 273-86, 2006.
Defazio, A., Yang, X. A., Khaled, A., Mishchenko, K., Mehta, H., and Cutkosky, A. The road less scheduled. In Proc NeurIPS, 2024. URL https: bettss:/openrevieee net / forum?.d==0eNkkk..
Chemingui, Y., Deshwal, A., Hoang, T. N., and Doppa, R. Offline model-based optimization via policy-guided gradient search. In Proc AAAI, pp. 11230-9, 2024. doi: 10. 00.1609/amai.v338 381100.000..
Deka, A., Liu, C., and Sycara, K. P. Are Actor resid- ual critic for adversarial imitation learning. Proc CoRI., volume 205 of Proc Mach Learn Res, pp. 1446-56, 2023. https:::: http///pceeenngss///// mlr ..ress/ html.
Chen, C., Zhang, Y., Fu, J., Liu, X., and Coates, M. Bidi- rectional learning for offline infinite-width model-based optimization. In Proc NeurlPS, pp. 29454-67, 2022. doi: 00.5555/3600227033024466
Du, Y., Jamasb, A. R., Guo, J., Fu, T., Harris, C., Wang, Y., Duan, C., Liò, P., Schwaller, P., and Blun- dell, T. L. Machine learning-aided generative molec- ular design. Nat Mach Intell, 6:589-604, 2024. doi: 10. 00080/42255-02--000000
Chen, C., Beckham, C., Liu, Z., Liu, X. L., and Pal, C. Parallel-mentoring for offline model-based optimization. In Proc NeurlPS, pp. 76619-36, 2023a. doi: 10.5555/ 3666122.366946..
Chen, C., Zhang, Y., Liu, X., and Coates, M. Bidirec- tional learning for offline model-based biological se- quence design. In Proc ICML, volume 202, pp. 5351-66. PMLR, 2023b. URL https: paattp:://prceeee /pwpceedinggs.... mlr.
Duci, J., Hazan, E., and Singer, Y. Adaptive subgradient methods for online learning and stochastic optimization.
J Mach Learn Res, 12(61):2121-59, 2011. URL http:
/ wtw.m...rgg/pppeeeeeeemm http hwwhh/////cchiiiccc 1la. html.
chwn23ao.... press/v2022 2ttps//v222//hee2300.. html.
Eriksson, D. and Jankowiak, M. High-dimensional Bayesian optimization with sparse axis-aligned subspaces. In Proc Conf Unc Artif Intell, volume 161, pp. 493-503. PMLR, 2021. URL https: pttttts://proceeddiii /ptp//eeeocccssssee/ee/pp/. mlr press/ v1 61 / keerissss21111 a. htmll.
Chen, C. C., Beckham, C., Liu, 7.,, Liu, X., and Pal, C. Robust guided diffusion for offline black-box optimiza- tion. Trans Mach Learn Res, 2024. URL https: / //openreview net www.:/nee/ofuuummeeeooe httprrmmmmd444cm255555
Chen, M., Zhao, H., Zhao, Y. Fan, H., Gao, H., Yu, Y., and Tian, Z. ROMO: Retrieval-enhanced offline model-based optimization. Proc Int Conf Dist Artif Intell, pp. 1-9, 2023c. doi: 10.1 145/362767636627685
Eriksson, D., Pearce, M., Gardner, J., Turner, R. D., and Poloczek, M. Scalable global optimization via local Bayesian optimization. Proc NeurIPS, volume 32, 2019.
10

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Flam-Shepherd, D., Zhu, K., and Aspuru-Guzik, A. Lan- guage models learn complex molecular distribu- tions. Nat Commun, 13(3293), 2022. doi: 10.1038/ 441467-022-30839.
Gonzalez, J., Dai, Z., Hennig, P. and Lawrence, N. Batch Bayesian optimization via local penal- ization. In Proc Int Conf AISTATS, volume 51 of Proc Mach Learn Res, Pp. 648-57. PMLR, 2016. ml malr. press/v51/
www.https///poocee gonzalezl6a.c html.
Fu, and Levine, Offline model-based optimization via normalized maximum likelihood estimation. In Proc ICLR, 2021. URL https:/ http://ppeeeeee.eeee.te forum? FeeellddFFiiiie F...FFFMMK0488B.
Gonzalez de Oliveira, R., Navet, N., and Henkel, A. Multi- objective optimization for safety-related available E/E architectures scoping highly automated driving vehicles. ACM Transactions on Design Automation of Electronic Systems, 28(41):1-37, 2023. doi:
Gardner, M. J., Hall, N., Fung, E., White, O., Berriman, M.. Hyman, R. W., Carlton, J. M.. Pain, A., Nelson, K. E., Bowman, S., Paulsen, I. T., James, K., Fisen, J. A., Rutherford, K., Salzberg, S. L., Craig, A., Kyes, S., Chan, M.-S., Nene, V., Shallom, S. J.. Suh, B., Peterson, J., Angiuoli, S., Pertea, M., Allen, J., Selengut, J., Haft, D., Mather, M. W., Veidya, A. B., Martin, D. M. A., Fair- lamb, A. H., Fraunholz, M. J., Roos, D. S., Ralph, S. A., McFadden, G. I., Cummings, L. M., Subramanian, G. M., Mungall, C., Venter, J. C., Carucci, D. J., Hoffman, S. L.., Newbold, C., Davis, R. W., Fraser, C. M., and Barrell, B. Genome sequence of the human malaria parasite Plas- modium falciparum. Nature, 419:498-511, 2002. doi: 10. 00.1338/natur001097
Goodfellow, I., Pouget-Abadie, J., Mirza, M. Xu, B.. Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial nets. In Proc NeurIPS, volume 27,
2014. doi: 11.
Grimmer, B., Shu, K., and Wang, L. Accelerated gradient descent via long steps. arXiv Preprint, 2023. doi: 10. 48550/arXiw.2309.0999
Haldar, R. and Mukhopadhyay, D. Levenshtein distance technique in dictionary lookup methods: An improved approach. arXiv Preprint, 2011. doi: 10.48550/arXiv. 1101.1232.
Gashmard. H., Shakeripour, H., and Alaci, M. Predicting superconducting transition temperature through advanced machine learning and innovative feature engineering. Sci Rep. 14(3965), 2024. doi: 10. 10.1038//41598. 191--0248---4 5444554400-.
Hamidieh, K. A data-driven statistical model for predicting the critical temperature a superconductor, Comp Mat Sci, 154:346-533 54, 2018. doi: 07.052.
Gaulton, A., Bellis, L. J., Bento, A. P., Chambers, J., Davies, M., Hersey, A., Light, Y., McGlinchey, S., Michalovich, ChEMBL: A large-scale bioactivity database for drug discovery. Nu- cleic Acids Research, 10DDI) 2011. doi: 10. 20.1099/ar//97777..
Hansen, The CMA evolution strategy: A comparing review, pp. 75-102. Springer Berlin Heidelberg, 2006. doi: 00.1007/3-540-32444.1
D., Al-Lazikani, B., and Overington,
Hansen, The CMA evolution strategy: A tutorial. arXiv Preprini, 2016. doi: 1005550/aXxi...644.07777
Gomez, F., Schmidhuber, J., and Mikkkulanenn Ac- celerated neural evolution through cooperatively co- evolved synapses. J Mach Learn Res, 9(31):937- 65, 2008. URL http: /tte://mmmrrce www.ii..og//ppprrpe/e gomez08a. html.
Hernández-Garcfa. A., Saxena, N., Jain, M.. Liu, C.- H., and Bengio, Y. Multi-fidelity active learning with GFlowNets. Trans Mach Learn Res, 2024. URL https: / openrevieew net. forum? d=dLaazw2zzz
Gómez-Bombarelli, R. Wei, N., Duvenaud, D., Heránndez-Lobator J. M., Sánchez-Lengelingg B., She- berla, D., Aguilera-lparraguire, J., Hirzel, T. D., Adams, R. P., and Aspuru-Guzik, A. Automatic chemical de- sign using a data-driven continuous representation of molecules. ACS Cent Sci, 4:268-76, 2018. doi: 10. /t1//csseetscci.b00052.
Ho, J. and Ermon, S. Generative adversarial imitation learning. In Proc NeurIPS, pp. 4572-80, 2016. doi: 10.555/3157388.315576699
Hoang, M., Fadhel, A., Deshwal, A., Doppa, J., and Hoang, T. N. Learning surrogates for offline black-box optimiza- tion via gradient matching. In Proc ICMI, pp. 18374-93, 2024.
Gong, C., He, Q., Bai, Y., Yang, Z., Chen, X., Hou, X., Zhang, X., Liu, Y., and Fan, G. The f-Divergence rein- forcement learning framework. arXiv Preprint, 2021. doi: A8550//rXiv.2109.1
Hoeffding, W. Probability inequalities for sums bounded random variables. Am Stat Assoc, 58(301):13-30 1963. doi: 10. 10.1000/1122 1559. 1963. 10500830.

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Huang, A., Zhan, W., Xie, T., Lee, J. D., Sun, W., Krish- namurthy, A., and Foster, D. J. Correcting the mythos of KL-regularization: Direct aligmment without overopti- mization via Chi-squared preference optimization. arXiv Preprint, 2024a. doi: 00.483550/axx...447..15
Kingma, D. P. and Welling, M. Auto-encoding variational
Bayes. In Proc ICLR, 2014, doi: 10.485500arXivv13122 6114.
Kong, D., Pang, B., Han, T., and Wu, N. Molecule de- sign by latent space energy baasey---eeddeee modeling and gradual distribution shifting. In Proc UAI, volume 216 of Pro- ceedings of Machine Learning Research, pp. 1109-20, 2023, URL httt:::::/pptt httppppooeeenngseeemee mlr. ..s.press///////ooommm v216 //66//kog2aa.. html.
Huang. X., Li, S., Dobriban, E., Bastani, O., Hassani, H., and Ding, D. One-shot safety alignment for large lan- guage models via optimal dualization. In Proc NeurIPS, 2024b. doi: 10448550/a2xi..245..9522
Huo, Z., Liu, W., and Wang, Q. Multi objective optimization method for collision safety of networked vehicles based on improved particle optimization. J Control and De- cision, 10: 13. 2022. doi: 10. 10000/033070662022222 2080771.
Korshunova, M., Huang, N., Capuzzi, S., Radchenko, D. S., Savych, O., Moroz, Y. S., Wells, C. I., Willson, T. M., Tropsha, A., and Isayev, O. Generative and reinforcement learning approaches for the automated de novo design of bioactive compounds. Commun Chem, 5(129), 2022. doi: 10.
Hvarfner, C., Hellsten, E. O., and Nardi, L. Vanilla Bayesian optimization performs great in high dimen- sions. In Proc ICML, volume 235, pp. 20793-817, PMLR, 2024. URL SoLLhtttss//proceeeee http::///ooceceeeeeeeoeee mlr. press// v235/h 2t555/vaaffee2222 html.
Kostrikov, I., Nachum, O., and Tompson, J. Imitation lear- ing via off- off-policy distribution matching. Proc ICLR, 2020. URL https: /pwpreveeeeeeeeeeeeeeeeeeeeeeeee net forum? id=Ryg-JC4FDr.
Jain, M., Bengio, E., Garcande A.,
Krishnamoorthy, S., Mashkaria, M., and Grover, A. Diffusion models for black-box optimiza- tion. In Proc ICML, volume 202, pp. 17842-57, 2023. URL URLnttpss//prooeeddng mlr press// w200//rrishhmmmoorryy3333 htmll.
J. Dossou, B. F. P., Ekbote, C. A., Fu, J., Zhang. T., Kilgour, M., Zhang, D., Simine, L.., Das, P., and Bengio, Y. Biological sequence design with GFlowNets. In Proc ICML, volume 162 of Proc Mach Learn Res, pp. 9786- 801. PMLR, 2022. URL https: ttttps:/pproceedine /ppp//peeeeng.... mlr. m.r.p /v v1 62/ jain2aa. html.
Kumar, A. and Levine, S. Model inversion networks for model-based optimization. In Proc NeurlPS, pp. 5126- 37, 2019. doi: 10.5555/3495724.44961555
Jang. S., Yoo, S., and Kang, N. Generative design by re- inforcement learning: Enhancing the diversity of topol- ogy optimization designs. Computer-Aided Design, 146: 103225, 2022. doi: 1w..0//...d.22022.113....
Kushner, H. J. A new method of locating the maximum of arbitrary multipeak curve in the presence of noise. J Basic Eng, 86(1):97-100 1964. doi: 10.1115/1.3653122
Kamil, A. T., Saleh, H. M., and Hussain, I. A multi-swand structure for particle swarm optimization: Solving the welded beam design problem. J Phys: Conf Ser, 1804: 012012, 2021. doi:
Li, S., Phillips. M., Yu, X., Kirby, R. M., and Zhe, S. Batch multi-fidelity active learning with budget con- straints. In Proc NeurIPS, pp. 995-1007, 2022a. doi: 10.5555/3600270.300033 300000000
Kantorovich, L. and Rubinstein, G. S. On space of totally additive functions. Vestnik Leningrad. Univ, 13:52-9, 1958.
Li, S., Wang, Z., Kirby, R., and Zhe, S. Deep multi-fidelity active learning of digh-dime outputs. In Proc Ini Conf Artif Intell Stats, volume 151, pp. 1694-711. PMLR, 2022b. URL https: hatp:://po//eeeeeeeeeeee mllr. press/v151/ 111151//122bhhtte
Ke. L., Choudhury, S., Barnes, M., Sun, W., Lee, G., and Srinivasa, S. Imitation learning as f-divergence minimiza- tion. In Algorithmic Foundations of Robotics XIV, pp- 313-29, 2021. doi: 10.1007/978-3-030 000079978-3000--6223--8810
Li, S., Kirby, R. M., and Zhe, S. Batch multi-fidelity Bayesian optimization with deep ruto-segressive net- works. In Proc NeurlPS, pp. 25463-75, 2024. doi: 10.5555/3540261.3544
Kim, M., Berto, F., Ahn, S., and Park, J. Bootstrapped training of score-conditioned generator for offline design of biological sequences. In Proc NeurIPS, pp. 67643-61, 2023. doi: 10.5555/3666122.366966
Liao, X., Li, Q., Yang, X., Zhang, W., and Li, W. Mul- tiobjective optimization for crash safety design of vehi- cles using stepwise regression model. Structural and Multidisciplinary Optimization, 35(6):561-9, 2008 doi: 10. 00.1007/50015 000007//0015-007.0000
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In Proc ICLR, 2015, doi: 10.48550/arXiv. 1412.6980.
12

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W. Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., dos Santos Costa, A., Fazel-Zarandi, M., Sercu, T., Can- dido, S., and Rives, A. Evolutionary-scale prediction of atomic-level protein structure with language model. Science, (79(6637)11123-3333 2023, doi: 10. 126/science. ade2574.
S., Baccus, S. A., Mernandez-Boussanee T., Ré, C., Hsu,
P. D., and Hie, B. L.. Sequence modeling and design from molecular to genome scale with Evo. Science, 386(6723); eado9336, 2024. doi: 10.
Nguyen, T., Agrawal, S., and Grover, A. Expt: Synthetic pretraining for few-shot experimental design. In Proc NeurIPS, 2023. doi: 10.5555/3666122.6666 109.
Loshchilov, I. and Hutter, F. Decoupled weight decay regularization. In Proc ICLR, 2019. URL https: //openreview. net/ forum? wommiidkBkkiiggg/
Nishiyama, T. and Sason, I. On relations between the relative entropy and Tii-Squaredddivergeee general- izations and applications. Entropy, 22(5), 2020. doi: 10.3390/622050563.
Ma, J., Cao, B., Dong, S., Tian, Y., Wang, M., Xiong, J., and Sun, S. MImd: A programming-fiee Al platform to predict and design materials. npj Comput Mater, 10, 2024. doi: 10. 1108//415524404--112233
Ohana, R., McCabe, M., Meyer, L. T., Morel, R., Agoes, F. J., Beneitez, M., Berger, M., Burkhart, B., Dalziel, S. B., Fielding, B., Fortunato, D., Goldberg, A., Hirashima, K., Jiang, Y.F.,, Kerswell, R., Maddu, S.. Miller, J. M., Mukhopadhyay. P., Nixon, S. S., Shen, J., Watteaux, R., Régaldo-Saie Saint Blancard, B., Rozet, F, Parker, L. H., Cranmer, M., and Ho, S. The Well: A large-scale collection of diverse physics simulations for machine learning. In Proc NeurIPS, 2024. URL https:
Ma, Y. J., Yan, J., Jayaraman, D., and Bastani, O. Offline goal-conditioned reinforcement learning via f-advantage regression. Proc NeurIPS, pp. 310-23, 2022. doi: 10.5555/300270.3602222.
Mashkaria, S. M., Krishnamoorthy, S., and Grover, A. Generative pretraining for black-box optimiza- tion. In Proc ICML, volume 202, pp. 24173-97, 2023. URLhttts:://poot 2202/mashkaria22 html.
/ //oppnreviee eww net. forum? datt:mmpdd000000011100
ngs. woags.mlr.pree..
Palizhati, A., Torrisi, S. B., Aykol, M., Suram, K., Hum- melshøj, J. S., and Montoya, H. Agents for sequential learning using calttiple-ideliig data. Sci Rep, 12(4694), 2022. doi: 10.1038/s415988 159-002-01--22
Matthews, A. G., Hensman, J., Turner, R., and Ghahra- mani, Z. On sparse variational methods and the Kullback- Leibler divergence between stochastic processes. In Proc Int Conf Artif Intell Stats, volume 51, pp. 231-9, PMLR, 2016. URL : proceedings mlr.presss v51 1. htmm/ttttttsstttm..mmmmmm/m 6, html.
Papalexopoulos, T. P., Tjandraatmadja, C., Anderson, R., Vielma, J. P., and Belanger, D. Constrained discrete black-box optimization using mixed-integer program- ming. In Proc ICML, volume 162, pp. 17295-322. PMLR, 2022. URL httptttts///ppcceeeeeee v1 62 /12/pppalll wxwp//papaaxxopoulloooo exwpoulosssso.ooooooo os html.
mlr mtt.pssssss/spp.////mmm
Maus, N. T., Jones, H. T., Moore, J. S., Kusner, M. Brad- shaw, J., and Gardner, J. R. Local latent space Bayesian optimization over structured inputs. In Proc NeurIPS, 2022. doi:
Pele, O. and Werman, M. Fast and robust Earth mover's distances. In Proc ICCV, pp. 460-7, 2009. doi: 10.1109/ ICCV.2009.54991.9
Maus, N. T. Wu, K., Eriksson, D., and Gardner, J. R. Dis- covering many diverse solutions with Bayesian optimiza- tion. In Proc Int Conf AI Stats, volume 206, pp. 1779-98. PMLR, 2023. URL https: pettps://proceeee press/v206 maus23a. html.
Pogue, E. A., New, A., McElroy, K., Le, N. Q., Pekala, M. J., McCue, L., Gienger, E., Domenico, J., Hedrick, E., McQueen, T. M., Wilfong, B., Piatko, C. D., Ratto, C. Lennon, A., Chung, C., Montalbano, T., Bassen, G., and Stiles, C. D. Closed-loop superconducting materials discovery. npj Comput Mater, 9, 2023. doi: 1031038/ s4 541524-023-0113-33
/ptppeeeeoggss.../////p//o monss.mrr.. mml.
Mullis, M. M., Ramo, I. M., Baker, B. J., and Reese, B. K. Diversity, ecology, and prevalence antimicrobials in nature. Front Microbiol, 10, 2019. doi: 10.3389/fmich. 2019.02518.
Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. In Proc NeurIPS, pp. 53728-41 2023. doi: 10.5555/3666122. 3668460.
Nachum, O. and Dai, B. Reinforcement learning via Fenchel-Rockafellaa duality, arXiv Preprint, 2020, doi: 10.485500/rXii.22001.11888
Nguyen, E., Poli, M., Durrant, M. G., Kang, B., Katrekar, D., Li, D. B., Bartie, L. J., Thomas, A. W. King, S. H., Brixi, G., Sullivan, J., Ng, M. Y., Lewis, A., Lou, A., Ermon,
Ray, T. and Liew, K. M. A swarm metaphor for multiobjec- tive design optimization. Engineering Optimization, 34: 141-53, 2002. doi: 10.1080003052110021099
13

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Sample, P. J., Wang. B., Reid, D. W., Presnyak, V., Mc- Fadyen, I. J., Morris, D. R., and Seelig, G. Human 5' UTR design and variant effect prediction from mas- sively parallel translation assay. Nat Biotechnol, 37:803- 9, 2019. doi: 10. 1038/041 158109991118-0--116-----
Wang, T.-H. J., Zheng. J., Ma, P., Du, Y., Kim, B., Spielberg, A., Tenenbaum, J., Gan, C., and Rus, D. DiffuseBot: Breeding soft robots with physics-augmmnttd generative diffusion models. In Proc NeurIPS, volume 36, pp. 44398 423, 2023. doi: 10.5555/3666 22.3668044.
Sobol, M. On the distribution points in a cube and the approximate evaluation of integrals. USSR Comp Math Phys, 7(4):86-112, 1967. doi: 10.10160001-5553366) 90144-9.
Wang, Y., He, H., and Tan, X. Truly proximal pol- icy optimization. In Proc Conf Une Artif Intell, volume 115, pp. 113-22. PMLR, 2020. URL www.reeedinns.....p..... mws.smmr.... wwt.m...rrrs/l1s/.........
https://pret /ttps://prrceeediggs wang20b. html.
Stanev, V., Oses, C., Kusne, A. G., Rodriguez, E.,, Paglione, J., Curtarolo, S., and Takeuchi, I. Machine learning mod- eling of superconducting critical temperature. npj Comput Mater, 4, 2018. doi: 10. 00.103//41122--1155 8-0005-8..
Watanabe, K. and Isobe, N. Sinkhorn algorithm for sequen- tially composed optimal transports. arXiv Preprint, 2025. doi: 00448500/aXXi..2112.3111
Tan, R.-X., Xue, K., Lyu, S.-H., Shang, H., Wang, Y., Wang, Y., Fu, S., and Qian, C. Offline model-based optimization by learning to rank, arXiv Preprint, 2024, doi: 10.48550/ arXiv.2410.11500
Wildman, S. A. and Crippen, G. M. Prediction of physic- ochemical parameters by atomic contributions. J Chem Info Comp Sci, 39(5):868-77 1999. doi: 10.1021/ ci9903071.
Terjék, D. and Gonzälez-Sánchez, D. Optimal trans- port with f-divergence regularization and generalized Sinkhorn algorithm. In Proc AISTATS, volume 151 of Proc Mach Learn Res, pp. 5135-65. PMLR, 2022. https:::: httpss////coeeeeeeeeeeeeee weeediinnnnni.ere.emeeeeeeeeeeee www. mlr.press/ v151/terjek22a.2 html.
Williams, R. J. Simple statistical gradient-following algo- rithms for connectionist reinforcement learning. Mach Learn, 8(3-4):229-50 1992. doi: 10. 001107/BF000992696
Wilson, T., Moriconi, R., Hutter, F,, and Disenroth, M. P. The reparameterizzation trick for acquisition functions. arXiv Preprint, 2017. doi: 10.485550/rxivv1712.. 172.000044.
Todorov, E., Frez, T., and Tassa, Y. MuloCo: A physics engine for masel-based control. In Proc Int Conf Intell Rob Sys, pp. 5026-33, 2012. doi: 10. 10.1109/RRO.2011 6386109.
Wilson, J. T., Hutter, F., and Deisenroth, M. P. Maximizing acquisition functions for Bayesian optimization. In Proc NeurIPS, pp. 9906-9911, 2018. doi: 10.5555/3327566. 3327655.
Trabucco, B., Kumar, A., Geng, X., and Levine, S. Conser- vative objective models for effective offline model-based optimization. In Proc ICML, volume 139, pp. 10358-68, 68. PMLR, 2021. URL https: http:///ooceeeeeeeeee/ee mlr. press/vil 39/11/9/00 (1333///truuco2211
Wu, D., Niu, R., Chinazzi, M., Ma, Y.A.,, and Yu, R. Disen- tangled multi-fidelity deep Bayesian active learning. In Proc ICML, pp. 37624 34, 2023. doi: 10.5555/3618408. 3619975.
html.
Trabucco, B., Geng. X., Kumar, A., and Levine, S. Design- Bench: Benchmarks for data-driven offline model-based optimization. In Proc ICML, volume 162 of Proceed- ings of Machine Learning Research, pp. 21658-76, 2022. URL BRLhttp:: http:////ooceeeessee/ee///csssse malr.press! v1 62. htt/ttrabccoo22a html.
Xiong, H. Dped: Discrete principal coordinate descent for binary variable problems. Proc AAAI Conf Artif Intell, 36 (9):10391 8, 2022. doi: 10.1609/aaai, v36ii,........... 23....222...........
Yakovchuk, P., Protozanova, E., and Frank-K Kamenetskii, M. D. Base-stacking and base-pairing contributions into thermal stability of the DNA double helix. Nucleic Acids Res, 34:564-74, 2006. doi: 10. n0.1093/aar/kkj454.
Tripp, A., Daxberger, E., and Hemándeez-Lobato, M. Sample-efficient optimization in the latent space of deep generative models via weighted retraining. In Proc NeurIPS, pp. 11259-72, 2020. doi: 10.5555/3495724. 3496669.
Yao, M. S., Zeng, Y., Bastani, H., Gardner, J. R., Gee, J. C., and Bastani, O. Generative adversarial model-based optimization via source critic regularization. In Proc NeurIPS, 2024. doi: 10448550arXx..2202266533
Tsybakov, A. B. Introduction nonparametric estimation. Springer New York, NY, 2008. doi: 10.1007/b13794.
Yu, P., Zhang, D., He, H., Ma, X., Miao, R., Lu, Y., Zhang, Y., Kong, D., Gao, R., Xie, J., Cheng, G., and Wu, Y. N. Latent energy-based based odyssey: Black-box optimization via expanded exploration in the energy-based latent space. arXiv Preprint, 2024. doi: 00.48550//rxiv.2440. 16730.
Wang, C., Jian, Y., Yang, C., Liu, H., and Chen, Y. Beyond reverse KL: Generalizing direct preference optimization with diverse divergence constraints. In Proc ICLR, 2024. doi: 10. 00.48550/arxi..230..1.
14

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Yu, S., Ahn, S., Song, L.,, and Shin, J. Roma: Robust model adaptation for offline model-based optimization. In Proc NeurIPS, 2021. doi: 10.5555/3540261 10.555//540266.33406111 1.540614.
Yuan, Y. Chen, C., Liu, Z., Neiswanger, W., and Liu, X. Importance-aware co-teaching for offline model-based optimization. In Proe NeurIPS, pp. 55718 187, 2023. doi:
Yuan, Y., Zhang, Y. Chen, C., Wu, H., Li, 7.,, Li, J., Clark, J. J., and Liu, X. Design editing for offline model-based optimization. arXiv Preprint, 2024, doi: 10.48550/arXiv. 2405.13964.
Yun, T., Yun, S., Lee, J., and Park, Guided trajectory generation with diffusion models for offline model-basee optimization. In Proc NeurIPS, 2024. URL ht tps: //openreview..o net oot.//oooommiiiioooeoooae
Zeni, C., Pinsler, R., Zügner, D., Fowler, A., Horton, M., Fu, X. Wang. Z.. Shysheya, A., Crabbé, Ueda, S., Sordillo, R., Sun, L., Smith, J., Nguyen, B., Schulz, H., Lewis, S., Huang, C.-W., Lu, Z., Zhou, Y., Yang. H., Hao, H., Li, J., Yang, C., Li, W., Tomioka, R., and Xie, T. A generative model for inorgamic materials design. Nature, 2025. doi: 10.0088/411586-025-886626.
Zhang, G., Martens, J., and Grosse, R. Fast convergence of natural gradient descent for overparameterized neural networks. In Proc NeurlPS, pp. 8082-93, 2019. doi: 00.5555/3454287.4555000
Zhang, X.-C., Wu, C.-K., Yi, J.-C., Zeng, X.-X., Yang. C.-Q., Lu, A.PP. Hou, T.-J,, and Cao, D.-S. Pushing the boundaries of molecular property prediction for drug discovery with multitask learning BERT enhanced by SMILES enumeration. Research, 2022:0004, 2022. doi: 10.34133/research..00.
Zhou, H., Ma, X., and Blaschko, M. B. A corrected expected improvement acquisition function under noisy observations. In Proc Asian Conf Mach Learn, volume 222, pp. 1747-62. PMLR, 2024a. URL https::/ppoceeddnggs mlr press/v222/ zhou24a. html.
Zhou, Z., Ji, Y., Li, W., Dutta, P., Davuluri, R. V., and Liu, H. DNABERT-2: Efficient foundation model and benchmark for multi-species genomes. In Proc ICLR, 2024b. URL htthttps: http:p/////peeeeee....o ner/forum? id 1B=4EEEB4
Zhu, C., Byrd, R. H., Lu, P., and Nocedal, J. L-BFGS- B: Fortran subroutines for large-scale bound-constrained optimization. ACM Trans Math Softw, 23(4):550-60, 1997. doi: 10. N0.1145/779232.27926.
15

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Appendix
Table of Contents
A. Proofs
16
Proof for Lemma 3.1: Diversity Collapse in Reward Optimization
Proof for Lemma 3.3: Catropy-Divergennc Formulation
Remark A.I: Equivalence of Lemma 3,3 and Canonical State-Matching
Proof Lemma 3.4: Explicit Dual Function (7)
B. Additional Implementation Details
19
C. Additional Background and Preliminaries
C.I. f J-Divergence and Fenchel Conjugates
C.2. Constrained Optimization via Lagrangian Duality
C.3. Wasserstein Distance and Optimal Transport
D. Additional Results
23
D. Additional Design Quality Results
D.2. Additional Design Diversity Results
D.3. Imposing Altemative f-Divergence Diversity Objectives via Mixed-Divergence Regularization
D.4. Theoretical Guarantees for DynAMO
D.5. Comparison with Offline Model-Free Optimization Methods
D.6. Empirical Computational Cost Analysis
D.7. T- --eeighted Distribution Visualization
D.8. Distribution Analysis of Quality and Diversity Results
D.9. Why Is Diversity Important?
E. A Ablation Studies
44
E.I. Sampling Batch Size Ablation
E.2. Adversarial Critic Feedback and Distribution Matching Ablation
E.3. B and Hyperparameter Ablation
F.4. Oracle Evaluation Budget Ablation
E.S. Optimization Initialization Ablation
A. Proofs
Lemma 3.1. (Diversity Collapse in Reward Optimization) Suppose that there exists finite set of globally optimal designs
n such that zrj = arg max r()) and nd == r(x;) is the optimal reward given finite, non-uniform reward function r(z). Given any distribution g", we can decompose into the form q" (x) E, w75(x xÿ)) Σ, w/ = xz) + q(x),
where wy 2 for all j. and ((x) and q(z})= 0 for all j. Then, ((x) satisfies dx q(z)
Proof. First, note that if da ((x) = 0, we have
E Ex (2) [ra)) = f dr Σ 1,ōlx xz)r(x))= = Ew, / dæ 2z)r(z) Ewrr" r* Σ wy (16)
which is optimal. Next, we prove that if dx q(x) > then Exng [r(x]] rr To this end, we define
1 - <r r(x) s n n
1 CX Vn (17)
Xi : ( - X, = x x
for each n E N. Note that all X, are disjoint by construction; also by construction, we have X' \ {2z} = - X..
Furthermore, note that since q(z) = 0 for x = æ for some j. we have 0 < I q(x) = Eie Sx, dx q(x), so it must be that Sx. dx ((x) for some m. As consequence, have
f xm dx (())(() - -((x)) m Xm
1 / dx ((x) > 0.
16

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Thus, the expected reward would be
f dxx"(())r()))
Bx [r(x)]
= =far dx
r(z) + Σ /
Σ wzō(r - -z5)+ + E wff = 23))
dr dx((())(x))
n=1 Kn
(18)
< {(a dæ
Ewjō(z zz²) + Σ w/y(2 = xz))
1" Σ
dr q(x)))"
n=1
Nin
/ dæ q" (x)r*
so q" is suboptimal. The claim follows.
Lemma 3.3. (Entropy-Divergeem Formulation) Define ((x) in (5). An equivalent representation of J(z)) is
((x) -H(4"()))) (1+ (z) (pb(2))
(19)
where H(-) is the Shannon entropy and (-I(- the KL divergence.
Proof. Firstly, note that
3 J(T) = Eng [ra(x)] - DKL. (0- (1bb)
= r Eag (ogerolo)) - -3DxI (pbb)
(20)
= E bloge (ro()) BDK (1=)
where = denotes an equivalent representation of the objective (i.e., scaling J(T) by 7 > does not change the optimal
policy **), Further rewriting,
e (x)
(To(()) -BDKL (pb)
((T) Egr
log
+ Eg log Z" (DDx(0"((pbb) ~ Eg log
(21)
Zr
Z
where we omit the constant Ey= log 27 because the expectation value argument is independent of the policy æ. The remaining expectation value can be re-expressed via importance weighting:
9" PDD
( Eps,
log
3KL (Pb)
(22)
ZT
Assuming that the surrogate r1( well-trained the offline dataset D (i.e., r(x) ro(a:) Var D), have
<<((x))
n"'
((T) = Epg
log
1x = Epp
logp5
-3Dxx(""11p5)
(23)
Pb
pi
from Definition 3.2. Further rewriting, we have
9"
9" ]
J(T) = Epj
log
PD
- (9 (pb)
PD
PD
q²t
Enp
log
+ Epp
log aº' - -3Dri. (qx (1%))
(24)
PDD
9"
- Ppp
log
Egy |- log 9a1
-3DKL (q" ((p))))))))
PD
17

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
From the definition of KL-divergence,
q
9"
( - --1++
log
Enp |- log 9")
PPD
PD.
9"'
(25)
-(1 + 3))pp
fKL
Eng Life (q
PD
-1 + 3KL (1" 10"01111 (lpb) -H(a")
up to a constant, where (x)(x) = x log and f((x) = - log x are convex functions, H(-) is the Shannon entropy, and
DxL (-I1-)) is the KL divergence.
Remark A.I (Equivalence of Lemma 3.3 and Canonical Satte-Matching). Continuing from (25), one might notice that J/ (
can be equivalently rewritten as
J(x) = -1 (1+ p)
q
log
9"
Eg |- log q")
D
P
9
9"
q"
-(1 + 3)1E
log
Ep
log q"
PDD
PD.
Db
9
9"
q log((")) PD
= -(11 + 3)EEp
log
(1 + B) Pb
(26)
PD
PD.
9-
9"'
= -(1 + B3)Ep.
log
PDD
Pb
(1*)L/((+3))
(41)0/(1+)))
-11 B)
log
Pbb
Assume that there exists a probability distribution (x) such that (z) x (P)(
Then
3/(1+0)
9"'
q"'
(1+)=-11+ log
-3EE
log
-B (pb)
(27)
P
(x)
PDD
(b)
In other words, the optimization objective considered in (5) and in Lemma 3.3 is equivalent to pure state-matching
objective -3DxL ( predicated on the existence of 'rescaled' probability distribution (x)) as defined above.
Lemma 3.4. (Explicit Dual Function of (7)) Consider the primal problem
max
J(T) = -(((") - (1 + Dx
(28)
s.t.
Epp (x)) Egg |c* (=)] s Wo
The Lagrangian dual function g(A) is bounded from below by the function ge(A) given by
ge(A) = 3 [X(Epg'' cx() Wo) Epg e dc (x)-1)
(29)
Proof. Define [k) == u log u. From (10), the dual function gAA IR+ + IR is given by
((A)= = min
(1 + B) JKL
- Eg log (a*) + BA (Epp c² (x) c. - Wo) (())-W))
Hell
P
min xell
(1 B) B)pp fKL.
9"
(Bpg fki.
9"
+ Ry log + BA (Bppcc(a) c - Wo)
(30)
ppp
-min rell
SEpp fRI.
9"
En log pb + 3A (Epg c² (xx E Wo)
Pi
where we define 3A E R4, the Lagrangian multiplier associated with the constraint in (7). Rearranging terms,
v(A)
min xell
BEpg [-
[-
\xe"
9"
SKT.
9"
E, logrb + BABp, (x) -3AW
(31)
Pip
PD
18

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Because the sum of function minima is lower bound on the minima of the sum of the functions themselves, we have
q²
q-
s(2) 2 Ppg xell
min
Ae
+
JkI.
max
[Bp= logpbl + min [BAB -3WW]
PiD
PD
mell
nell
(32)
g
q"'
~ 3
min sell
Ac
+
fRI.
BX co (x) - BAW
PD
PD
ignoring the term ea [En log Pb] that is constant with respect to X. In general, simplifying (32) is challenging if not intractable. Instead, we note that minimizing over the set of admissible policies II achieves optimum that is lower bounded by minimizing over the superset
g(A) 2 Ppp
min |- (x) + fKI. (=)] + py c² (x) -3WW SER
(33)
= 3 Eyp (Ac" (1:) A(R (x) Wo)]
where (+) is the Fenchel conjugate of a convex function f(-). The Fenchel conjugate of fKI(U) = a log u fk((2) e" following Borwein & Lewis (2006), and
(A) 2 B - Epggee000)
(x)-1 + ((Epp,c''x) - -Wo)]
(34)
Define the right hand side of this inequality as the function ge(A) and the result is immediate.
B. Additional Implementation Details
Oracle Functions for Optimization Tasks. The task-specific oracle reward functions r(x) are developed by domain experts and assumed to exactly return the noiseless reward of all possible input designs in the search space X'. The oracle functions associated wit tasks from the Design-Bench MBO evaluation suite are detailed by the original Design-Bent Bench authors in Trabucco et al. (2022); briefly, the TFBind8 (i.e., TFBind8 -eacndd---xxee--e in Design-Bench) task uses the oracle function from Barrera et al. (2016); the UTR htt.:///ee......ooo...eeeeeeoaa. (2020); the ChEMBL the oracle function from Trabucco et al. (2022); the Superconductor (Superconduuttoo---adddoooooooo
task uses the oracle function from Angermüeller et al. task uses task uses the oracle function from Hamidieh (2018); 8); and the D' D'Kitty (DKi ttyMorpho ogy-Exact-v00 task uses a MuloCo (Todorov et al., 2012) simulation environment and learned control policy from Trabucco et al. (2022) to evaluate input designs. The Molecule task uses the oracle function from Wildman & Crippen (1999).
Data Preprocessing. For all experiments, we follow Mashkaria et al. (2023) and normalize the objective values both the offline dataset 2 and in those reported in Section according to:
9 Mmin
y
(35)
Umax
Mmin
where ÿ = r<z) is the original unnormalized oracle value for an input design I, and Umux (resp.. Vmin)) is the maximum (resp., minimum) value in the full offline dataset. A reported value of y > 1 means that an offline optimization experiment proposed a candidate design better than the best design in the offline dataset. Note that in many of the MBO tasks, the publicly available offline dataset D only a subset of the designs in the full offline dataset; is therefore possible (and frequently the case) that m y < 1 in our MBO tasks.
As introduced in the main text, we learn VAE (Kingma & Welling, 2014) model to encode and decode designs for discrete optimization tasks to and from a continuous latent space, and perform our optimization experiments over the continuous VAE latent space. Following prior work (Maus et al., 2022; Tripp et al., 2020; Yao et al., 2024), we co-train a Transformer-based VAE autoencoder (consisting of an encoder e.p : X + X' parameterized by sot and decoder dæ X: X > E parameterized by 7*) with the surrogate model rg : X + R (parameterized by 0*) according to
0* 1pp",
arg min E(Z,(<))) log logdg(zle((x)) + BD tallroe))))) (V) (1)le rxx)1 (36)
<xxxxx
where (0, I) is the standard multivariate normal prior and a = 1, B = 10-4 are constant hyperparameters. We can then perform optimization against ro trained on the 256-dimensional continuous latent space of the VAE, and then decode the
19

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
candidate designs using d-(+) derive the corresponding discrete design following prior work from Maus et al. (2022);
Gómez-Boomarellii al. (2018). We again use an Adam optimizer with learning rate of ry 3 x 10-1 for both the VAE and the forward surrogate. In this way, the search space for our discrete tasks becomes the X' C R for d = 256, the surrogate model is simply rg: X' + E. and the reward function r : X + R is now
r r(x) := Ed [+())
(37)
where F: - - TR. is the original expert oracle reward function over the discretized input space X, and r(x) is the corresponding oracle reward function that accepts our continuous inputs from X' as input. Note that for the MBO tasks over continuous search spaces (i.e., the Superconductor and D' D'Kitty tasks), we treat X' = X and fix both the encoder exp and decoder de to be the identity functions, as no transformation separate continuous search space is necessary.
Optimization Experiments. All baseline methods run evaluated using their official open-source implementations made publicly available by the respective authors. In DynAMO, we initialize all optimizers using the first b elements from a d-dimensional scrambled Sobol sequence (Sobol, 1967) using the official PyTorch quasi-random generator Sobo SobolEng iEnging ne implementation, where is the sampling batch size and d is the dimensionality of the search space. Note that the Sobol sequence only returns points with dimensions between 0 and I; for each task, we therefore un-normalize the sampled Sobol points do according to 20 Jmin + (5t (Imaxx Fmmin), where Tmax, Tmin are the maximum and minimum bounds on the search space for our experiments, respectively. We fix miin -4.0 and Tmax +4.0 for all d dimensions across all tasks.
In all experiments reported in Table 1, each optimizer continues to sample from the search space in batched acquisitions of b samples-we we set = 64 for all our experiments unless otherwise stated. After each acquisition, we score the sampled designs using the (penalized) forward surrogate model (i.e., the Lagrangian in (9) for DynAMO). If the maximum prediction from the recently sampled batch is not least as optimal as the maximum prediction of the previously sampled designs, then we define the acquisition step as a failure; sequence of 10 consecutive failures triggers restart in the optimization process where the optimizer starts from the scratch beginning with sampling with the Sobol sequence to initialize the optimizer as described above. After 3 restarts are triggered, we consider the optimization process terminated, and all designs across all restarts are aggregated to choose the top l = 128 final candidate designs to be evaluated using the oracle reward function.
Exeluded Baselines. We exclude Boosting offline Optimizers with Surrogate Sensitivity (BOSS) from Dao et al. (2024) and Normalized maximum likelihood Estimation for Model-based Optimization (NEMO) from Fu & Levine (2021) from our experiments because they do not have open-source implementations.
Excluded Optimization Tasks. In our experiments, primarily evaluate DynAMO and baseline methods on optimization tasks from Design Bench, a suite of offline MBO tasks introduced by Trabucco et al. (2022). The following tasks from the original authors were excluded from our experiments: (1) Ant Morphology, excluded due to reproducibility issues as per GitHub Issues Link and OpenReview Discussion; (2) Hopper Controller, excluded due to errors in the original open-source implementation per GitHub Issues Link and prior work (Tan et al., 2024; Mashkaria et al., 2023); (3) NAS (Neural Architecture Search) on CIFAR 10, excluded due to its prohibitively expensive computational cost for evaluating the oracle function as noted in prior work (Tan et al., 2024; Yu al., 2021; Fu & Levine, 2021; Nguyen et al., 2023); and (4) 100 excluded due to its domain and experimental similarity with the TFBind8 task already included in our evaluation suite. We augment our evaluation suite with the UTR task from Trabucco et al. (2022) and the Molecule task from Yao et al. (2024); Brown et al. (2019) to provide a comprehensive experimental evaluation of DynAMO and baseline methods across a wide variety of scientific domains.
C. Additional Background and Preliminaries
C.1. /-Divergence and Fenchel Conjugates
Definition C.1 (f-Divergence). Suppose we are given two probability distributions P(x), Q(x) defined over common support X. For any continuous, convex function f E., + R that is finite over R_.,, we define the f-divergence between P(z), as
D((((=))P())) := Egwp(()) [ (l())
((m) ]
(38)
We refer to f as the generator of Dy(-1l)) Two commonly used f-divergences are the Kullback- -ivereeieeeeence (defined by the generator [K)( = u log u) and the x² D'-veeegeence (defined by the generator fx²(u) = (u - /2).
20

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Definition C.2 (Fenchel Conjugate). The Fenchel conjugale (i.e., FegenddreFFenchl transform) of function f /:u R defined as
(*) :- -if (-(u, v) + f(u) u E U}
(39)
where (u, vi) is the inner product, and - V + R is the Fenchel conjugate defined over the dual space V of U. Importantly, the Fenchel conjugate function is guaranteed to always be convex Borwein Lewis (2006) regardless of the (non jeonvexity of the original function f. This allows us make important convergence guarantees Appendix D.4 in solving the Lagrangian dual problem in Algorithm 1. Fenchel conjugates are commonly used in optimization problems to rewrite difficult primal problems into more tractable dual formulations Ma al. (2022); Borwein & Lewis (2006); Agrawal & Horel (2021)-we leverage a similar technique in our work in Algorithm 1.
Lemma C.3 (Fenchel Conjugate of the KL-Divergence Generator Function). Recall that the generator function of the KL-divergence is JKI(VV u log for E R4++. The Fenchel conjugate of this generator is (kk( e"
Proof. The proof follows immediately from the definition of the Fenchel conjugate in (39).
(v) == sup {wo log R++}
(40)
We differentiate the argument on the right hand side with respect to a to find the supremum given particular v V:
Ou lue - u log u]
= log u = 0 + u² = en-1
(41)
==
is easy to verify that u" is a maxima. Plugging this result into (40),
(u) u"v u² log u" ve--1_ (1-1)e--1 =c"-
(42)
Lemma C.4 (Fenchel Conjugate of the <iveeeeree Generator Function). Recall that the generator function of the diveeeegen is (u²( = { - for E R4+. The Fenchel conjugate of this generator is (²2(v) = n²
2 + n.
Proof. The proof follows immediately from the definition of the Fenchel conjugate in (39).
1 2 7 - 1)2 E TR+
M (v) =
sup
(43)
We differentiate the argument on the right hand side with respect to to find the supremum given particular V:
Ou uv 2 (u 1)² = ----u +1 -
(44)
is easy to verify that u² is a maxima. Plugging this result into (43),
1 j(r- 1)² = (o 1)v
} ((v + 1) 1)2 = v² +
1
y2
(z) = u²v
00-
+
(45)
2
2
2
Additional details and related technical discussion are offered by Borwein & Lewis (2006); Nachum Dai (2020); Ma et al. (2022); Amos (2023); and Terjék & Gonzälez-Sáncher (2022).
C.2. Constrained Optimization via Lagrangian Duality
In our problem formulation in (7) for DynAMO, we reformulate any naïve MBO problem as separate constrained optimization problem, which generally of the form
minimizezEx f(x)
(46)
subject to f((x) 5 Vi E mm)
21

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
given a set of m constraints. In general, satisfying any arbitrary set of (potentially nonlinear) constraints is challenging if not intractable, and it is often desirable instead solve related unconstrained optimization problem. One common mechanism to perform such problem transformation is to define the Lagrangian of (46) as
(x;; X) = f(x) + (2, [f((e) f2(2)
fm(#)])
(47)
where A E R and L: X x R + IR is a real-valued function. It can be shown (Boyd & Vandenberghe, 2004) that the constrained optimization problem in (46) is equivalent to the unconstrained problem
minimize maximizexeRe [(x; X)
(48)
in terms of the Lagrangian, where f represents an element-wiss inequality. The dual problem of (48) is constructed by reversing the order of the minimization and maximization problems:
maximizeXeRg minimizenex C(x; X) maximizeXRR 9(A)
(49)
where we implicitly define the dual function g(X) == x ((x; X). In general, it is guaranteed that the optimal solution to the dual problem in (49) is lower bound for the optimal solution for the original problem in (46) from weak duality; if (x) and fa( are convex and bounded from below such that Slater's 's condition applies, then strong duality guarantees that the optimal solutions to the dual and original problems are equal.
As an additional remark, we note that in our problem formulation in (7), Slater's condition is not satisfied as both the surrogate reward function (x)))) and adversarial source critic e (xx) may be arbitarily non-convex. However, find empirically that the guarantee of weak duality is sufficient to make the approach of Lagrangian duality both tractable and effective in solving (7) to give us DynAMO.
Furthermore, we also find that solving the dual optimization problem in (49) requires us to first solve for the dual function (hi)t is a challenging task in general, and prior work has attempted to either approximate g(x) under specific assumptions on the search space X' (Yao et al., 2024) or forego solving for g(X) entirely by instead treating X as a hyperparameter to be manually tuned or set heuristically (Trabucco et al., 2021; Yu et al., 2021; Chen et al., 2023c). In our work, we show how penalizing the optimization objective via a KL-divergence term as in (7) is sufficient to yield an exact solution for the dual function g(X) (see Lemma 3.4). This is advantageous because can be shown (Boyd & Vandenberghe, 2004) that g()) is convex; assuming that the gradient of g(X) has a bounded Lipschitz constant, we can therefore arrive in an e-neighborhood around the optimal X within O (()/ time (Bubeck, 2015; Grimmer et al., 2023; Zhang et al., 2019). We leverage this convergence guarantee to solve for X* naively via gradient ascent DynAMO (see Algorithm 1).
C.3. Wasserstein Distance and Optimal Transport
In our motivating problem formulation in (7), we introduce constrained optimization problem where the constraint is function of source critic e* X + IR with bounded Lipschitz norm. In this section, we show how this choice in adversarial source critic is connected to classical theory in optimal transport.
In general, the Wasserstein distance (p(((())(((=zz)) is distance function between pairs of probability distributions P(x) and (xx). Given metric space (M, d), we define the p-Wasserstein distance as
inf W, (())))))(())) 7Er(P.Q) ([0,,a"),, x())) 1/v
(50)
for 2 1, and where N(P, Q) is the set of all couplings hetween P(c)), Qxr). Intuitively, one can think of the p- pasaserstnein distance as representing the cost associated with the optimal (i.e., montmiiniii strategy of "transporting" the 'mass" of one probability distribution to another.
For any two arbitrary multidimensional distributions, exactly computing the Wasserstein distance between them is computa- tionally expensive (Pele& & Werman, 2009; Watanabe & Isobe, 2025; Cuturi, 2013) and in many cases intractable in practice. Instead, a common technique used by Arjovsky et al. (2017) and others is to leverage the Santtorovich-Rubinee theorem (Kantorovich & Rubinstein, 1958) to exactly rewrite (50) (specifically for the p = 1 Wasserstein distance) as
duality
W1 (P))))(Q())
sup [Ew))())(()) [xx
(51)
||
22

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Basce Optimization
where ||e|| is the Lipschitz norm of source critic function e X + R. Intuitively, we can think of the function c(xr) as assigning of value of in-distribution-ness relative to P(x): larger value of c(x) (informally) means that the source critic predicts the input x to be more likely to have been drawn from P(z) as opposed to Q(z). In DynAMO, we follow Arjovsky et al. (2017); Yao et al. (2024): and others to constrain the MBO optimization problem by requiring that the estimated Wasserstein distance between the distribution of generated designs and r-weighted distribution of designs from the offine dataset is less than a constant Wo according to (51)-see Algorithm for additional details.
D. Additional Results
D.1. Additional Design Quality Results
We supplement the results shown in Table 1 with the raw Best@ 128 oracle quality scores reported for each of the 6 tasks in our evaluation suite in Supplementary Table A1.
In Section 4, we define the @est@k score to evaluate the quality of observed designs according to a hidden oracle function used for evaluation of candidate fitness. Achieving a high Best@ B = score ensures that a desirable design found. Consistent with prior work on batched optimization methods (Trabucco et al., 2021; Mashkaria et al., 2023; Krishnamoorthy al., 2023), we are also interested in the Median @k score defined as
Median @ (ataaan@((((5) ((==
(/))) )/m) == median r(x;)
(52)
evaluate whether a batch of candidate designs (as opposed any singular design) is generally of high quality according to the oracle rzz). We report the Median @ k score for A 128 in Supplementary Table A2; in general, we find that DynAMO does not perform well as other note that in many offline optimization applications, we are often not as interested in how the median design performs, but rather if we are able to discover optimal and near-optimal designs. For this reason, we chose to focus on the Best@ 128 oracle scores in Table to evaluate the quality of designs proposed by an optimizer in our main results. Nonetheless, future work may explore how better tune DynAMO (e.g.. the and hyperparameters in Algorithm 1) achieve more desirable Median @ 128 scores.
menttiiiii baseline methods according to this metric. However, we
D.2. Additional Design Diversity Results
We supplement the results shown in Table 1 with the raw Pairwise Diversity scores reported for each of the 6 tasks in our evaluation suite in Supplementary Table A1.
In Section 4, we describe the Pairwise Diversity metric previously used in prior work (Kim et al., 2023; Jain et al., 2022; Maus et al., 2023) to measure the diversity of samples obtained from a given offline optimization method. We can think of Pairwise Diversity as measuring the between-candicee diversity of candidates proposed by a generative algorithm. However, this is far from the only relevant definition of diversity; other possible metrics might measure the following:
1. Sandidate-Datasee Diversity: How novel is a proposed candidate compared to the real designs previously observed in the offline dataset?
2. Aggregate Diversity: How well does the batch of candidate designs collectively cover the possible search space?
To evaluate (1), we follow prior work by Kim al. (2023) and Jain et al. (2022) and evaluate the Minimum Novelty (MN) for batch of is final proposed candidates with respect the offline dataset D, defined as
MN( M
D) = = E.F
min d(af xx) RED
(53)
where ' is the task-specific dataset of offline sample designs and of is the ith candidate design proposed by an optimization experiment. Following (14), we define the distance function d(.,-) as the normalized Levenshtein edit distance (Haldar & Mukhopadhyay, 2011) (resp., Euclidean distance) for discrete (resp., continuous) tasks.
For (2), we report the I,, Coverage (L1C) of the candidate designs, defined as
dim
LIC(fz F 11))) = dim(z)
max 1
2zkel
(54)
j
kml
23

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Basce Optimization
Table AL. Quality and Diversity of Designs Under MBO Objective Transforms (Full). We evaluate DynAMO against other MBO objective-modee modifying methods using six different backbone optimizers. Each cell consists of 'Bett@ 128/Pairwise Diversity' oracle scores separated by a forward slash. Both metrics are reported mean 195% confieecce intevval) across 10 random seeds, where higher is better. Dataset D reports the maximum oracle score and mean pairwise diversity in the oflline dataset. Bolded entries indicate overlapping 95% confidence intervals with the best performing algorithm (according the mean) per optimizer. Bolded (resp.. Underlined) Rank and Optimality Cap (Opt. Gap) metrics indicate the best (resp., second best) for given backbone optimizer.
Grad.
TVBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitty
Rank 4
Opt. Gap
Dataset
43.9/65.9
59, 39.4/57.3
60.5/60.0
88.9/36.7
40.0/66.0
88, 88.4/85.7
Baseline
90.0"* (.0))
80. 10.9 (
60.244
88. 2244, 1 103
36.0 16.0th
65. 40.0
5.0/5.5
6.8/5332
COMs
60.4% 60,4%-
60.2 20022 2
60.2 60.2/4.1/7999 9
88, 440444 gean
22.5
71. g0.00000000
7.3/6.5
-3.0V-53.5
COMs*
93.1666//6666 (1
67. 0th 157.444
1/.66 6
97.1 17. gon
41 /99.559
91.8 91.8 /2. 1ass
2.5/2.8
12.3//6.9
RoMA"
62.0
60.9 60.9122177,99 1777
60. 22. pra0
88.8 2/24, 21s
36, 36.0/==/0.0mm
65. 60.400,,,00m
6.7/5.7
-1.2/-53.3
RoMA'
66.5 66.5590/20.39000
77.8 17.8 grae
63. 63.3% /o 2000
84,5 grom
49.0 49.0/19//441 1(1
95,2
3.8/5.8
9:2/-46.8
ROMO
462, (99))
66.8 8 1
63. 63.0%%/53 gmo
/48.
18.77%/5117771
8/22
4.2/2.8
10.9/-12.7
GAMBO
73.1 /3.7,3377
77.1% 111.11111111 2220
/2.
1 1005
3,2/5.3
10.51-52.1
DynAMO
90.3*47/669966
86.2" 168.2'
"4.4"-/7772""
91.2 93.001
44.2 44.2"4//129/555)
89.8 10045
2.8/1.2
14.2/27.8
Adam
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitty
Rank
Opt. Gap 1
Baseline
62.9(159//2.00
69. 69,7th /11 1
64.9"" g4 ghh
/1.6 116
37.8 37.8% 16,,4044
58.4 58.4 26.4m
4,5/6.0
0,5/-52.4
COMs
62.9 /13.6
65.1 (s
/5..0
42.44 22
$7.3 16.3
6.0/5.3
-3.0-552.4
COMs'
67.1 157
64.6 /81.555
95.3 95.3" /3, 13,7133)
39.6551179.39
67. 07.185//1188
3,2/3.0
8.1/-12.3
RaMA"
62.9
69.7 69.7" 110.
62,9 62.g (4,77
84, 116.
37 16,44 (14.5)
58. 16,2
4.5/6.3
0,5/-52.4
RoMA"
96.5 06.1.30
77.8 77.8 13.5 m3
92.3 gmmm
49.8 49.894944944
95.7 /1..
2.8/5.2
14.5/-45.8
ROMO
95.6 6. 75.,
61.0% 155. y
63. /53.5.
90.4 /500 7000
((0a) 31.8
71.0 71.0% 2..
4.8/2.8
6,41-20.5
GAMBO
94.0 1
60.0
60 /12 (
91. 11.4 6 61ss
88.4 (1.8 6
5.3/5.8
8.6/551.9
DynAMO
(54.8
86.2 16.220,72..24
"s.2" /848
91 28 189.9/5.)
/
14.912211261555
2.8/1.2
14.5/35.7
CMA-ES
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitty
Rank
Opt. Gap
Baseline
87.6 87.6727/47.2" 2
36.22* (599
"6.1"""
166.""
49.0 "1.
72.2 12.2"
3,7/3.8
14.4/9.5
COMs
15.7
64.8 /63 12330
1198)"/58.88 822
18.8 18.8""
62. 22.922167.220 2aam
5.7/5.2
7.6/-9.7
COMs"
68.0 68.0"""/4488"
77.5
63. 6/36,77 7
/4655" g
10.00
7.3/7.8
7.1/-38.2
RoMA"
87.6 87.6%%//6677
86.22
66. 1133522
166.""
71
3.7/3.5
14,4/9.5
RoMA'
75.3 1050)
79.8 7773 99152))
64.6 500,5
118458 71.
44.6
22 12
5.0/4.8
14.1/8.D
ROMO
18.3""
86.2"*
66.5*
113"
45.71199997,71159
77.3*
4.2/4.3
15.7/-3.1
GAMBO
90.4 4339
86.2"0//3.4454
/6.241 g(48)
121 3
2,2/4.3
16.7/16.8
DyBAMO
89.8 773.69
/73,
63. 63.9 172.0a
197.7
50.6 50.6
3.3/1.8
17.5/55.2
CoSyNE
TFBind8
UTR
CLEMBL
Molecule
Superconductor
D D'Kitty
Rank
Opt. Gap 1
Baseline
61.7 /5.66
57. 37.3"*/11 12.7
63.6 6-.8221 m118
94,8 11222
37.00m
62.7 60.77
5.3/4.5
-0.61-52.1
COMe
70. 16,5515
73. 18.2 7002
63.5
11.4444
28.6 28.6%5%/0.00
63.8
5.7/5.7
1.1/-51.6
COMs'
66.9 (15,444) 4499)
56.2 2m 22,
63.3
63.35"0118.5500
11779//17. 3.0
28.0 28.07 334.3
70.6 70.6"11/16.6"
5.713.3
3.5/-40.4
RoMA"
61. 61.7000
57, 57.3% m22
63.6 (1
94.8 (0.gt
37.0""
62.7"
5.3/4.5
-0.6/-52.2
RoMA*
71..75
/1 1,94
62.125 ga.9
/10.244
38. 38.0
2.8/5.0
6.2/-52.0
ROMO
79.7 117.770
62.0 85.
64.1 (27.611
00.83"//7..3"
30.6"
/0.11"
10.
4.2/5.2
3.1/-50.8
GAMBO
(5.2
68.0 122 mmo
64.2" (15,
(7,
62.7" /0.g
3.7/6.3
5.IV-53.6
DynAMO
91.3 91.3" 118.
77.2 220
135,0
114 (22
67.5 47.50* 77.0
2.3/1.2
12.3/-20.7
BO-qEI
TFBind8
UTR
CLEMBL
Molecule
Superconductor
D'Kitty
Rank
Opt. Gap 1
Baseline
1999
84.5 /124
5.8/5.2
18.7/47.1
COMs
93.3
993,2
43.25%0/1922m3
86.2 /14788)
4.3/4.0
19.2/51.4
COMs*
84.5""
85.6""
65.1 mM 6"
121 1210"
47.3"*
6.0/5.7
17.9/40.3
RaMA-
82.9 86.3""1/74.1
121 12155//33,5
85.8 (159)
2.7/3.3
21.0/48.4
RoMA'
84.1 14.1111/4.2
66.6 6986660
121
5.2/6.3
18.3/32.9
ROMO
86.331
63 An. 194.89 gu
11855 1m
48.5 (196777
85.5 85.5 2 (160
5.0/6.0
19:2/34,9
GAMBO
66.8"
50.8 50.9
86.71.11177700
2.2/4.0
20.8/30.0
DynAMo
174.8'
86.2 mm /74.6"0.
67.0" 19944
121 193.5
53.5
3.0/1.3
20.7/74.2
BO-QUCB
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitty
Rank
Opt. Gap
Baseline
88, 9333
(6.22%.)744.3%
(9.4
193 9.5)
$1.38
84.5 84.5 194, 1a9
3.7/3.0
1/435
COMs
88.5 (a))
86.2 16.2mm,774 (74.
66.0" 1/992" mn
85.4 85.4 1500
4.5/4.5
19.0/45.5
COMs*
85.90.0/72.300 s
65.6 199
122 (
42.7 (900
5.2/5.7
18.6/51.3
RaMA-
86.9 86.9550)73.9
199
193777)
51_3(3%/198800
84.5001194,19 1am
3.8/3.0
19.2/43.6
RoMA-
84.6"*//88.2**
84.3" 1/63,33*
66.9 66.991998.. g
121
4.7/6.5
18.5/39.9
ROMO
95.2223177400
86.2"%/66.2222
64.7 gus)
118220/22,40.m
50.2
85.5
4,7/6.2
19.9/33,2
GAMBO
95.
k3.6"1m/22,ya
4.7/5.0
20.2/30.3
DynAMO
97,.3
66.7 6199..0)
86.9/=5/175
3.5/1.8
20.5/59.4
24

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Table A2. Additional Model-Based Optimization Quality Results. Each cell is the Median @ 128 oracle score (i.c., the median oracle score achieved by 128 sampled design candidates), reported as mcan confidence ineeval) across 10 random seeds, where higher is better. Bolded entries indicate overlapping 95% confidence intervals with the best performing algorithm (according to the mean) per optimizer. Bolded (resp., Underlined) Rank and Optimality Gap metrics indicate the best (resp.. second best) for given backbone optimizer.
Grad,
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitty
Rank
Opt. Gap
Dataset D
33.7
42.8
50.9
87.6
6.7
77.8
Baseline
58.6
59,3
85
36,0
65.1
4.3
10.5
COMs
53,
58.3
59.1
84,0%a
22,5
71.07
5.8
8.1
COMs
43,9
59.0
63.3
93
21.3/54)
89.9
4.0
11.8
RoMA"
$1.1
58.6
$9.1
85
36,0
65.1
5.0
9.3
RoMA'
48.2
77.
63.
84,5
38.2
88,5
3.2
16.8
ROMO
58,7033
37. 3
27.4
61.8
27.0%
46.0(117)
6,5
-6.8
GAMBO
63,8
75,3%
60.1
91.6
46.0
90.1
1.8
21.2
DynAMO
47.0
69.8
61.9/3.)
85,9m.
23.4
68.7(221
4.5
Adam
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitty
Rank
Opt. Gap 1
Baseline
54.738
60.4112.7
$9.2
87,9
37,4
56.81928
3.3
COMs
%4.8%
((177)
S9,2
90.8 11u
22.5
57.1
4.0
COMs
4
(0.5
63
89,3
23.3
56.4(133)
4.8
RoMA"
54.7%88
60.4(1227
59.2
87,9
(7.4
56.8(193)
3.3
RoMA'
(0.1
77.4
63.3
84.7
34.9
63,7
3.3
12.4
ROMO
63.3
50
26.1
30
5.7
-6.3
GAMBO
49.5
55,7022
57.7
37.4
87,8(43)
5.0
12.1
DynAMO
47.73m
69.0
(2
86.4
23.0%
65.6
4.7
9.1
CMA-ES
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitty
Rank
Opt. Gap 1
Baseline
50,727
71.7
63,3
83.9/1
37,9
59,3
3.2
11.2
COMs
45.0
68.0%k))
60,5
89.9/
18,8
89.8
5.3
7.1
COMs'
44.3%
62.0
59,7/43)
91.6"'3)
29.0
61.2
5.0
8.0
RoMA
50.7
71,7
63,
83
37,9
59.3
3.2
11.2
RoMA*
47.4
58.0
59.9/44)
91,6
31.6
60.4
4.5
8.2
ROMO
48,gan
7
60.0
84
22.8
61.6
3.5
8.7
GAMBO
44.2
72,7130
62
86. (0
21,4
54,9/he
5.5
7.1
DynAMO
4
65.gik9)
59,3(18)
99,0
22.5
60.6
4.8
CoSyNE
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitty
Rank J
Opt. Gap 1
Baseline
55,3
53.6
60.
87.4
36.6
59.3
4,3
8.9
COMs
51,7700s
70.
62.g
83. 98.
28.3'54
58.917.1
5.3
9.3
COMs
53,9244
41. 41.1
63.3
10733
2..4
61.2
4.2
RoMA
55,3
(3.6
87,4 87.4
36.6'44
59,3114.
4.3
RoMA"
67.7
(0.2
103
37.8
48.9/13.9
3.5
ROMO
(9.9
58.5
1 (1.4)
88.4
29.9/4.)
70.7
3.2
13.2
GAMBO
59,5
63,5
55,4
84.2
36.6
59.3144.)
4.5
DynAMO
m3.
(3.4
59,3
(9.0 (12.)
20,5(5.0
60.6
5.3
BO-gEI
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitty
Rank 1
Opt. Gap 1
Baseline
48.5
59,g(20)
63.3
86,700
28.7
72.4
4.3
10.0
COMs
50,9/mm
63,3
8m
7..
4.7
COMs
mm
66.0
63,
20)
66.3
4.5
8.0
RoMA
50.g/t
60.0
63,
m6.4
28,7118)
78,5
3.5
11.2
RoMA!
52
61.01L1)
63.3
93,3/5m
26.411.1
74,30.3
2.7
11.9
ROMO
49.
59.0
63,
86.8
24.6
73.8
4.7
9.6
GAMBO
46.4
(3.4
63.
86.3
28.9
79.1
3.5
11.3
DynAMO
51.5
65.6(AD
63,
86,7 70.0
23,5:24)
77.0
3.3
11.3
BO-qUCB
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitty
Rank J
Opt. Gap 1
Baseline
50.
62.1
63,
86.6
31,71.2)
2.7
11.5
COMs
fmm
61.0
63.3
86
19.8
74.2
4.5
9.4
COMs
mm
6l.6
63.3
87_5
20.1 (300)
54.201.9
4.5
5.8
RoMA"
5
62.1
m3
86,70u
31,7
74.
2.7
RoMA"
52,5000
63.3
91.4
29.5
65.8
3.2
10.6
ROMO
49,
58
63
86.5 mu.s
24.3
75.0
3.8
GAMBO
4
63
22.
73.
4.8
DynAMO
48.g
65.
63. mm
22,7
50.4
4.7
6.3
25

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
where dim(x) is the number of design dimensions and o is the kth dimension of design =f" Note that the LIC metric is only defined for designs sampled from continuous search space; compute the LIC metric for discrete optimization tasks, we use task-specific foundation models to embed discrete designs into a continuous latent space. For DNA design tasks (i.e., TFBind8 and UTR), we use the DNABERT-2 foundation model with 117M parameters /DD//ENERT---111 7M) from Zhou et al. (2024b) to embed candidate DNA sequences into a continu- ous latent space. Similarly for molecule design tasks (i.c., ChEMBL and Molecule), we use the ChemBERT model (jonghyun lee waaalllee/hheeeereeeee continuous latent space.
(zhihan1 996
from Zhang et al. (2022) to embed candidate molecules into a
We report MN and LIC metric scores in Supplementary Table A3. We find that compared with other MBO objective- modifying methods, DynAMO achieves the best Rank and Optimality Gap for 3 of the 6 optimizers evaluated (Grad., Adam, and BO-gED. For the remaining 3 optimizers evaluated, DynAMO is within the top 2 evaluated methods in terms of both average Rank and Optimality Gap for the LIC (L, coverage) metric. Altogether, our results support that DynAMO is competitive according to the MN and LIC diversity metrics in addition to the Pairwise Diversity metric reported in Table 1.
What is the best notion of diversity? In our work, we focus on the Pairwise Diversity metric in our main results (Table >>-however, this does not mean that this metric is the best for all applications. Rather, our focus on the Pairwise Diversity metric is determined by our problem motivation. Compared with the minimum novelty and L1 coverage diversity metrics, the definition of pairwise diversity best captures the notion of diversity that we are interested in-that is, capturing many possible 'modes of goodness' in optimizing the oracle reward function. We note that these modes of goodness may not necessarily be significantly 'novel' according to our task-specific distance metric, and so we treated the Minimum Novelty metric as only a secondary diversity objective for evaluating DynAMO. (Indeed, because DynAMO encourages a generative policy to match distribution of designs constructed from the offline dataset, DynAMO may not increase the minimum novelty of designs compared to those proposed by the comparable baseline optimizer.) Similarly, we find that the L Coverage metric is more sensitive to outlier designs when compared to the Pairwise Diversity, and therefore also treat it as secondary diversity evaluation metric for our experiments in Supplementary Table A3. Future work might explore other methods that focus on improving not only the Pairwise Diversity metric, but also other diversity metric(s), too.
D.3. Imposing Alternative f-Divergence Diversity Objectives via Mixed-Divergence Regularization
The MBO problem formulation proposed in (7) introduces a weighted KI KI.-divergence regularization of the original MBO optimization objective. However, alternative distribution matching objectives have been used in prior work (Agarwal et al., 2024; Gong et al., 2021; Ma et al., 2022), and one might hypothesize that we can similarly generalize (7) as
max J = (ro(x)) 1 Dy (lbb) HEll
(55)
s.L.
Bxnpg(a)(c (c (x)) - Exzng (z) = Wo
to any arbitrary f-divergence metric (-11)-) that measures the difference between two probability distributions Q, P over a =
space Q defined by D,(Q1IP)== S dP S text, we specialize to the KL-divergence where ful(u)= == u log u traditionally used in the imitation learning literature.
ar) for a convex univariate generator function f. For example in our main
However, we found that such naive approach does not generalize well to alternative f-divergences: recall that core weighted sum over distribution entropy and divergence (i.e.. Lemma 3.3) in order to admit an explicit, closed form solution for the dual function in Lemma 3.4. Such an approach is intractable using standard algebraic techniques. This is not ideal, as a number of prior works have proposed that alternative divergences- as the X-divergeecce defined by the generator f (w) = (u- - 1)21//)-00 /an better penalize -uito--distribution surrogate behavior and better quantify model uncertainty when compared to the KL--divegeecc (Tsybakov, 2008; Nishiyama & Sason, 2020; Ma et al., 2022: Wang et al., 2024).
contribution of our work was the ability to reformulate the optimization objective
In this section, show how to overcome this limitation and demonstrate how our theoretical and empirical results generalize to alternative f-divergence objectives for enforcing distribution matching in the sampling policy. Firstly, we look to recent work by Huang et al. (2024a) and others describing 'mixed f-regularization' defined by a mixed generator function (u) == (u) + u log u for some weighting scalar y E [1, +00), which admits a 'mixed f-divergence' given by
(y(())p:)) = (Q))( + D (Q
(56)
26

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Table A3. Additional Model-Based Optimization Diversity Results. Each cell is a value mn/11cs where mn is the Minimum Novelty
and 11c the Tı Coverage. Both metries are reported mean (95% confidence interval) across 10 random seeds, where higher is better. Bolded entries indicate overlapping 95% confidence intervals with the best performing algorithm (according to the mean) per optimizer. Bolded
(resp... Underlined) Rank and Optimality Gap (Opt. Gap) metries indicate the best (resp., second best) for a given backbone optimizer.
Grad.
TFBind8
UTR
CLEMBL
Molecule
Superconductor
D'Kitty
Rank 4
Opt. Gap
Dataset
0.0/0.42
0.0/0.31
0.0/1
0.0/0688
0.0/6.26
0.0/0.58
Baseline
$1,72.000 yo. 2.22
y0.42
95.00m770.007
102% 260
3.3/6.3
74.5/- 1.44
COMs
22.1
51.4 20
97.4 10.224
89.0 /0.40
94.1 94.1" 70.00 (.00)
100
20.00
3.3/7.7
75.2/-1.45
COMs
10.9 90.49
31. 31.7% 30.31
52. /1.11 19
13.7 /0.66 (i
99.6 "0. /0.37 87
"0 10. t0
6.2/2.7
51.4/-1.0
RoMA-
10.
$1.70% (0.21
20.26
79.5 79. 40.0
95.0" 10.000
102
110,00
2.8/5.8
74.5/-1.44
RoMA"
18.1 10 10.27777 27.227
40, 189 10.44 44
18. N0.41 ("oo)
95.3 m. mu
7. 7.1 /1 /1.28
0.2"" 40.55
5.8/3.5
29.9/-1.07
ROMO
16. 10. 33
32.9 12. 10.0000.30000 30m00
5. 5.0mm/1.31 (.11
23, 1 /,,6 (9
78. 50.,3 34.344
153" 13
6.0/2.7
$1.5//0.11
GAMBO
14.0 1
46. 16.772200022 20.24
96.8
20.25
76.8 N0.37 37.
83.8 83.8" R0o0
31, "0.000.00
6.0/5.7
58.3/-1.42
DynAMO
10.33 0044
00.52 /2.22
98.6 /1.46
85.8 (2
95.0 /6.477 (..4
107%h 15.85
2.2/1.3
76,7/1.25
Adam
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D' D'Kitty
Rank 4
Opt. Gap
Baseline
23,72 11 10000
51.1 51.19./00.22 /0.2.....mmm on
95. 95.5 20.23
79.3 yo..3
103 103* 24.24
3.5/5.8
74.5/-
COMs
23. m300
51.6 ( 10.20 20
95.5 y0..44
78.6 (0.66" 49
94,1 94.1" 7000 8.000
102 "02 00.03
3.3/6.8
74.2/-1.43
COMs*
13.0" 13.0""00.40 "0.44
31. 31.7% /1.73% 30.31
53.0 11.000//1111000
15.0 ofll y0.50m mn
99.7 10.7m=300.58 y0
99, 9th 10 10.8g
6.0/2.7
52.0V-0,98
RoMA"
23 y0. 12
51.1 /0.2 20
10.2 1
94.8 70.277
103 "0. t0.04
3.3/6.8
74.5/-1.39
RoMA"
" 28 28
40. (8240.466
18. 20,4 mae
95.3 /o, (mnzii
47.6 (1.7788
5.1 1, 70
6.0/3.7
37.6/-0.91
ROMO
13.6 20. 2y
32.9%.1l.. 300000000
21 1.55me
23 1o 7. 74m0
98. 100011 1.4...44000
99 10 oggnoo
6.0/3.7
48.3/-0.9
GAMBO
23.72110. mees))
$1..3440 /0, 20
95.0 =0.005
80,0 so
(L5s 84.8 84.8""
27. 27.39 10.0g
4.3/5.2
60.4/-1.35
DYnAMO
14.7 (00333000)
/0.255
98.7 /1,44
417.66 (.7)
108 7 76.91
3.0/1.2
74.7/1.50
CMA-ES
TFBind8
UTR
CbEMBL.
Molecule
Superconductor
D'Kitty
Rank J
Opt. Gap
Baseline
10.33
47.8 10 y0 4g
96.5 96.55*.//2.18'" u8
100 100" 13,77
3.8/4.2
72.3/0.36
COMs
13.6 13.6""700.344 mu
46.9 4 00.22mm
97.3 37.332/1.3 (1.1 m1
/2, 10 sn
100" /0.31 10
100 "00 (0.4
4.7/4.7
73.7/-0.69
COMs*
11.10 30.20000)
44.0 v0 43
98. 11.20
77.4 158m0
86. 16.11%00000600
39 "0."000.
6.3/7.0
59.3/-0.96
RoMA-
47.9 90 4gmmm
96.6 7/2
82 73.0
14.151
1337771
3.5/4.0
72.3/0.51
RoMA*
N0.66000
47. 77,7% 42.400000
10030//0,30000
3,2/5.7
73.5/-0,24
ROMO
16.2
(0.38
47. /7.0%*//0.44mmm
10.4
97.7%-/22.011 (1ze
/5. 13 13th41)
51.9 70.50
3.5/4.0
66.4//0.1 0.17
GAMBO
24.3
10.31
95.0 0/.0gl.5//..17mmm
72.5 2 m55)
85.6 25..3%13.377..
41.5 /3.12
5.5/4.3
62.0/0.27
DynAMO
12. ma4
48.0 /0.56
96.7 96.7 /182
81.8 /1.553/22.5440
112 /3.299110
4.0/2.2
74.3/0.62
CoSyNE
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D D'Kitty
Rank 4
Opt. Gap
Baseline
24.5 10
49,73.110 22
86.6 y0.77
/,, 8000
91.9 "1.92"/0.10"
3.2/5.3
74.1/-1.41
COMs
15.4"*"0.23"
/00.20000000 tn
90.0 30
86.2 (6.2""9 29
93.8 93.8m*
101 10.060
5.0/5.8
71.9/-1.99
COMs
20244
42,7 4 m0.32
34. 1,31
50.0 /1,39m
87, 87.8
1/129
30.8 8 20.25
7.5/1.5
43.0V-0.81
RoMA-
10.10
49,7
y0. 10.22
86.6 "2.00077
93.2"" "000
91. 91.92 "0
2.5/5.2
7..1/-1.41
RoMA'
17. 19.1 (0.22
49.39/ m0.4
59,5 (0.53
93,2 8000
65.5 25.
10.5
4.033.7
69.0V-1.30
ROMO
22.g 10.18 10.1
48. m0
75.8 11 1220
"2.2
81.85
10.06
5.0/4.5
59.4/-1.25
GAMBO
22.8 2 20....0...12. mm
50.8 /0..222 is
90.8 04 10
91. 91.90 13
8 109000
29.6 3 m000
4.5/6.3
62.00- 1.42
DynAMO
17.8 50,21 mI
48.4 18mmm
96.7
80.2 n0.2n2 33,4000
94.5 /1155
112
4.0/3.3
75.0/-0.90
BO-qEI
TFBind8
UTR
ChEMBL
Molecule 85.4 15 22.55
Superconductor
D'Kitty
Rank J
Opt. Gap
Baseline
20 (966
$1.55*."00
/0.5550 m0
17.6 (7.77
94.6 11711/844440 84:001
106" t 76.6 1933
3.3/5.0
76.2/1.70
COMs
=1.8
00.56
97. 07.3%%\/2_444mm
92.5 2111176611 (1
105 175.1 19
4.0/3.0
75.6/1.8
COMs
12.7 70700004 "a44
43.2
"0.57 00577mm
8/2.5% mis
mm
85, "1.55""
35.5 5 "1 mon)
7.5/3.8
56.6/0.90
RoMA-
21.6" 40.40
$1.7
97.7 42 (Mo0
".. m0.5
94.6 (884 (001)
105 105 4645
3.2/5.2
76.1/1.42
RoMA*
45.0%
98.5
88.2 (M6.2%6111999 91.9
1 2/772
3.7/5.0
76.0/1.67
ROMO
47.5 47.50
00,55m.00
87.7 87.7 22.50
87.91111 /2.48
86.6
18.09)
30.8 25 12.1221411 14
5.5/5.3
59,3/0.98
GAMBO
nn N0. 40
51.8 yo.......stt...... mo.5mmm
97.8 /2.33
84.9 (an
85. 85.1 11911744::0)000
14. 4.33 1/1 2gmme
5.7/6.3
58.2/0.82
DynAMO
21.0 "0 m2
0.556mm
97.4 0 12.77188)
85.2 t9 (2.54
126 u m.92
3.0/2.2
79.4/2.02
BO-qUCB
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitty
Rank 1
Opt. Gap
Baseline
mey
97.9 12 m0
85.3 1/2.551mmm
93. *1y7777 78
98. 8 1/6 th
3.5/4.7
74.8/1.77
COMs
$1.7 /0.7,6 (0.5mmm
97. 177,4%//244000
85.4
92.9
99. /6 g4
3.8/3.7
74.7/1.80
COMs*
51.6 (0.55m000 43.5
84.1 /2.52
84.6 1/7,51 (889
39.4 t monn
7.5/3.7
57.2/0.90
RoMA-
2 agines)
97. 01/8377773
(5.5 (m54
93. 28.888777788 77,777778888
98.8 ( 64m00
3.5/4.8
74.9/1.77
RoMA*
13. g3. meee mees
98.8 " 11.818550 mie
88.5 18.5 /2.01 m0.0
94. 94.1 /11/666
112 2/2
3.3/5.2
75.4/1.69
ROMO
16.0 "0000000000000000000000 m00
47.7
88.1 22.55 m00
/2. m2.49
85, 85.4 48-00
20.9 2 magas
5.7/5.3
58,2/0.89
GAMBO
21.9 21.9a mo
/1.7
97.5m 32m
85,2
81.9 11.901977,7777700
25 25 34gion
4.7/6.0
60.7/0.82
DynAMO
20.4"550.444 m0
/1.7
97.1 1 12,77
85.3" /2.54
94.7
17,0
3.7/2.3
76.6/2.00
27

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
for probability distributions Q, P.! Given mixed f-divergence, can define modified MBO objective in (5):
Jy(T:)) = EE- [r]]r](=]] - & =D - [-]]ro(x)] _ DK (x1l)) D
87
(57)
= J(() 87 1)(0))155)
where ro is again the forward surrogate model, (TT) is as in (5), (x) is the r-weighted probability distribution as in Definition 3.2, and q (x) is the sampled distribution over designs admitted by the realized sampling policy т. Given this expression for the modified MBO objective, it easy to rewrite J (m:?) similar to Lemma 3.3 in the main text:
Lemma D.1 (Generalized Entropy-Divergenee Formulation for Mixed fakedd-Diiie f-Divergence). Define J)(m:)) as in (57). An equivalent representation of (=:))) is
m = HH(0 -1) (obbb (e
(58)
where H(-) as the Shannon entropy and D-(-Il-) as the f f-divergence.
Proof. The proof is trivial using Lemma 3.3:
Jy(x)= =- En-1ro - - (D = J(T) - 81 D00
(59)
~ J-- ((T) (0 (1bb)
~ -H((a") (1 + DKL (pbb) BrD,(9"11Pb))
up to constant independent of the policy æ.
We now consider its derivative optimization problem constrained by source critic feedback analogous to (7):
J)(T:)) Eg [re(x)] 3 - D (pbb:))
#Ell
(60)
s.t. ExnpB(e) (aaeceec(((" (3r) Exng (x) < Wo
where (x:) is again an adversarial source critic and Wo is some nonnegative constant. We can show that (60) admits an explicit dual function which can be used to tractably solve this optimization problem.
Lemma D.2 (Explicit Dual Function of (60)). Consider the primal problem
max J = Eg [re(x)] 2 D
wEll
(61)
s.t.
Exppg(r)l (z) Exng (x) (x)] Wo
for some convex function f where & domíJ). The Lagrangian dual function g(A) is bounded below by the function ge( (X) given by
gc(A) = 71) Wo) - Eppee de' (x) -Bppp (* (x))
(62)
where (+) is the Fenchel conjugate of f.
Proof. Recall that the generator function of the mixed f-divergence penalty is given by (u) ff(()) log for some
weighting scalar y E [1, +00). Define fK)(u) log u. From (10), the dual function g()) RR+ - R of the primal
It is trivial to verify both that (u) is convex and that & dom(fx) given function fu) that also satisfies both these conditions.
28

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
problem is given by
((A)
min lell
(1 + BBpppp fKL
9²'
+ pb
9
Egg log (a") + (3( + 7)2 (1p3 [x)c"(x) Wo)
PDD
(PD)
min sell
(1 + Bys fKt.
9"
+ 3yBp5
9""
- -Ber5 lkl.
9"
+ Egg log
PDD
PiD
PDD
+ (1 + 7)
(Epg (x)) - Egg (xx) - Wo)
9"'
9"'
-min =EIl
ppp fKI.
+ Byp
Egp log pb + +1(++)) (Bp5 (x)
Wo)
PiD
PD
(63)
where define B(1 + 7)d E + as the Lagrangian multiplier associated with the constraint in (60) (recall that R. is closed under multiplication). We rearrange terms to rewrite g()) as
BBp [
q-
)
[- (-(xoto) (x)) g"
( +
9"
g(A)
min Tell
Ac (z)
fKI.
3y1Bp3
PD
PD
p
PDD
(64)
Eg log pb + B(1 + 7)ABp, e² (a) 3(1 + 7)
The sum of function minima is lower bound the minima of the sum:
(xc(())
9"
--5
9"'
(Ao(()) g"
9"
g(A) 2 pp min
+ fxL
+ BTppp min
+ f
#ell
PDD
PD
Tell
PD
DD
max = log + +min #Ell
13(1 + 7) >)DBpp e² (xx
-B(1+ ?)
TEll
(65)
-Axcc (x) (x)
9"'
9"
9
q"
p min sell
fKL.
ByBB
min
+ f
PDD
pD
Hell
PDD
PD
+ 31 + 7) e¹ (x) 311 + 7WW
ignoring the term max [ log lo! that is constant with respect to A. We then perform the same tactic of minimizing over the superset R,,, 2 {a 3x E s.t. qº (z)/p² = 2} as in Appendix A:
s(A) 2 pppp min |- (Ac (x) =) + fKL (=)] + ByEpp min |- (Ac (x) 2) + [(=)] + 3(1 +7)A(EB, cx(x) Wo) =ER+
=E8+
=B [Bpp fka (Ae* (x)) BBp fr (Ac (z)) + () c² cx)) -W)]
(66)
where f*(() is the Fenchel conjugate of a convex function f(). The Fenchel conjugate of fki.()) = u log u is fki(v) (v) = e--1 (Borwein Lewis, 2006), so
((A) 2 3 [ (0) - -ylEpp S (Ac + (1+ (1++)) (x) Wo)]
(67)
Define the right hand side of this inequality as the function gr(A) and the result is immediate.
Corollary D.3 (Explicit Dual Function of (60) Using Mixed -iveeenngenc,) As an example, we can consider the mixed Dxa (xx(Q|1 (P) as used Huang el (2024a). The x2 -ddivergence fx²(u (u 1)2/2. and its Fenchel conjugate is (z)())= u (v²/2) (2) Lemma C.4. Directly applying Lemma D.2, our lower bound the our dual function is
x2 d-vverreence defined by (Q] =>D>> generator function function
g(A) 2 ge(A)= = 1 (o 1 -Bpp (Eac(o)) +)0())))>>0()))))))))))))))))))) + (1 (1+))((Bgg((())
Wo)
(68)
To experimentally evaluate the utility of distribution matching using a mixed x2 K--Deveereee,,, we substitute the (-1.) divergence with the mixed D-iivergeece (),((11::)) (setting 7 = 1.0 for experimental evaluation) and its associated
29

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
dual function bound from (68) into Algorithm 1. Practically, we find that this only requires updating the dual function per Corollary D.3 and the Lagrangian of (60) given by
_ 1>D (lbb) + (x((1"1165) + 3(1 + 7) - - cx(x(())) Wo]
[(x; X) = - -roo)(ro())
(69)
in Algorithm 1.
Experimental Results. We compare DynAMO implemented with mixed x² divergence penalty (with 7 1.0) against our original DynAMO implementation (i.e., ? = 0) in Supplementary Tables A4-A5. Empirically, we find that using the mixed x--divergence penalty offers limited utility compared with KL-divergence alone: the latter is non-inferior to the former according to both the Rank and Optimality Gap metrics for all 6 optimizers assessed according to the Best@ 128 oracle score. Furthermore, DynAMO outperforms DynAMO with mixed x² divieegeee according to the Rank and Optimality Gap metrics for 5 out of the optimizers assessed according to the Pairwise Diversity metric. Based on our qualitative analysis, we hypothesize that the over-conservatism often attributed 2022; Huang et al., 2024a; Wang et al., 2024) may adversely affect the generative policy's ability to sufficiently explore the design space when compared to using KL-divergence base distribution matching alone. Further work is needed to tune the relative mixing parameter 7 and/or explore how other alternative f-divergence metrics may be used with DynAMO.
-=-divergence-bar -baed penalties in related literature (Ma et al.,
D.4. Theoretical Guarantees for DynAMO
In this section, we seek to place an upper bound on the difference between true diversity objective
(T) == (s - DDD (x)llp"(x))
(70)
T
realized by the final generative policy # E II learned by DynAMO (denoted as x² in the main text), and the true diversity- penalized objective realized by the true optimal policy x² := arg max J (T). Note that this objective J (α)) is not equivalent to the offline MBO objective ((T) introduced in (5); importantly, the objective (Tx) is a function of the true, hidden oracle reward r(z) as opposed to the forward surrogate model (z)).. Furthermore, the KL-divergence penalty is computed with respect to the true w-eeighted probability distribution p" (Ir), as opposed to its empirical estimate computed from the offline dataset D as in Definition 3.2. In principle, (70) captures the true trade-off between diversity and quality of designs that we hope achieve by the theoretically optimal zero-regret generative policy x' that maximizes (70) over II.
Our main result is in Theorem D.9 below, although we first step through the relevant assumptions and intermediate results necessary to arrive at our bound on (70). Firstly, we assume the following:
Assumption D.4 (Surrogate Model Error Bound). There exists a finite =0 E IR., such that
(a) (x)p'(")(((a)) re(x))² < e0/4
(71)
for any choice in 2 0, where p" (ar) is the true weightted probability distribution over X.
Assumption D.5 (Policy Realizability). Both the true optimal sampling policy x* according to (70) and optimal sampling policy म according to (7) are contained in the (finite) policy class IT. Assumption D.6 (Bounded Importance Weights). Define the importance weight w(() as the ratio between probability distributions qº (x:) and p(z). There exists finite M E R. such that for all possible permutations of { x*) and p(x) E (w" (x), }x)). we have w(x) := q"' (x)/((x) S M for all x E X.
Remark. This assumption is mild assuming that (1) D is large enough such that (zx z (x): and (2) the policy π has been learned with sufficiently large 3 according to Algorithm or a similar distribution matching objective, such that the distribution of designs learned by the generative policy qd (x) well-approximates the expert distribution (x)). Because the optimal policy x² should also well-approximate p" (1) (and therefore pb(2)) (x) by assumption), the assumption that such a finite M exists is reasonable.
Under these assumptions, we first place a bound on the error of the forward surrogate model over the distribution of generated designs from the optimal policies according to both the offline objective J(z) and true objective J'()): Lemma D.7 (Bounded Prediction Error). Assume there exists an M RR finite satisfying Assumption D.6. Then with probability at least 8 we have (for any 8 > and both क = n"" and π 7)
2 log (2 (1 80 + M
(x) (n) |m(x) - ro(2)) =
(72)
2
n
30

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Table A4. Quality of Design Candidates Using Mixed x² --iierrgence DynAMO. Using Corollary D.3 and (60), we show that it possible to extend DynAMO to leverage mixed x2 -diveeggeece that equally weights both x² -dviveegee and KL-divergence to penalize the original MBO objective. We evaluate this specialized implementation of DynAMO against baseline DynAMO and vanilla optimization methods, and report the @est@ 128 (resp., Median @ 128) oracle score achieved by the 128 evaluated designs in the top (resp., bottom) table. Metrics are reported mean" 995% confidence mecvaa) across 10 random seeds, where higher is better. Bolded entries indicate average scores with overlapping 95% confidence interval to the best performing method. Bolded (resp.. Underlined) Rank and Optimality Gap (Opt. Gap) metrics indicate the best (resp., second hest) for given backhone optimizer.
Best@ 128
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitty
Rank
Opt. Gap
Dataset D
43.9
59.4
60.5
88.9
40.0
88.4
Grad.
90.0
80,9/12.)
60.2
88.8(40)
36.0(68)
65.6(14.)
2.8
6.8
DynAMO-Grade
90.3(477
86.2
64.4(25)
91,2(00
44,2(7.8)
89.8
1.2
14.2
Mixed x² DynnAOO-GGr
59.3
86.2
64,4(24)
120(14)
42.0/50)
83.6
2.0
12.5
Adam
(2.9
69.7(10.5)
62,9(19)
(2.3(89)
37.8(63)
58.4
2.7
0.5
DynAMO-Adam
95.2
86.2
65.2
11,2:00)
45.5
84.9/120)
1.3
14.5
Mixed x² DynAMO-Adam
59,3
(6.0
64.4(2%)
120
42.0/54)
83.6
2.0
12.5
BO-qEI
87.3(58)
86.2
65,4(1.)
116
(3.1
84.4
2.8
18.7
DynAMO-BO-gu
91,9(4.)
86,2
67.0
121 121 (0.00
53,5
85.5
1.5
20.7
Mixed x² DynAMO-BO-qEI
92.2
86.3
62,8
123
51.7/44)
85.1 (1.5)
1.7
20.7
BO-qUCB
88.1 (5.3)
86.2
66.4
121
$1.3(3.6
84.5
2.2
19.4
DynAMO-BO-QUCB
95.1 (1.9
86.2
66,711.5)
121 (9.0)
48,1
86,9(45)
1.7
20.5
Mixed x DynAMO-BO-qUCB
85,704)
86.3
66.3
121
(1.5(4.)
83,8(1.1)
2.0
19.0
CMA-ES
87.6(8.3
86.2
66,1
106
49,0
72.2
2.0
14.4
DynAMO-CMA-FS
(9.)(4)
85.7158)
(3.9(09)
117(6.7
(4.6
78.5
1.7
17.5
Mixed x2 DynAMO-CMA-ES
(4
84,5(2.
(L.3
113(48)
45,0/49)
81.8(4.)
2.3
15.4
CoSyNE
61.7 (1.7
57.3
63.6
(7,0(4.)
62.70.3)
2.8
-0.6
DynAMO-CoSyNE
91,3'44
(1.2(1
63.9(99)
114
40.6 (8,)
67.5(1.4)
1.5
12.3
Mixed x² DynAMO-CoSyNE
94,3223)
78.3
63.1
100
37.4
82,3(4.1
1.7
12.4
Median 128
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitly
Rank
Opt. Gap
Dataset D
33.7
42.8
50,9
87.6
6.7
77.8
Grad.
(6.1
58.6
59,3(86
85,3
36.0
64.1
2.0
10.5
DynAMO-Grad.
42.0
69.8
61,9(22)
23,4
68,7
1.5
9.5
Mixed x DynAMO-Grad.
46.
66.9
58,3
86,6
20.6
64.4
2.5
7.1
Adam
54.7
(1.7
59,2(86)
87,9(10.)
(7,4
56.8 (6.8
1.8
9.5
DynAMO-Adam
47.7
69.0/5.
(2.4(1.)
(6,4
23.0(60)
65.6
1.7
9.1
Mixed x² DynAMO-Adam
45.2
66
58.3(6.5)
86.6
2..6
64.4
2.5
7.1
BO-qEI
48.5(1.5)
59.9(2.m
63,3(t0)
86,7(06)
28.7(138
(2.4
1.8
10.0
DynAMO-BO-qEE
m1.5
63.
(3.3
86,7
23.5(24)
77.0
1.7
11.3
Mixed x² DynAMO-BO-du
44.8" (144)
66.1 (3.0
(3.3
86.4
27.7(35)
73.9
2.3
10.4
BO-qUCB
50.3
(3.4)
63.3(00)
86.6
31.7(12)
74.4
1.5
11.5
DynAMO-BO-QUCB
48.8
65.9
(3,3
86,5
22.7
50.4 (446)
2.0
6.3
Mixed x DynAMO-BO-QUCB
68.2
(3,
86,5
(0.
74.5
2.0
9.6
CMA-ES
10.7(27)
71.7
(3,3(00)
83,9(1
(7
59,3
1.5
11.2
DynAMO-CMA-ES
(2.4
(%.
59,3
(9,0
22,5(5.1)
60.6
2.2
8.8
Mixed x DynAMO-CMA-ES
48,5(34)
7..0
(3.2
87,0
(9,4
43,7
2.3
5.4
CoSyNE
55,3%80
53.6(10.2)
60,8(3)
87.4
36,6(44)
59.3
2.3
8.9
DynAMO-CaSyNE
(3.8
(3.4
59,3(38)
99.0
20.5
65.6
2.3
9.5
Mixed x² DynAMO-CoSyNE
"9.9
(5.4
(0,9(31)
(143)
23.4(4.5)
66.6
1.3
11.0
31

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Table A.5. Diversity of Design Candidates Using Mixed -Divergence DynAMO. Using Corollary D.3 and (60), we show that it is possible to extend DynAMO to leverage mixed x² -ivvegeene that equally weights both x -ivvergeee and KL-divergence to penalize the original MBO objective. We evaluate this specialized implementation of DynAMO against bascline DynAMO and vanilla optimization methods, and report the pairwise diversity (resp., minimum novelty and L2 coverage) oracle score achieved by the 128 evaluated designs in the top (resp.. middle and bottom) table. Metries are reported mean 195% confidence interval) across 10 random seeds, where higher is better. Bolded entries indicate average scores with an overlapping 95% confidence interval to the best performing method. Bolded (resp.. Underlined) Rank and Optimality Gap (Opt. Cap) metrics indicate the best (resp.. second best) for a given backbone optimizer.
Pairwise Diversity 128
TFBind8
UTR
CbEMBL
Molecule
Supercoaductor
D'Kitty
Rank
Opt. Gap 1
Dutasct P
65.9
57.3
60.0
36.7
66.0
85.7
Grad.
12,5/
7.8%8
7,97
24. 1133
0.00
0.0%%
3.0
-53.2
DynAMO-Giad
66,9/m
68.2
77.2
43.0
129'55
104
1.3
27.8
Mixad x
DynAMO-Grad
16.802
72.6
47.0/3/
91.0/1
182/450
74.220
1.7
18.2
Adam
12.0
11.0 20
4.8%
16. the
6,4
6,2
3.0
-524
DynAMO-Adum
$4.8%
72.3
84.8%
89,922
15gins
126
1.7
35.7
Mixed x" DynAMO-Adam
16.8
99,2
91.0'
182
74.2
1.3
27.4
80-jEl
73, 7M,
g
99.3
93,0
1999aa)
124
2.5
47.1
DynAMO-BO.qEI
74,8
74.6
93,5
1982
2771955
1.3
74.2
Mixel x DynAMO-BO-git
73.6
74.3
99,4
93,9/24
167
29/ 17
2.2
27.5
BO-
73,9
74.3%A
99,4
93.6
198
2.2
43.5
DynAMO-BO-
74_
74.4"
99.3"1
211 211
17g
1.7
59.4
Mixed x² DynAMO-BO-qUCC -UCBB
7374
74.3
99.5
93,71821
177
2.2
29.0
CMA-ES
47.2
93,5
66
164
2.5
9.5
DynAMO-CMA-BS
73.6
73.1
72,0%
94,0'*
97,8 gib
292
1.3
55.2
Mixed x DynAMO-CMA ES
52.9
21.5
67.72km
69,71124
154
86,
2.2
18.6
No
5,6%
12,77A
28. 28
12,2
0.0%
3.0
-52.1
NynAMO-CUSyNE
111
20,
35.0
22.
74,4
77.0239
1.7
-20.7
Mixed x
DynAMO-CaSyNNE
22
34.
34,1 22
704
29.4
1.3
-17.5
Minimam Novelty & 128
TFBind8
UTR
CLEMBL
Molecule
Superconductor
D Kittty
Rank
Opt. Gap
Datasct 2>
0.0
0.0
Grad.
21
$1,72M
97.4
79.5
95.g/h.)
2.3
74.5
DynAMO-Grad
21
$2,20.
98.6 (1155)
85
95.0/a4
107
1.3
76.7
Mixed x DynAMO--Gaad
14.6
51,914
9972
25.
85. ()
34 34.
2.3
61.7
Adam
23.7
95,5th
79.3
94.8
103
2.0
24.5
DynAMO-Adam
14.7'
46.2 2
98.7
94.9
To
1.7
74.7
Mixed x" DyuAMO-Adam
20.4
$1.914
87
85,2
85
34
2.3
60.7
BD-gEI
21.8
$1_5
97.6 9..6
1.8
26.2
DynAMO-BO-jh
21.0
$1.9/12
97.4 97.4
126
2.01
79.4
Mixed x DyRAMO-BO-qEi
(4.6
$1.9
80,
16.6
2.2
BO-qUCB
$1.701
97.9 97.9
85.3
93,8
98.8 9
1.8
24.8
DynAMO BO-
51,700
97.1
85.
94.7723)
409
2.0
76.6
Mixed x DynAMO-BO
19.8
97.4
85.6"'
79.8"'
14.9%%
2.2
58.2
CMA-ES
16.5
96,5
100
2.3
72.3
DynAMO-CMA-ES
12,gmm
18.0
96.7
91,5(07
11277
2.0
74,3
Mixed x DynAMO-CMA-ES
23.0
$1.8
98.0"**
83
87.1")
48,6 (
1.7
65.4
No
24,5
49,7A1
98,5
91 gf2n
1.7
DynAMO-CoSyNN
17.855
48.42x
96.7135
2,5
75.0
Mixed x² DynAMO-CCSyyNE
23,5
$2.5
99.9
83.922
30.3
2.0
61.8
L Coverage 128
TFBind8
UTR
CbEMBL
Molecule
Supercoaductor
D'Kitty
Rank
Opt. Gap
Datasct P
0.12
0.31
1.12
0.68
6.26
0.58
Grad.
0.16
0,209235
0:2g
0,42
0.0ga000
0.00
3.0
-1.44
DynAMO-GGad
0,36
0.52
2.49
6.4711..0
s.ggt
1.3
1.25
Mixed x² DynAMO-Grad.
0,16
0,54
0,87
2.20
6.671150
1.2
0.40
Adam
0,11
0,22
0,23
0,4
0,27
0.24
3.0
-1.35
DynAMO-Adam
0.33
0.5g
7.06
6,91
1.0
1.50
Mixed x" DynAMO-Adam
0,16
0,54
0,47
6,6711.00
2.0
0.33
80-qhl
2. 3p000
2.11
7 7.84
6,61
2.5
1.70
DynAMO-BO-qEI
0.42
0.56
2.47000
2.5g
7,873300
7.920s0
1.2
2.02
Mixed x2 DynAMO-BO-gtt
0.3y/
2. 1.
2.56
6.11 6.11
2.5
0.62
BO-
0.40
0,5
2
2.52
7.78
6.6
2.5
1.77
DynAMO-BO-qI0 BUC
0.40
0
2.47
2.5g
*m
1.2
2,00
Mixed x SyyAMO-BO-qUCC
0,39
0,56
2.5
5.96
L.gian
2.3
0.59
CMA-ES
0,33
2.1
1822
3.26
3,77
2.3
0.36
DynAMO-CMA-ES
2.34
4.75
3.2
1.7
0.62
Mixed& x ES
0.3
0,52
1997
5.5g
4.03
2.0
0.71
Co
0.10
0.220k30
0.39min
0,27
0.10
0. 0.10
2.8
-1.41
DynAMO-Casy
0l64
0.4
0.9g
2.01
-0.9
Mixed 2 DynAMO-CoSyyNE
0.20
0.371230
0.7gm44
0,53
3,81229
1.10
1.2
-0.48
32

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
where n [D] the number of datums in the offline dataset D.
Proof. Under Assumption D.4, Jensen's inequality gives
2 =:
Ex-p (a) (r(x) - rr(()) & ( &
cD
(73)
4 2
Furthermore, Assumption D.6 and Cortes al. (2010) yield
Re (w) (r(x) ro(2)) Ben (x) [r(x) - -ro(z)]
(x) Pb(x)
r(0) - ro(x)) -m(0
By Bxnp (2) |r(x) - (x) R (x)
(74)
M (2
with probability at least 6. In the offline setting (as in our work) and assuming that the forward surrogate model ro(x) has been well-trained according to (2) or a similar learning paradigm (e.g., see Trabucco al. (2021); Yu al. (2021)), we reasonably assume that eß & Exg-ag [r(x) r=))) We therefore have upper bound on the prediction error of the forward surrogate model over the distribution q (zr) over generated designs:
(x) (w) [r(x:) - ro(()) & E. Exwpp (r(x) - ro(2) + My
2 log 2
(1/00)
60
MV
logg /
(75)
1
2
7
with probability at least - 8.
Under Assumption D.6, also place upper bound the true and realized KL-divergence penalties:
Lemma D.8 (Bounded KL-Divergence). Assume there exists an M E R2 finite satisfying Assumption D.6. Then with probability at least
8 we have (for any 8 >> O and both = n²" and π = f))
(x (3r)lle (x)) (x
s M (ol10((
(76)
Proof. According to the definition of M and the definition of the KL-divergence from Definition C.I,
9"
9" (x)
DRL (g (x) (x)) = (x)
log
< M log M
(77)
p²
(x)
p' (x))
From Hoeffding's inequality (Hoeffding, 1963),
2ea
PP ((DDxlg" (x)) - DxL (x))1 E) < |I
exp
(78)
M log M
for any e > 0. We can choose to define == (M log M) log( IT /8)/2 such that
(M log M)
log (1100)
M M [1
(g (z))l" Dra Deale
M Moll(((UI/0)) (79)
V2
V2
with probability at least 8.
We are now ready to prove our main result:
Theorem D.9 (Bounded Diversity-Penalizee Objective (*))).. Assume that there exists an M R4, finite satisfying Assumption D.6. Then with probability at least 1 5, we have (for any 8 0)
2
8 7
8|11|
2M
log
(80)
Vn
where n [D] the size of the offline dataset DD
33

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Basce Optimization
Proof. Firstly, we combine Lemmas D.7 and D.8 using the triangle inequality bound the difference between the true reward J (ft) and the offline reward J(t), where á IT maximizes (TT) defined in (5).
J') (sroo()))) 9 (q (wllo))) (1)(ll))))
(m) ((Br)))(((m(ttalla
- (Eor
DRa (a (x)llo
8 (q (x)lle" Dralu (x)ll==x))
s (2) ro(())
2 12lo(811T]//)
B Vlg((1
(81)
+ M
n
ED
2
8
8
-
+
log
2
Vn
with probability 1 (6/2). Because मे == argmax FEN (TT) we must have J(x*) S ((t). Substituting this into the left hand side of (81) gives
J(x))-
cD
+M
(
log vn = V
2
38)
8|T
(82)
2
Separately, we have (with probability 1 1 (8/2))
8 T DKL (e))lo(()))
EE [re(x)] - DKi (0" (+))lb(()))
[²(x*) ((x')====
= E (x)x((()))((())))
S B (a) (r)) rr(()) + (K (x)llp - DKL (x)11p((x))
8
2 2log(8|111)
+ M B
(83)
+ MM
log( 4111/8)
2
E0
2
8
8|11
+ MM
log
Vn
T
8
following the derivation in (81) except for x² (as opposed to fi that maximizes (TT) (as opposed to J(T)). Summing (82) and (83) gives
2
8
8|T||
so 2M
log
(84)
with probability
Note that we only prove Theorem D.9 in the unconstrained optimization setting: in principle, a tighter bound could exist in the adversarially constrained formulation introduced in (7), as a bound on the 1-Wasserstein distance between d (x) and P5 (x) will almost surely place a favorably tighter bound on the forward surrogate model prediction error than Lemma D.7.
D.5. Comparison with Offline Model-Free Optimization Methods
In our main experimental results reported in Section 5, we focus on comparing DynAMO against other model-based optimization (MBO) methods-the that is, optimization methods that explicitly (1) learn a proxy forward surrogate model ro(x) for the oracle reward function from the offline dataset; and (2) optimize against ra and rank final candidate designs according to a scoring metric involving rg. Alternatively, recent work have also proposed methods that instead do not learn forward surrogate model ro(x); we refer to such methods model-free algorithms.
Survey of Existing Model-Free and Additional Hodel-Based Methods. Mashkaria et al. (2023) introduce BONET (i.c., Black-box Optimization Networks), which learns an auto-regressive model on synthetically constructed optimization trajectories that simulate runs of implicit black-bor box optimization experiments. The auto-regressive model is trained to learn a rollout of monotonic transitions from low- to high- scoring design candidates using the offline dataset. Nguyen et al. (2023) propose ExPT (i.e., Experiment Pretrained Transformers) as agno-agi method of pre-training a transformer
34

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
foundation model learn an inverse modeling of designs from input reward scores and associated contexts. DDOM (i.e.. Denoising Diffusion Optimization Models) leams generative diffusion model conditioned on the oracle reward values in the offline dataset (Krishnamoorthy et al., 2023). Similarly, GTG (i.e., Guided Trajectory Generation) trains diffusion model to learn from symthetically constructed optimization trajectories conditioned on final scores. MINs (i.e., Model Inversion Networks) from Kumar& & Levine (2019) learn and optimize against an inverse mapping from reward scores to candidate designs. Tri-Mentoring and ICT (i.c., Importance-aware Co-Teaching) co-learn an ensemble of multiple surrogate models (Chen et al., 2023a; Yuan et al., 2023). Separately, PGS (i.e., Policy-Guided Search) from Chemingui et al. (2024) learns a policy to optimize against a surrogate model (although only limit their method to first-order optimization algorithms), and Match-Opt from Hoang et al. (2024) proposes black-box gradient matching algorithm to learn better forward surrogate models. Finally, RGD (i.e., Robust-Guided Diffusion) uses forward surrogate model to guide the generative sampling process from a diffusion model (Chen et al., 2024). Other model-free optimization methods have been proposed specifically for the biological sequence design problems (Kim et al., 2023; Chen et al., 2023b; Jain et al., 2022); we exclude these from our analysis and instead focus on tank-agnostic optimization algorithms. We also exclude Design Editing for offline Model-based Optimization (DEMO) from Yuan et al. (2024), Noise-intensified Telescoping density-Ratio Estimation (NTRE) from Yu et al. (2024), and Ranking Models (RaM) from Tan al. (2024) from our analysis since there are no presently available open-source implementations.
Experimental Results. We compare representative implementations of DynAMO (i.e., DynAMO with Gradient Ascent (DynAMO-Gradd), Bayesian optimization with Upper Confidence Bound acquisition function (DynAMO--O-qUCCB) and Covariance Matrix Adaptation Evolution Strategy (DynAMO-CMA-ES)) against other model-based optimization methods using the respective backbone optimizer described by the original authors (i.e., RoMA from Yu et al. (2021) using Adam Ascent, COMs from Trabucco et al. (2021) using Gradient Ascent, ROMO from Chen et al. (2023c) using Gradient Ascent, GAMBO from Yao et al. (2024) using BO-qEI) against model-free optimization methods in Supplementary Tables A6-A8. We find that DynAMMOaugment
optimizers can be competitive in proposing high-quality designs-in particular, -UUBB achieves both the second best Rank and Optimality Gap across all six tasks according to the @est@ 128 oracle score metric. However, the improvement in diversity of designs using DynAMO is significant: DynAMO-BO-qUCO achieves the best Rank and Optimality gap according to both the Pairwise Diversity and L1 Coverage metrics, and DynAMO-Gras achieves the best Rank and Optimality gap according to the Minimum Novelty metric. Furthermore, DynAMO-BO-qUCB attains the best mean Pairwise Diversity score compared to the model-free optimization methods evaluated in 5 out of the 6 tasks assessed. Altogether, our results suggest that DynAMO is a promising technique to propose a diverse of high-quality designs compared with existing state-of-the-art offline optimization methods.
D.6. Empirical Computational Cost Analysis
To evaluate the empirical cost associated with running DynAMOO, we report both the runtime and maximum GPU utilization of optimization methods both with and without DynAMO augmentation in Supplementary Table A9. Briefly, all experimental results reported in Supplementary Table A9 were conducted on one internal cluster with 8 NVIDIA RTX A6000 GPUs, and one 24-core Intel Xeon howweereeer, only single GPU was made available for each program instance reported in our experiments. Across all six optimization methods evaluated, DynAMO increases the runtime (resp., maximum GPU usage) of the optimization method (averaged over all six tasks) by a mean of 181.7% (resp. 1.9%). While our experiments reveal that DynAMO is indeed associated with additional computational costs, they also show that DynAMO is empirically tractable to run even using a single GPU. Furthermore, we note that as discussed by Yao al. (2024) and others, the primary real-world application of offline optimization solvers is in generative design tasks where the true oracle reward function is prohibitively expensive or inaccessible. In these settings, we argue that it is often worth leveraging additional compute to use DynAMO (or other offline optimization methods) to generate the best results possible before final oracle evaluation.
D.7. 7- Weighted Distribution Visualization
In Definition 3.2, we define the w-weighted probability distribution to serve the reference distribution for generative policy to learn from in (7). This reference probability distribution is important and should idcally capture the diversity of high-quality designs contained in the offline dataset. To investigate if this is indeed the case, we plot the empirical T-weighted distributions for each of the six offline optimization tasks in our experimental evaluation suite using T = 1.0,
"One might argue that MINs (Kumar& & Levine, 2019) are also form of model-based optimization, the method involves learning a surrogate function f IR > X. However, the method proposes design æ given an input score value, and therefore does not make available an output proxy score by which to rank candidate designs. We therefore include MINs as a model-free optimization algorithm.
35

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
T-Weightol Reference Distribution
r. Weighted Reference Distailution
T- W Weighted Reference Distribezion
Unifomn Sampling
Unifomm Sampling
Uniform Sampling
Probability Distribation
10
20
40
30
20
30
40
TFBind8
UTR
ChEMBL
y- Weigghted Reformnce Distribution
W Weighted Refarence Disa lbution
WeWeighted Reference Distribution
Unifurm Sampling
Uniiform Sampling
Uniformm Sampling
Probubility Distribution
10
12
80
Molecule
Superconduector
D'Kitty
Figure AI. Sample 7- Weighted Probability Distributions. We plot (r 1.0)-weighted distributions PD(v) (blue) versus the original distribution of oracle scores y in public offline dataset D (orange) for offline optimization tasks our experimental evaluation suite: (1) TFBind8 (top left); (2) UTR (top middle); (3) ChEMBL (top right); (4) Molecule (bottom left); (5) Superconductor (bottom middle); and (6) D'Kitty (bottom right). DynAMO penalizes a model-based optimization objective to encourage sampling policies to match the diversity of (high-scoring) designs in the r-weighted distribution. The æ-axis represents the normalized oracle scores.
which is the value of the temperature hyperparameter used in our experiments in Table 1. The resulting plots are shown in Figure A1; in general, heavily (i.e., a distribution with negative skew), while still capturing variety of different possible designs.
that our r-weighted reference distributions weight optimal and near-optimal designs more
D.8. Distribution Analysis of Quality and Diversity Results
In our experimental results in the main text and in the Appendices, primarily focus reporting summative statistics: for example, the Best @ 128 oracle score and the average Pairwise Diversity metric over the final batch of & 128 samples. In this section, we isolate a single representative experimental run and plot the distribution of scores achieved by all k: = 128 designs from single experimental run to better interrogate the robustness and empirical properties of DynAMO.
In Supplementary Figure A2, we first plot the distributions of the oracle reward score r(x()) and minimum novelty score miny'ed d= .x)) achieved by each of the k = 128 designs the set (xff } proposed by the CMA-ES optimizer with single experimental run. (Recall that
and without DynAMO augmentation designs and d(.,) is the normalized Levenshtein distance metric for this task.) We see that in general, DynAMO not only enables the optimizer to discover more optimal designs with higher probability, but also yields a wider-tailed distribution of oracle scores compared to the baseline method. Separately, we see that DynAMO augmentation decreases both the median and mode Minimum Novelty score compared to the baseline method, in agreement with our discussion in Appendix D.2.
the static, offline dataset of reference
In the bottom row of Supplementary Figure A2, we visualize a heat map of pairwise diversity scores; that is, the color of pixel (i.j) is correlated with the distance d(r.f' az5) for any S i,j S pair of generated designs proposed by the
36

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Number of Generated Designs
Raseline
Number of Generated Designs
DynAMO-CMA-ES
DynAMO-CMA-ES
60
80
60
40
20
20
0.250
0.750
1.000
TFBind8 Oracle Score
Minimum Novelty
Baseline CMA-ES
DyRAMO-CMA-ES
Rank of Generated Design
25
0.8
0.6
75
100
0.2
125
0.0
100
50
100
Rank of Generated Design
Figure A2. Distribution of Generated Design Quality and Diversity Scores. We plot the distributions of the (top left) oracle (top right) minimum novelty; and (bottom) pairwise diversity of the = 128 proposed designs from single representative experimental run using the CMA-ES backbone optimizers with and without DynAMO on the TFBindS task. Dashed blue (resp., dotted green) lines in the top panels represent the mean score achieved by the Baseline CMA-ES (resp., DyAAMO-CMA-ES) method from the experimental run.
optimization method. Even a cursory visual inspection reveals that DynAMO augmentation of the backbone CMA-ES optimizer significance improves the pairwise diversity of candidate designs when compared the baseline method.
D.9. Why Diversity Important?
Our principle motivation for obtaining a diverse sample of designs offline MBO to enable downstream secondary exploration of other objectives that we might care about in real-world applications. For example, given a batch of proposed candidates drugs that were optimized for maximal therapeutic efficacy treating disease, we might then try to quantify each candidate's manufacturing cost, difficulty of synthesize, profile of potential side effects, and other objectives. In this setting, obtaining highly similar designs from offline MBO may result in strong therapeutic efficacy, but also equally unacceptable values of other secondary objectives.
To validate this motivating claim that diversity is important to obtain a wide range of secondary objective values, we compare the range and variance of secondary objective values within a batch of proposed candidate designs. We consider the following 3 offline MBO tasks:
1. Vehicle Safety is continuous, 5-dimensional optimization problem from Liao et al. (2008) to find an optimal set of car dimensions that minimize the total Mass of the vehicle. The problem initially stems from the multi-objective optimization literature (Blank & Deb, 2020: Liao et al., 2008; Huo et al., 2022: Gonzalez de Oliveira al., 2023). (1) minimize the worst-case Accelerationiinduee biomechanical damage of the car
where the secondary goals occupants in the event of a collision; and (2) minimize the worst-case toe board Intrusion of the vehicle in the event of an 'offset-frontal crash.' We treat the Mass as the offline MBO optimization objective and Acceleration and Intrusion as the downstream secondary objectives. We negate all objective values prior to max-min normalization as described in Appendix B to frame this as a maximization problem in accordance with the setup in Table An offline dataset of n = 800 designs was synthetically constructed, and we used the oracle function from Liao et al. (2008) to compute all objective values.
37

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
2. Welded Beam is continuous, 4-dimensional optimization problem from Ray & Liew (2002) to find optimal set of dimensions for welded steel beam that minimizes the total manufacturing Cost. Similar the Vehicle task, this problem was initially proposed in the multi-objectivv optimization literature (Blank & Deb, 2020; Liao et al., 2008; Kamil et al., 2021; Deb et al., 2006) where the secondary goal is to (1) minimize the end Deflection of the beam." Again, we negate all objective values prior to max-min normalization as described in Appendix to frame this as a maximization problem. An offline dataset n = 800 designs was synthetically constructed, and we used the oracle function from Ray & Liew (2002) to compute both objective values.
3. UTR is a discrete, 50-dimensional optimization problem from Angermüeller et al. (2020) and Sample et al. (2019) with the goal of finding an optimal 50-bp DNA sequence that maximizes the gene expression from 5' UTR DNA sequence. This is an offline MBO problem from the Design-Bente Bench benchmarking suite (Trabucco et al., 2022) that we use to evaluate offline MBO algorithms in our main experimental results in Table and elsewhere. However, a secondary objective is to minimize the GC Content of the resulting DNA sequence, which correlated with the difficulty of cloning and sequencing the DNA sequence using standard DNA amplification and analysis methods in the laboratory setting (Benita et al., 2003; Yakovchuk et al., 2006; Gardner et al., 2002). To evaluate this secondary objective, we use the same experimental setting as for the initial UTR experiments described in Appendix B and evaluate the GC Content of the k = 128 proposed designs as the secondary objective according to Benita et al. (2003).
We used the standard deviation of secondary objective values achieved by proposed set of designs to quantify the range of secondary objective values, and the pairwise diversity metric (PD @ 128) to quantify the diversity of designs. We evaluated both baseline and -eeeMMo-nnhacedd optimization methods on the three tasks above (Supplementary Table A10). Our results consistently demonstrate that a greater diversity score of the final proposed designs (i.e., higher PD 128 score) is correlated with a greater range of captured secondary objective values. As a result, a diverse set of designs (such as those optimization methods) can better enable downstream evaluation of the trade-offs between different objectives for given design.
proposed by DynAMO-enhanee
'The original problem from Ray & Liew (2002) proposed as constrained optimization problem with 5 sets of constraints on the maximum considered shear stress, bending stress, buckling load, and other material testing parameters. To simplify experimental setting, we consider the unconstrained version the optimization problem here.
38

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Table A6. Comparison of Design Quality Against Model-Free Optimization Methods. We evaluate DynAMO and other model-based optimization methods against model-free optimization methods. We report the maximum (resp., median) oracle score achieved out 128 evaluated designs in the top (resp., bottom) table. Metries are reported mean 95% confidence ieevval across 10 random seeds, where higher is better. max(D) reports the top oracle score in the offline dataset. All metrics are multiplied by 100 for easier legibility. Bolded entries indicate average scores with overlapping 95% confidence interval to the best performing method. Bolded (resp.. Underlined) Rank and Optimality Gap (Opt. Gap) metrics indicate the best (resp., second best) for given backhone optimizer.
Best@
TFBind8
ChEMBL
Molecule
Superconductor
D'Kitty
Rank
Opt. Gap
Dataset D
43.9
59.4
60.5
88.9
40.0
88.4
Grad.
90.0(4.)
80.9
60,2(8%)
88.8(4.)
36.0(30)
65.6
16.0
6.8
BO-QUCB
88. 88.1
8..2
66.4
121
51.3
84,5
7.3
19.4
CMA-ES
(7.6
86.2
(1.
506
49.0(
72.2
10.2
14.4
BONET
95.5
92.9
63,3
97,3
39.0
93,7002
8.0
16.8
DDOM
(3.0
(5.3
(3
87,9
44,7(22)
63.0
13.0
9.4
ExPT
(5.3(5-7)
(44)
63.3
93.0
48.5
82,3
12.0
13.3
MINs
8..0
68.3
63.9(09)
(0.7)
45.8(2.))
91.5
11.2
11,8
GTG
92.1 10.00
70,2
(3.3
85.0
52,5
96,4
9.7
13.1
Tri-Mentoring
82.4
66.6
68.4
88.9
50.9(1.1
94,0
10.2
11.7
ICT
93.3
66.6
68.4
88.9
48.g(144
95.5(11)
9.0
13.4
PGS
79.6
67.1%
(M)
88.9
(4.8
72.3
11.3
8.3
Match-Opt
90
(..4
(3
87.8
35,2
72,2
15.5
6.2
RGD
(7.9(42)
68,7
(3,4
90.2
43
88
13.2
10.1
COMs
(3.4
%7.0%
64.6(1.;
(1.6)
41.2(48)
91.8
10.2
RoMA
96.50..1
77.8
63,3(00)
84,7
49.8(1.)
95,7(16)
9.7
14.5
ROMO
98.7
66.8
(3.0(08)
91.8
38,7(2.5)
87,8(99)
12.7
10.9
GAMBO
(1.9)
86.3
(6.8(07)
121
50.8(33)
86,7(11)
4.8
DynAMO-Grad.
90.3
06
64.4
91.2
44.2
89.
9.5
14.2
DynAMO-BO-QUCB
(
86.2.00)
66.7(15)
121
48.1
86,9(45)
6.3
DynAMO-CMA-ES
(9.8(3
85.7(58)
63.9009
117
50.6
78,5(58)
9.3
17.5
Median @ 128
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitty
Rank
Opt. Gap
Dataset D
33.7
42.8
50.9
87.6
6.7
77.8
Grad.
(6.1
58.6
59,3(8m)
85.7
36,0(67
65.1
10.7
BO-qUCB
50.3
62, 62,1
(3.3(00
86.6 (0
31
74,4
6.8
11.5
CMA-ES
(2.7
71.7
(3.3(00)
83.9
(7.
59.3
7.2
11.2
BONET
53. (0.0)
63,3
(1.2
37
92.1
5.2
DDOM
(5.
m..))
(3.3
83.4 (0.3)
21.9(1.7)
57.6 (3.2)
12,2
6.7
ExPT
(4.7(6.8)
(..)
(3,3(00
(3.8
34.1
67. (141)
9.3
9.5
MINs
(1.3
"..)
63.3'00
88.3
32.
68.9
9.7
GTG
(3.4
64.2
63.3
(0.0)
28.0/0.0
90.6
9.3
12.2
Tri-Menten Tri-Mentoring
46. 10.m)
61.1 (00)
(3,
88.9
34,2(1.))
88,4
6.5
13.7
ICT
(9.
61.1 (0)
57.1
88.9
(.2)
18.4
7.0
15.4
PGS
(2.7
m0.
58,400
28.2
70,9
12.3
Match-Opt
(1.7
57.9
63,3
03.
14,5009
60,5
15.0
3,4
RGD
41.
(..)
(3.3(00)
86.4
20.7006
70
12.3
6.7
COMs
43.9
9..
63,3
93.2
(1.3(50)
89
8.5
11.8
RoMA
(4.3)
(3,3
84,7
34.
63.7622
7.3
12.4
ROMO
58,7(33)
0.)7
21.4
61.824)
27.0
46,0
15.8
-6.8
GAMBO
(6.4
(3.4
(3,3
86.3
28.9(1.1)
197)
7.8
11.3
DynAMO-Grad.
27.0
61,92%
85,9
23,4(85)
68,702
11.0
9,5
DyREMO-BO-QUCE
48.8 (8.8(1.)
63,3
86.5
22,7(20)
(0,4
9.7
6.3
DyRAMO-CMA-ES
(2.4
65,8 (5.9
(9,3
99.0
22.5(5.1)
60.6 (5.0)
11.0
8.8
39

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Table A7. Comparison of Design Diversity Against Model-Free Optimization Methods, We evaluate DynAMO and other model-based optimization methods against model-free optimization methods. report the pairwise diversity (resp., minimum novelty) oracle score achieved by the 128 evaluated designs in the top (resp.. bottom) table. Metries are reported mean (96% cunfidence interval) across 10 random seeds, where higher is better. All metrics are multiplied by 100 for easier legibility. Bolded entries indicate average scores with an overlapping 95% confidence interval the best performing method. Bolded (resp., Underlined) Rank and Optimality Gap (Opt. Gap) metrics indicate the best (resp., second best) for a given backhone optimizer.
Pairwise Diversity 128
TFBind8
ChEMBL
Molecule
Superconductor
D'Kitty
Rank +
Opt. Gap
Dataset D
65.9
57.3
60.0
36.7
66.0
85.7
Grad.
12,5(k0)
7,8
7.9
24.1 (4.1)
0.0
0.0
18.7
-53.2
BO-qUCB
73,9/45)
74,3
99.4
93.6
198
94.199%)
3.8
43.5
CMA-ES
(7.
44.6
93,5
66.2
12.8
164
10.8
9.5
BONET
46.7(2.)
24.6
14.9(t
5,5
0.2
0, 1.0
17.3
-46.6
DDOM
(1.3
47. (00.3)
(1.
97.2
1.9
50,7033.)
12.5
-16.9
ExPT
(5.3
16.5
21.3(1.5)
5,3
8.132.3)
0,2
17.5
-50.8
MINs
67.0%
56,8
t3.5
84.6
4.3
11.8
-11.9
GTG
60,9
44,6
(
0,9
(14
2,7
15.0
24.2
Tri-Mentoring
(8.5
S7.6
85 (0.5
39,9
47,700
62.5
10.5
-3.3
ICT
44.8(6.)
57.5
89.g(1.)
78.9
22.3
PGS
(1.8)
57.4
63.2
39.9
36.7
162
10.5
8.9
Match-Opt
65.1 6
55,9
99.8
97.2
10.9
202
7.5
26.6
RGD
67.1
28.4
99.8
97.30M
88.4
76.2
5.0
19.3
COMs
66.6
27.4
81.6
3,8
99.5(258)
21.1
10.7
-6.9
RoMA
().3
3.8
5.
1.800
19.4
(4.8
17.2
-45.8
ROMO
62. (88)
57. (..1
93.9
48.7
51,7017)
22, (5.5)
11.5
-12.7
GAMBO
74.3
99.30..)
93.3004
193012
17,705)
5.5
30.0
DynAMO-Grad.
66.9/5%
68,2
77.2
"3.0
129
104
DynAMO-BO-QUCE
(4.3
(M,)
99.3
93.5mm
211
175(447)
2.8
59.4
DyAAMO-CMA-ES
73.6
73.1
7..
94.0
97.8
292
5.2
55.2
Minimum Novelty 128
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitty
Rank
Opt. Gap
Datasct D
0.0
0.0
0.0
0.0
Grad.
21.2
51.7(29
97_4
79
95(117)
102,2
5.8
74.5
BO-qUCB
21.6
51.7
97.9%.4
85,3
93.g(06)
(t..)
CMA-ES
16,5(2
47.8
96.5
73.0
100
100
7.8
72.3
BONET
94,6
18
11.2
10.3
1,3000)
1.
14.7
31.2
DDOM
|1.)
3..6
96.5
9..2
98.0
9.3
73.1
ExPT
(1.
39.0
54.2
(L2)
69.8
3,5
15.2
32.2
MINs
(2
38
48.7
22.1 (197)
6.1
GTG
(3,
37
99.3
99.4 (0
67.7
0,2
10.5
53.0
Tri-Mentorin
13.90 13,9
31.8.00
74.7
75,5
44.0
64.300.0
14.3
50.7
ICT
(..3
31,7
80.8
75.6
46.2 46.2
66.2 66.2
13.2
51.6
PGS
(1.5
1..0
42.0
(4.8
16.3
32.7
Match-Opt
12. (0.)))
40.0
98.5
94.9
91.9
85.8(2
8.7
70.5
RGD
(2.3
39.0
98.6
9..2
90.4
90.5(2.9)
8.5
70.9
COMs
10,9
31,708
52,4(11.)
13,70.13
99,)
(00
14.0
51.4
RoMA
18.3
40.1
(8.
95,3000
47,6
1.2
11.0
37.6
ROMO
16. (0.5)
32.
3.
23.1 000)
(8.5
(5
51.5
GAMBO
(5.4
51.8
97.8
84.909)
85.1 (B.)
14.3015)
58.2
DynAMO-Grad.
2
52.2(1-3)
91.6
95.0
107.2%6
3.8
76.7
DynAMO-BO-QUCE
21.4
51.7
97.1 10.5
85,30.0
94,70z
109
DyBAMO-CMA-ES
12
(1.5
96,713.5
(1.8
94.5
112
7.8
74.3
40

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Basce Optimization
Table Comparison of Design Diversity Against Model-Free Optimization Methods (cont.). evaluate DynAMO other model-based optimization methods against model-free optimization methods. We report the L1 coverage score achieved by the 128 evaluated designs. Metrics are reported mean 958 confidence inteval) across random seeds, where higher better. All metrics are multiplied by 100 for casier legibility. Bolded entries indicate average scores with overlapping 95% confidence interval to the best performing method. Bolded (resp., Underlined) Rank and Optimality Gap (Opt. Gap) metrics indicate the best (resp., second best) for given backbone optimizer.
Li Coverage@ 128
TFBind8
UTR
CHEMBL
Molecule
Superconductor
D'Kitty
Rank
Opt. Gap
Dataset
0.42
0.31
0.68
6.26
0.58
Grad.
0,16
0.20
00
0.42
0.0g
0.00
19.5
-1.44
BO-qUCB
0040
0.0r)
(..5)
mM5)
7.79(004
6.64
4.3
1.77
CMA-ES
0,33
0.48
2.18
1.82
3.26
3.78
8.5
0.36
BONET
(000
0.22
0.72
0.54 (0.00)
0.03
0.07
17.8
-1.33
DDOM
0,43
0.29 (0000)
(0.5))
0.85 (0.02)
9.24
0..6
10.8
0.41
ExPT
0,22(005)
(0.0)
0.60 (0.3))
(0.3))
0, (0.08
0.45
0. t0000
18.0
-1.29
MINs
0.43
0.30
1. 0.3
0,70 (..00
0. 16
0,43
11.0
-0.94
GTG
0.42
0.30
1.61 1.61 00000
1.96 1096 (0/00)
8.770000
0.31 30.00
8.7
0.62
Tri-Mentoring
0.4g
0.30
1.0g
(0.00
(.94
3.79
-0.26
ICT
0.34
0.30 (.000)
0090
(h:0
0.56
(.73
12.8
-0.51
PGS
0.45(0u4)
0000)
t.40
0.60)
1.92
3.66
10.0
-0.22
Match-Opi
0.02
0.6g
0.86 m.00)
5.26
5.22
8.7
0.52
RGD
0.42
(0.00
(.61
m..6)
(.0g)
9.0
0.27
COMs
0.49/auz
0.31 (0.00)
1.1 (0.16)
0,61 (0.0%)
0,370011
0.81 (0.7)
11.0
-1.00
RoMA
0028
0.46
0.41 0.41
0. 0.42
1.87
0.79
14.8
-0.91
ROMO
0.33
0.000
(002))
0.62
0.34
(213
11.8
-0.11
GAMBO
0.4
0.05
0.30
(1.5)
0.45001
(.29
6.3
0.82
DynAMO-Grad.
0.36
0.50
(.48)
2.50
6.47(129
5.85
6.7
DynAMO-BO-QUCB
0.40
0.55
2.4770087
m.04
7.8g
7.80
2.8
2,00
DynAMO-CMA-ES
0.03)
m..6
(.82
m.54)
4.75(2.16)
3.29
6.3
0.62

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Table A9. Computational Requirements of DynAMO. To evaluate the computational cost of augmenting MBO problem with DynAMO, we compare both the total runtime and maximum GPU utilization of vanilla optimizers with that of their DynAMO equivalents on six MBO problems. Runtime reported mean" (958 confidincee interval) in seconds across 10 random seeds. Maximum GPU utilization reported for single experimental run. The average metric across all six tasks is reported in the final column.
Runtime (seconds)
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitty
Average
Grad.
58. (2:0
240.0
104.783)
458.7(24)
24.7
10357
1874
DynAMO-Grad.
517.0
2686
297.0 1000
1091 (231)
917.5(5544)
13302
3135
Adam
26. (0.
249.2
(5.3
(546)
24.4
8797
1653
DynAMO-Adam
646,7% (46,7(185))
616.6 106.1)
709.6 641103)
953.7
6941 (205)
24444
5719
BO-qEI
182.4 (5
339.6(61.7)
(94.)
1548
19938
(196
838.0
DynAMO-BO-qEI
449.9% 1(11))))))))
952.8
494.4
1307(221))
1122
5973(1245)
1717
BO-qUCB
(36.4
47..4
589 g(
524.9
8324
1738
DynAMO-BO-qUCB
408 (66.)
1531 (1466)
932.3
1525
2100
11648
3024
CMA-ES
109.4
130
66.0
604, (1244 (1244
20.0
14027(2651)
2493
DynAMO-CMA-ES
7994
9551
751.3
8088
4796(445)
13727
7943
CoSyNE (33.9
110.3
(50.0
25.g
9876,8 (
1842
DysAMO-CoSyNE
8823
11388
1374
8002
6261 (300))
171473350
8832
Max GPU Utilization (MB)
TFBind8
UTR
CbEMBL
Molecule
Superconductor
D'Kitty
Average
Grad.
307.9 9
786.8
503.9
1920
133.6
131.7
630.7
DynAMO-Grad,
374.1
994.8
503.9
1920
208.3
160.1
693.6
Adam
307.9
786.8
503.9
1920
133.6
131.7
630.7
DynAMO-Adam
374.4
995.0
503.9
208.4
160.1
693.7
BO-qEI
307.9 9
786.8
519.1
1920
372.6
544.1
741.8
DynAMO-BO-qEI
371.4
789.0
503.9
1920
479.4
445.2
751.6
BO-qUCB
597.3
786.8
642.3
1920
712.4
879.5
923.
DynAMO-BO-qUCH
516.2
786.8
646.9
718.0
406.1
832.4
CMA-ES
307.9 9
786.8
503.9
1920
133.6
131.7
630.7
DyBAMO-CMA-ES
307.9
786.8
503.9
1920
133.6
131.7
630.7
CoSyNE
307.9
786.8
503.9
1920
133.6
131.7
630.7
DynAMO-CoSyNE
307.9
786.8
503.9
133.6
131.7
630.7
42

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Basce Optimization
Table A10. Pairwise Diversity Predictor for Downstream Secondary Exploration. We evaluate the pairwise diversity achieved by 128 proposed designs (PD@ @ 128); and also the variance of the distribution of oracle secondary objective values of those same 128 proposed designs. Note that the secondary objectives are not explicitly optimized against offline MBO setting, Metrics are reported mean 95% confidence interval) across 10 random seeds, where higher is better (i.e., more diverse designs and better capture of the range of secondary objective values). All metrics are multiplied by for easier legibility.
Vehicle Safety
Welded Beam
UTR
Method
PD@ 128
Accelcration
Intrusion
PD@ 128
Deflection
PD 128
GC Content
Grad, (.0
0.1
0.0
0,0%0
0.1
7,8
0.7
DynAMO-Grm
2.5(0.2)
12.6
10.7
19.6
31.7
68.2 68.2
3,4
Adam 0.0
0.1
0.1 0.
0.0
1,5(1
11.0(12.1)
4.0
DynAMO-Adam 2.
12
10.3
11.1
49,5(213)
72.30.4)
14.0/3.)
CMA-ES
0.0
0.5
0.4
0..
0.0%0.0
44.6
36.5(8.3)
DynAMO-CMA-ES 8.6
28.5000
30.
43.9
19,8
73.1
42.0(2.3)
CoSyNE 0.0
0.6
0
1.0
16,8
12.79.8
1.622.)
DynAMO-CoSyNE 1.9
4,0(22)
2,2
55, (883))
65,8(132)
20.3(2.3)
1.00.3)
BO-qEI
1.
7,5004)
(.2
46.3(20)
8,0
73.8
44.8(0.8)
DynAMO-BO-qEI 2.1
13,
(0.5
78. (18m)
26.6(2.1)
74.6
45.7(0.6)
BO-qUCB
1.2
5.70.00
6.1 (0.)
46.5
7.8
74.3
45.3(0.4
DynAMO-BO-qUCB
0.1
12.3
0..
(3.9(42)
29.0(19)
74.4
45.2(0.5)
43

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
E. Ablation Experiments
E.1. Sampling Batch Size Ablation
Recall from (7) that a key component of our DynAMO algorithm is the estimation the empirical KL-divergence between the r-weighted probability distribution of real designs from the offline dataset and the distribution sampled designs from the generative policy. The latter distribution of generated designs is fundamentally dependent our sampling batch size b in Algorithm the larger the batch size per sampling step, the better empirical estimate of the KI. divergence between our two distributions. However, the batch size increases, there also exists greater likelihood of significant regret in the sampling policy when compared to the optimal sequential policy (Gonzalez et al., 2016; Wilson et al., 2017). To better evaluate the impact of the sampling batch size parameter b DynAMO, we experimentally evaluate sampling batch size values logarithmically ranging between S S 512. We use a BO-qEI sampling policy with the DynAMMOmodiifen objective on the TFBind8 optimization task, and evaluate both the Best 128 oracle score and Pairwise Diversity of the 128 final proposed design candidates across 10 random seeds.
Our ablation experiment results are shown in Figure A3. We find that the Best@ 1 128 design quality scores do not vary significantly function of the batch sizes that were evaluated; however, there exists optimal batch size (b 64 in our experiments) that maximizes the diversity of designs according to the pairwise diversity metric.
128 Oracle Score
W
TFBind8 Pairwise Divensity 128
95.0
02.5
74.5
90.0
TFBind8 Bestig1288
87.5
73.5
72.
Sampling Batch Size
Sampling Batch Size
Figure A3. Sampling Batch Size Ablation. We vary the sampling batch size b in Algorithm between 2 and 512, and report both the (left) Best@ 128 Oracle Score and (right) Pairwise Diversity score for 128 final designs proposed by DynAMO- SyynAM-BO-oae BO-qEI policy on the TFBind8 optimization task. We plot the mean + 95% confidence interval over 10 random seeds.
E.2. Adversarial Critic Feedback and Distribution Matching Ablation
Recall that instead of solving original MBO optimization problem in (3), DynAMO leverages weak Lagrangian duality solve the constrained optimization problem copie below for convenience:
max ((x) = EE [ro([)] DKL lIb) FEll
(85)
Ex-py (x) Exng (x) S Wo
We can think of this problem formulation as as the fusion two separable components: (1) Adversarial feedback via a source-critic model (*r) to prevent out-of-distribution evaluation of ro(ar); and (2) Diversity (via KL--dieegenne--base distribution matching with a diverse reference distribution p5) in Model-based Optimization. These two components together form the foundation of DynAMO presented in Algorithm To better understand how each of these two components affects the performance of DynAMO-aagmentee augmented optimizers,
separate these two components and study them individually.
AMO is our ablation method that solves the related optimization problem
max TEIT (r
(86)
s.t.
Ex-pg(a) (x) (x)a Wo
instead of (85). Note that AMO solves the same constrained optimization problem as DynAMO in the setting where We note that our derivation of the Lagrange dual function (7) Appendix A is invalid when O, and so we cannot
44

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
exactly solve (86) using the same methodology presented in Algorithm 1. Instead, we leverage the adaptive Source Critic Regularization (aSCR) algorithm from Yao al. (2024) to appraximate a solution to (86) in the Lagrangian dual space-we use their publicly available implementation of aSCR at github. com/ /mi chael-s sosssyaogggg gabo and defer to their work for additional discussion regarding the specific implementation details of aSCR.
DynMO is our separate ablation method that solves the related (unconstrained) optimization problem
max FEll
J(T) = Bap [ro(#)] DDI..
DRI. (qk..("")) (1p5)
(87)
instead of (85). To implement DynMO empirically, we modify Algorithm 1 by ignoring the subroutine to solve for the globally optimal Lagrange multiplier À using (13), and instead fixing À = 0 for the entire optimization process to effectively remove any contributions from the adversarial source critic c* (z)). All other implementation details were kept constant.
We compare DynAMO with AMO and My in Supplementary Tables A11-A13. Firstly, we note that DynAMO and AMO are competitive in proposing the high-quality designs according to the Best@ 128 oracle scores, alternating between having the highest and second best Rank and Optimality Gaps across all six tasks when compared with DynMO and the baseline optimizer for all optimizers evaluated. This makes sense, as the purpose of the adversarial source critic-dependent constraint in (85) and (86) is to minimize forward surrogate model (z(x) can provide a better estimate of the quality of sampled designs, leading to higher quality designs according to the true oracle function r(z). Separately, we find that DynMO and DynAMO also perform similarly in terms of the all 3 diversity metries evaluated. However, we find that DynMO (resp., AMO) struggles on proposing high-quality (resp., diverse) sets of final designs. These experimental results collectively allow us to conclude that both the adversarial source critic supervision and KL-divergence-baaee distribution matching are important for DynAMO to propose both high-quality and diverse sets of designs.
evaluation of ro()) during optimization- as result, the
E.3. 8 and- Hyperparameter Ablation
Fundamentally, DynAMO relies on two important hyperparameters that define the constrained optimization problem in (7): (1) the 3 hyperparameter dictates the relative weighting of the KL-divergence penalty relative to the original MBO objective; and (2) the temperature hyperparameter describes the distribution of reference designs weighted according to their oracle scores in the offline dataset. To better interrogate how these hyperparameters impact the performance of DynAMO--ugmmnteed MBO optimizers, we (independently) ablate the values of both ,3 and T logarithmically between 0.01 S 3, < 100. Similar to our experiments in Appendix E.1, we use a BO-qEI sampling policy with the DynAMO- modified objective on the TFBind8 optimization task, and evaluate both the Best@ 128 oracle score and Pairwise Diversity of the 128 final proposed design candidates across 10 random seeds.
Our experimental results for our / ablation study are shown in Supplementary Figure A4. Firstly, as the strength of the KL-divergence term & increases, the diversity of proposed designs (according to the Pairwise Diversity metric) increases roughly proportional to the logarithm of / (Supplementary Fig. A4). This is expected: as the distribution matching objective becomes more important relative to the ro( forward surrogate model, the generative policy is rewarded for finding an increasingly diverse set of designs that matches the w-weighted reference distribution. Similarly, we found that for sufficiently large values of 3 (i.e.. 3 2 0.03 in our particular experimental setting), the quality of designs (according to the @est@ 128 oracle score) decreases due to the inherent trade-off between design quality (according to re (xr) and diversity (according to the KL-divergence in (7)). Interestingly, for small values of 33 (i.e., B S 0.03) the quality of designs actually increases with B. This is because in this regime, naively optimizing against primarily ro(x)) leads to the policy exploiting suboptimal regions of the design space peree--inniziigggg the optimization objective with a "small amount of" the diversity objective helps the policy explore new regions of the design space that can contain more optimal designs according to the hidden oracle objective r(x).
Separately, the experimental results for our 7 ablation study are shown in Supplementary Figure A5. (Note that in these experiments, we fix B T so that the ratio 3/T in Algorithm remains constant.) As the value of 7 increases, the diversity of designs captured by the reference r-weighted probability distribution decreases and approaches (potential mixture of) Dirac delta functions with non-zero support at the optimal designs in the offline dataset. As result, distribution matching via the KL-divergence objective no longer encourages the generative policy to find a diverse sample of designs, as the reference distribution is no longer diverse itself for 7 > 1. Similar to our / ablation study, we find that there is a unique exploration-expe expllittticellliiiiininnoo trade-off phenomenon according to the @est@ 128 oracle score as a function of T: in our particular experimental setting, we find that for S 1, the @est@ 128 oracle score (modestly) increases, while for T 2 1, the score
45

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
TPBind8 Dest@g 128 Orucle Scote
TFBind8 Pairwise Diversity@) 12
76.0
#1
75.5
75.0
10
10
10
10"
10'
B Hyperparameter
B Hyperparameter
Figure A4. Hyperparameter Ablation. We vary the value of the KL-divergence regularization strength hyperparameter in Algorithm 1 between 0.01 and 100, and report both the (left) @est@ 128 Oracle Score and (right) Pairwise Diversity for 128 final design candidates proposed by SypAMO-BO-qPe policy on the TFBind8 optimization task. We plot the mean + 95% confidence interval over 10 random seeds in both plots. The dotted horizontal line corresponds the experimental score, which could plotted as point on the logarithmic r-axis.
TFBind8 Bestig 128 Orucle Score
TFBinds Pairwise Diversity@i) 128
94
74
92
19
72
86
12
10
10
10
10'
10a
10
10"
10"
10'
10
T Hyperparameter
T Hyperparameter
Figure A Temperature Hyperparameter Ablation. We vary the value of the temperature hyperparameter r in Algorithm between 0.01 and 100, and report both the (left) Best@ 128 Oracle Score and (right) Pairwise Diversity for 128 final designs proposed by DynAMO-BO-qEI policy on the TFBind8 optimization task. We plot the mean + 95% confidence interval over 10 random seeds.
decreases. For r Rs 1, we find that the generative policy is encouraged to match a sample of high-quality samples that is simultancously diverse enough for the generative policy to explore new regions of the design space.
E.4. Oracle Evaluation Budget Ablation
Recall that in our experiments, evaluate DynAMO and baseline methods using oracle evaluation budget of A: 128 samples consistent with prior work (Mashkaria et al., 2023; Yu et al., 2021; Trabucco et al., 2021; Chen et al., 2023c; Yao et al., 2024). More specifically, this means that any offline optimization method proposes exactly l: design candidates that are evaluated by the hidden oracle function r rxx) as the final step for experimental evaluation. In Table 1, we reported both Best@k and Pairwise Diversity @k metrics, where Best@ represents maximum oracle score achieved by the final design candidates; and Pairwise Diversity k represents the pairwise diversity averaged over the & candidates.
However, in different experimental settings might have different evaluation budget available- larger values of k are more costly but enable us to evaluate more designs that are potentially promising, whereas smaller, more practical budgets may preclude the evaluation of optimal designs according to r(z). In this section, we evaluate the performance of DynAMO as a function of the allowed evaluation budget 16 < l S 1024. We compare corresponding vanilla backbone optimization method on the TFBind8 task, and plot the mean and 95% confidence interval @est@ l and Pairwise Diversity @k metries as function of in Supplementary Figure A6.
optimizers against the
46

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
As expected, the Best@k k oracle score is monotonically non-decreasing as a function of k for all DynAMO-enhanced and baseline optimizers (Supplementary Fig. A6). We also find that in the limit of k > 1, the DynAMO optimizers are able to propose best designs that are more optimal than the designs by their baseline counterparts for first-order, evolutionary, and Bayesian optimization algorithms. Furthermore, DynAMO achieves a mean Best k score non- non-inferior to that of the baseline method for all A 2 128 across all the optimization methods evaluated on the TFBind8 task.
Separately, we find that the Pairwise Diversity of the k designs proposed by DynAMO-augmented first-order optimizers (i.e., DynAMO-Grad. and DynAMO-Adam increases as a function of k. This makes sense, as first-order methods generally produce optimization trajectories that are simple curves the design space as a function of the acquisition step. As a result, increasing i can be informally thought of as increasing the fraction of the trajectory curve connecting the initial and final samples during the acquisition process. In contrast, we find that the Pairwise Diversity decreases after a certain optimizer-dependens threshold k for evolutionary and Bayesian optimization- based backbone optimizers. This is because as both classes of optimization methods do not necessarily sample repeatedly from any given region of the input space; as a result, the pairwise diversity between any two sampled points may decrease as more of the design space has been explored as a function of k. Finally, we found that leveraging DynAMO improves the Pairwise Diversity of designs compared to the baseline objective for almost all optimizers and values of assessed, as expected.
Altogether, these results suggest that DynAMO helps optimization methods discover both high-quality and diverse sets of designs across wide range of oracle evaluation budgets.
E.5. Optimization Initialization Ablation
In Algorithm 1, we initialize DynAMO by sampling the initial batch of b = 64 designs according pseudo-random Sobol sequence as described in Appendix B. This initial batch of designs is used as the 'starting point' in our first-order optimization experiments. However, most first-order offline MBO algorithms reported in prior work (Trabucco et al., 2021; Yu et al., 2021) do not follow this same initialization schema. Instead, they perform a top-k initialization strategy where the top 1: = b designs in the dataset with the highest associated reward score constitute the initial batch of designs. First-order optimization is then performed on these initial top-k designs. However, it possible that for many MBO problems, these top-k initial designs constitute only small 'area' of the overall search space, resulting in lower diversity of final designs when compared to Sobol sequence initialization.
To interrogate whether the gains in diversity of designs obtained with DynAMO are due to our Sobol sequence-based initialization strategy, we evaluated baseline Gradient Ascent, COMs, RoMA, ROMO, GAMBO with Gradient Ascent, and DynAMO with Gradient Ascent using both Sobol sequence-based and lop-k-based initialization strategies. All algorithms were initialized using k = = 64 samples and used Gradient Ascent as the backbone optimizer (except for RoMA from Yu et al. (2021), which used Adam Ascent in line with the original method proposed by the authors).
Our results shown in Supplementary Table A14. Empirically, we found that the relative performance of Sobol eequence-inttialized and Top-k-initialized, optimizer largely depends on the specific algorithm; for example. COMs and RoMA strongly benefit from using Top-k initialization in obtaining high-quality designs. This makes sense, as the original authors for both methods use Top-k initialization for all their experiments. In contrast, the quality of designs proposed by GAMBO and DynAMO is better with Sobol sequence initialization.
While DynAMO using Sobol sequence initialization does indeed outperform the lop-k--nitializedd counterpart across all tasks, both initialization strategies consistently propose batches of designs with competitive pairwise diversity scores when compared to other first-order optimization algorithms. This suggests that DynAMO is able to provide significant advantage in proposing diverse designs that extend beyond the choice of initialization strategy alone. Separately for the other first-order optimization methods assessed, there is no clear ad advantage in obtaining diverse designs when using Sobol sequence initialization according to the pairwise diversity metric across all tasks. In summary, these results suggest that DynAMO is able to propose both high-quality and diverse sets of designs with performance exceeding what is possible with a switching to a Sobol sequence initialization alone.
47

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Table AIL Quality of Design Candidates in Ablation of Adversarial Critic Feedback (AMO) and Diversity in (DynMO) Model- based Optimization. We evaluate our method (1) with the KL-divergence penalized- MBO objective as (11) only (DynMO); (2) with the adversarial source crilic-dependent constraint introduced by Yao et al. (2024) only (AMO); and (3) with both algorithmic components as DynAMO described in Algorithm 1. We report the Best@ 128 (resp., Median 128) oracle score achieved by the 128 evaluated designs in the top (resp., bottom) table. Metries are reported mean 1958 confidence imteral) across 10 random seeds, where higher is better. Bolded entries indicate average scores with an overlapping 95% confidence interval to the best performing method, Bolded (resp., Underlined) Rank and Optimality Cap (Opt. Cap) metrics indicate the best (resp.. second best) for a given backbone optimizer.
@ 128
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitty
Rank
Opt. Gap
Dataset D
43.9
59.4
60.5
88.9
40.0
88.4
Grud.
90,0**
80,9/12n
60,2
88.8'4"
36,0%%i
65.6
3.2
6.8
AMO-Grad.
73.1
77.1
64.4
92.8
46.0%
90.6
1.8
10.5
DynMO-Grrd
61.3*%
63.6
59.8
89,3'5
36.3%%
70.4
3.5
0.0
DynAMO-Grud.
90.3
86,2
(2.4
91,
44,2
89.8
1,5
14.2
Adam
62.9(13m)
69.7 (9.7
62.9
92.31%%
37.8%..
58.4
2.8
0.5
AMO-Adam
24,0
(0.0
60,9
91.4
37.8%.3
2.8
8.6
DynMO-Adam
(129
68,7
63,7
92.0%
(8.6
66.5
2.5
2.5
DynAMO-Adam
(5,2
86.2
68,2
91.2
45.5(5.7)
84.9112
1.7
14.5
BO-qBI
87.3(58)
86.2
65.4
1173..
53.1
84.4
3.5
18.7
AMO-BO-qHI
94.1
86.3
66.8
121 (0,
(0.8
86,7!I.
1.5
20.8
DymMO-BO-qEI
(3
86.2
66.0
121
49.6
85,9
2.7
20.2
DynAMO-BO-qEI
91,9/44)
86.2
67.0
121 (0.0)
53,5
85,5(L.)
1.8
20.7
BO-gUCB
(5.1
86,2
4
121
51.3
84,5
2.2
19.4
AMO-BO-qUCB
86,2
121
50.2
83,6
2.7
20.2
DyMMO-BO-QUCH
26.2
6
121 (hu
49,9/3m
83,
2.7
20.0
DynAMO-BO-qUCB
(1.
86.2
66.7
121 (Mu
48.1
86.9/4.5
2.2
20.5
Buseline-MMA-BS
87.6 t.
86.2
66.
106
49.0
72.2
2.8
14.4
AMO-CMA-ES
90,4444
86.2
(M
45.265.5
72.2
1.8
16.7
DyaMO-CMA-ES
85.2
86.2
65.0
21.6
83,6
2.7
15.8
DyRAMO-CMA-ES
3.
85.7
63,9
117167)
50.6
78,5(55)
2.7
17.5
Co
61.7 61,700.0
57.3
63.6
94.g(t..)
37.0
62,7013.
3.5
-0.6
AMO-CoSyNE
79.8
68.0
64.2
99,4
62,7
2.2
5.0
DynMO-CoSye CoNyNE
59,3
63.
90.1 0025)
62,7
3.0
-0.7
AMAAMO-COSyNE
91,3'44
77.2
63,
114 (..0
40.6 (0
67.5
1.2
12.3
Median 128
TFBind8
UTR
CbEMBL
Molecule
Superconductor
D'Kitty
Rank
Opt. Gap
Dataset ID
33.7
42.8
50.9
87.6
6.7
77.8
Grad.
$8.1
58.6 (1.6
59,3
85.3
36.0
65.1
3.2
10.5
AMO-Grud.
1377
7
60.1 (3.3)
91.6
46.0
90.1
1.2
21.2
DynMO-Grud.
(0,5
58.6 (13n
59,7
36.
70.
3.0
10.1
DynAMO-Grad.
2..0
69,8
61.9/22)
85,9
23.4
68,7
2.7
9.5
Adam
54.7/5.5)
(227
$9,2
87.9)10.
37.4
56.8
2.3
9.5
AMO-Adam
49.5
55,7122
57.70.u
84.3
37.4
87.8(43)
3.0
12.1
DynMO-Adam
%4.0%
60.5
59,3
85.grtss
37.7'6.4
63.6615.
2.2
10.2
DynAMO-Adas
47.7
69.0)22)
62.4
86.4 ths
615.61141)
2.3
9.1
BO-qBI
48,50155
59,9
m3.
86,7
28,711.1
72.4
2.5
10.0
AMO-BO-qEI
46.4
63,4
"3.
86.3
28.
79.1
2.2
11.3
DynMO-BO-qEI
$0,5
(3.
63,3
"
79.1
2.3
11.5
DynAMO-BO-qhI
$1,5
65.6
mm
86.7
2
77.0
2.0
11.3
BO-qUCB
(0.3
(2.4
m3.
86.6 tuu
31.7
74.4
1.7
11.5
AMO-BO-QUCB
4.,
59, ((2))
mm.
22.
73.8
2.8
10.7
DynMO-BO-qUCB
(2.
mmm
16.4
3274
74.
2.0
112
DynAMO-BO-gUCC
65
63.
(6.5
22.7
50.4
2.5
6.3
CMA-ES
50.7
71.7
63.3
83,9 83.g
37.9
59,3
2.2
11.2
AMO-CMA-ES
72.7
62,70LD
86. (8.))
21. (200
54.9%%
3.2
7.1
DynMO-CMA ES
50.7225
75.
"3,
38,7(39)
65.8
1.7
12.8
DynAMO-CMA-ES
45.3
65.
(9,3
99.0
22,5(5.1
60.6
2.8
8.8
CoSyNE
(5.3
53,6
60.8
87,4
(6.6
59,3114.1
2.5
8.9
AMO-CoSyNE
59.
m3.5
mM
84,2
16.6
59,3
2.2
9.8
DynMO-CoSyNE
55,6%
(5.4
87,9/12.)
(6,6
59,3144
2.3
9,9
DynAMO-CoSyNE
11.8
(1.5
(3,3
(2.
20.5
60.m
2.5
9.5
48

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Table A12. Diversity of Design Candidates in Ablation of Adversarial Critic Feedback (AMO) and Diversity in (DynMO) Model- based Optimization. We evaluate our method (1) with the KL-divergence penalized-MBO objective in (11) only (DynMO); (2) with the adversarial source settiedeeendent constraint as introduced by Yao et al. (2024) only (AMO); and (3) with both algorithmic components in DynAMO described Algorithm 1. We report the pairwise diversity (resp., minimum novelty) oracle score achieved by the 128 evaluated designs in the top (resp.. bottom) table. Metrics are reported mean 195% contidence interval) across 10 random seeds. where higher is better. Bolded entries indicate average scores with an overlapping 95% confidence interval to the best performing method. Bolded (resp., Underlined) Rank and Optimality Gap (Opt. Gap) metrics indicate the hest (resp., second best) for given optimizer.
Pairwise Diversity @ 128
TFBind8
UTR
CHEMBL
Molecule
Superconductor
D'Kitty
Rank L
Opt. Gap
Datuset D
65.9
57.3
60.0
36.7
66.0
85.7
Grad.
12,5/k.)
7,8
7,90%
24.1
0,0
0,0
3.0
-53.2
AMO-Grad.
17,3
11.20003)
6,9
22.1
0,0
1,5
2.7
-52.1
DynMO-Grad.
20,9/15.
3,0
58,2
(3,5
0,0
0,0
3.0
-46.0
DynAMO-Grad
66,9
68,2
27.2
93.0/t.)
129
104
1.0
27.8
Adam
12.0
11.0
4.8
16.8(22
6.4944.5
6,2
3.0
-52.4
AMO-Adam
(1..)
(0.3
12, (11.))
19.6
0,3
2,6
3.0
-51.9
DynMO-Adam
13.1 (1144)
10.3%
57.0
23.8 (55
6.4(9455
0.0
2.8
43.5
DynAMO-Adam
54.8
72.3
84
89.9/33)
158
126
1.0
35.7
BO-qEI
73.7 73,7
73.8
99,3
9.5
190%
124
3.7
47.1
AMO-BO-qEI
74.0
74,3
99.3
93.3
193
17.79.5
2.8
30.0
DynMO-BO-gEI
74,5
74,3
99.3
93.3
200
13501.
2.3
50.8
DynAMO-BO-qEI 74,8
74.6 2.
99,4
93.5
198
277
1.2
74.2
BO-qUCB
73.9
%4.3
99.4
93.6
198
2.5
43.5
AMO-BO-qUCB
*..)
74.3%
99.3"9.1
93.4
22.0
3.7
30.3
DymMO-BO-QUCB
74,7
74.3
99.2
93.6
198
92.6
2.2
43.5
DynAMO-BO-QUCB
74.4
99.3
93,5
211
175
1.7
59.4
CMA-ES
47.2
44.6
93,5
66.2%%
12.8 (20
164
2.3
9.5
AMO-CMA-ES
19.6
53.4
84,8 (4
61.3(146
173
59,9
2.3
16.8
DynMO-CMA-ES
33.5
11.1
34.5"
4,515%
(5:4
14.4
3.8
-39.3
DyRAMO-CMA-ESS
73.6
(M
72.0(1
94.0%
97,8
292
1.5
55.2
CoSyNE
5,6(5
12,79.8
(1.2
12.2
0.0
0.0
2.8
-52.1
AMO-CoSyNB
(,2
9.1
(8.7
(.1
0.0
0.
3.7
-53.6
DynMO-CoSyNE
27.4
13.0
23,3
18, 18.1033
0,0
0.0
2.2
-43.3
DyRAMO-CoSyNE
18.1 1131
20.3
35.0
22.8(119)
74.4
37.0
1.3
-20.7
Minimum Novelty @ 128
TFBind8
UTR
CHEMBL
Molecule
Superconductor
D' D'Kitty
Rank
Opt. Gap
Dataset ID)
0.0
0.0
0.0
0.0
0.0
0.0
Grad.
21.2
51.72%
97.4
19.
95.0/a7)
102
2.3
74.5
AMO-Grud.
14.0
46,73.7
76.8 19.7)
83.8 83.g
31,5
3.8
58.3
DynMO-Grud.
21.
86.4 (0.2)
95,0/07
102
1.8
75.4
DynAMO-Grud.
21.1 (1.)
(2.2
95,0
107(67)
1.7
76.7
Adam
23,712.0
51.1 (3.9
95.5
79.3 (212)
94.8
103
2.7
74.5
AMO-Adam
23,7
%1.3334
95.0/5.0
80.0
g(54)
27.3034
3.0
60.4
OymMO-Adam
22.9(2.5)
51.6
99.
94,7
103
1.8
76.4
DynAMO-Adam
14,7
46.2/9.5
9..7
94.9/44
108
2.3
74.7
BO-qEI
(1.8
51.5
97.6
(5.4
94.
106
2.5
76.2
AMO-BO-qEI
(5.4
51.8
97.
84.9
85.1 15.1
14.391.5
3.3
58.2
DynMO-BO-QEI
(0.4
51.8
97.7
15,7
9..4
108(3
2.2
76.4
DynAMO-BO-qEI
51.9
97.4
85.2
94.8
126
2.0
79.4
BO-qUCB
$1.7%.1
97.9/44)
93.g006)
98.8 g8.g
2.0
74.8
AMO-BO-QUCB
21.9
51.7
97_5004)
(
81.9(19)
25.9(1.4
2.7
60.7
DymMO-BO-QUCH
20.7
51.8
97.1
04.
93,2(14
98.0
3.2
74.3
DYMMO-BO-
2..4
51.7
97.1
85_3(1.))
94.7
109/45)
2.2
76.6
CMA-ES
16,5
47.8 47.8
96
73.0 18mm
100
100
2.3
72.3
AMO-CMA-ES
24,3'8%
53,3
95.0(1.5
72.5 72.5 (236
85
41,50m
3.0
62.0
DynMO-CMA-ES
(4,3
46.1
(1.3(14)
100
100mm
2.0
73.7
DyBAMO-CMA-ES
12.9
48.0
9..7
(1.8
94_5107)
112
2.3
74.3
CoSyNE
(4,5
49.7
98.5(La)
86.6
93.2(L0)
91,9/2m
1.8
74.1
AMO-CoSyNE
22.8
(0.8
90.8
91.9344
86,0
29.6
2.5
62.0
DymMO-CaSyNE
19.3
46.3
88.3
85.7
29.6
3.2
60.4
DynAMO-CaSyNE:
17.8/5.5)
48.4
96.7035)
80.2
*4.5
112
2.5
75.0
49

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Table AI3. Diversity of Design Candidates in Ablation of Adversarial Critic Feedback (AMO) and Diversity in (DynMO) Model- based Optimization (cont.). We evaluate our method (I) with the KL-divergence penalized-MBO objective as in (11) only (DynMO); (2) with the adversarial source critic-dependent constraint introduced by Yao et al. (2024) only (AMO); and (3) with both algorithmic components as in Dyn AMO described in Algorithm 1. We report the L1 coverage score achieved by the 128 evaluated designs as across 10 random sceds, where higher better. Bolded entries indicate average scores with an overlapping 95% confidence interval to the best performing method. Bolded (resp., Underlined) Rank and Optimality Gap (Opt. Gap) metrics indicate the best (resp., second best) given backbone optimizer.
mcan (95% confidence
Li Coverage @ 128
TFBind8
UTR
CHEMBL
Molecule
Superconductor
D D'Kitty
Rank
Opt. Gap
Dataset D
0.42
0.31
1.42
0.68
6.26
0.58
Grad.
0.16
0,20
0,21
0.42
0.0g(000)
0.00mt00
3.3
-1.44
AMO-Grad.
0.17 0.17umm
0.25
0.37
0.00
0.09
2.5
-1.42
DynMO-Grad
(.20
0.22
1,23
(22
0.00
0.00
2.8
-1.29
DynAMO-Grad.
0,36 (un4)
0,52
1.46
2.49 (l06)
6,47
5.85
1.0
1.25
Adam
0.11 (0:00)
0,22
0
0.4g
0
0,24
-1.35
AMO-Adam
0.14 th04
0.22 top
0.35
(h50
(.26
0.0g
-1.35
DynMO-Adam
0.20
(9.1)
1.10
(.4
0.27
0.0g
3.0
-1.23
DynAMO-Adam
(.33
0.55 (003)
0.39
2.40 (016)
7.06 (.73))
(9.7
1.0
1.50
BO-qEl
0.41 (iu12)
0,55
2.370009
2.11 2.11
7.84
6.61
2.8
1.70
AMO-BO-qEI
0.40 (un)
(l55
t.38
2.53
t.4
1.29
0.82
DynMO-BO-qEI
0.40
0.55
2.42
2.55
(9.0)
6.72
1.80
DynAMO-BO-qEI
(.00
0.56 (.n1)
2.47
0.04
7.87
7.92
2.02
BO-gUCB
(.42))
0.5g
2.4)))
(.52
7.7g(000)
6.6410.00)
2.8
1.77
AMO-BO-qUCB
2.39
2.52
7.37
1.340004)
3.3
0.82
DyMMO-BO-QUCB
0.39
0.55
2.40
2.52
9.06
6.64
2.5
1.77
DynAMO-BO-UUCB
0.55
2.47
2.54
7.88
7.80
1.3
2.00
CMA-ES
0.33
0.4g
2.18
(22
3.26
3.77
2.3
0.36
AMO-CMA-ES
0.31 0.3e
2.170000
183(015
3,37
3.12
2.5
0.27
DymMO-CMA-ES
0.34 mum
(..1
0.66))
(z60))42)
185(2200
0,94
3.7
-0.81
DynAMO-CMA-ES
0.40
0.56
(.82
2.54 (lus)
4.75
1.29
1.5
0.62
CoSyNE
0. m.07
(l.1
0.39
0.27
0. 1.00000
0.000000
-1.41
AMO-CoSyNE
0.12
0.22
0.53
0.14
10
0.02
-1.42
DymMO-CoSyNE
0.2g
0.32
0.33
0.7g
0.02
2.3
-1.30
DyRAMO-COSyNE
0.21 (LI))
0u18
0.64
0.43
1.85
0.94
-0.90
50

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
196
100
Besiglk Grack: Soone
Bengll Craak Score
Bestäik Craake
70
Grad. DynAMO-Girad
CMA-ES DynAMO-CMA-ES
BO-BEI
Dy0AMO-80-gE1
206
106
sin
100
200
400
sEn
190
300
600
(40
1992
Oracle Evalustion Budget K
Oracle Evaluation Budget
Orack Evalustion Budggt
102
Besigik Oraale Soove
Bonig Omcele Soore
85-
23-
Bengg
Adam
CaSyNE
BO-QUCR DyyAMO-BO-qUCC
DysAMtsAdam
DyBAMO-CaSyNE
1902
1000
Ocucle Evalisation Budget K
Orsele Evaluation Budget
Oraele Evaluation Budget k
77-
Parwic Divenity @k Score
Parwiie Divanitty (ikk Szore
Parwiie Divenity gik Scone
74-
30
75
73
Grad.
CMA-ES
80-qEl DynAMO-RO4HH
DynAMO-Gind
DynAMO-CMA-ES
200
600
620
s00
1060
200
491
610
500
1000
200
891
666
1000
Omncle Evaluation Budget
Craek Evaltation Budget
Oracle Evaluation Rudget k
30
74
Pairerde siverittklk Scare
Pairwise Diverity@k Scare
Pairwrse Diversit@kk Seare
24
22
74
A.am DynAMD-Adem
CoSyNE
BOqUUCR PynAMO-BO-UUCB
DynAMO-CuSyNE
200
500
402
800
660
1600
Oruice Evaluation Budget
Oracle Evalaation Buiget
Oraole Evaluation Budget
Figure A6. Oracle Evaluation Budget A Ablation. We vary the allowed oracle evaluation budget in Algorithm 1 between 16 and 1024, report both (first rows) Best@ 128 Oracle Score and (last two rows) Pairwise Diversity score for & final designs proposed by both DynAMO augmented and base optimizers on the TFBind8 task. We plot the mean t 95% confidence interval over 10 random seeds.
51

---

Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
Table A14. Optimization Initialization Ablation. evaluate both Sobol sequence requence-based based Top-k initialization strategies for DynAMO with Gradient Ascent and other first-order MBO methods. We report the maximum oracle score (resp., pairwise diversity score) achieved 128 evaluated designs the top (resp., bottom) table. Metries are reported mean 195% (95 seeds, where higher is better. max (D) reports the top oracle score in the offline dataset. All metrics are multiplied by 100 for easier legibility. Bolded entries indicate the higher average scores for given optimization method.
imaeva)) across 10 random
Best 128
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D'Kitty
Win Rate
Dataset
43.9
59.4
60.5
88.9
40.0
Grad. (Sobol)
90.0/4.3
80,9(12D
60,2
88,8
(6,0
65.6
3/6
Grad. (Top-k)
19.
64.0
63,3
90.1 (0.)
27.0
67.8
3/6
COMs (Sobol)
84,7
(2.4
63,3
91.4
17.3
82.g(2%)
0/6
COMs (Top-&)
93.4
67.
64.6
97.1
41,2
91,8
6/6
RoMA (Sobol)
96,510.
77.8
63.3
85,5(2.4
46,5
93.9r1.
4/6
RoMA (Top-k)
96,5
77.8
(3,3(9.0)
84,710.0
49.8(14)
95.7
6/6
ROMO (Sobol)
97.7(1.
67.0
68.3
90,
45,5
(
ROMO (Top-k)
98.1 (8.7)
(6.8(L.)
63.0
91,8%99
38.7
87.8
3/6
GAMBO (Sobol)
(3.19
(M.1
(4.4
92.
46.0/5.8
90.6 90.6
S/6
GAMBO (Top-k)
78.5
(8.3
63.g
90.6
27. 27.1
77.8
1/6
DyaAMO (Sobol)
90.3
86.2
(4
91. 2 (0.))
44,2
89,8
6/6
DynAMO (Top-k)
81.9
64,4
(3.3
90.8 (0.3
29.4(4.4
75.3
0/6
Pairwise Diversity
TFBind8
UTR
ChEMBL
Molecule
Superconductor
D' D'Kitty
Win
Dataset
33.7
42,8
50.9
87.6
6.7
77.8
Grad, (Sobol)
12,5(%.0
(.8
7.9
24.1 (12.)
0.0
0.0
3/6
Grad. (Top-k)
(,3
40,3
(3.1
28.4
m.
0.0
COMs (Sobol)
(5.5
59,3(1.D
72.6
43,9(16.5)
33,8(17)
2/6
COMs (Top-k)
66.6
(2.4
81.6(4.)
3,8
99,5
21.1
4/6
RoMA (Sobol)
3,8
5.9
1.8
70.3
8,1
4/6
RoMA (Top-k)
(
3.8
1,8
49.1
14.8
5/6
ROMO (Sobol)
(4.4
56.9
59.3
39.0
58,3 (12.)
10,
3/6
ROMO (Top-k)
62, (0.5)
57.1 (LL))
53,9/0m
48.7
51,7(317)
(5.5
3/6
GAMBO (Sobol)
15, 1i1.
10.3
(11.3)
19.6 19.6
0.3
2.6
GAMBO (Top-k)
59,2(5.0
54.1 (2.5)
79,33.4
(3.4
0.0
0.0
4/6
DynAMO (Sobol)
66.9/%9
68
77.2/21.5)
93,0
129/53)
104
6/6
DynAMO (Top-k)
(5.2(10.5)
(52)
76.8(4.)
36.4
120
85.7
52


Reviews:
[{"review_id": "Ndlhr12ghj", "reviewer": "Reviewer_kDhU", "summary": "The paper introduces Diversity in Adversarial Model-based Optimization (DynAMO), a new approach for offline model-based optimization (MBO) that aims to generate diverse and high-quality design candidates. The core idea is to frame diversity as a distribution matching problem by optimizing a KL-divergence-based objective, ensuring that the generated candidates capture the diversity inherent in the offline dataset. The challenge is not only to find high-reward designs but also to ensure diversity, avoiding solutions that cluster around a single optima. Additionally, an adversarial source critic is incorporated to prevent the surrogate model from overestimating rewards on out-of-distribution designs, improving the robustness of offline optimization.\n\n## update after rebuttal\nThe authors addressed most of my concerns. I update my rating towards weak acceptance.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer kDhU for their thoughtful comments and feedback on our proposed work, which we believe have substantially improved the quality of our manuscript. Please find our responses to your comments below.\n\n1. **(Performance According to the Median@128 Metric)** Thank you for this comment. DynAMO does indeed trade reward (according to the Median@128 metric) for diversity. Diversity enables us to find a greater subset of optimal and sub-optimal designs than baseline methods; naturally, this may result in a lower median score since DynAMO cannot simply return points near the global maximum (e.g., see **Supp. Fig. A2**). In many applications of offline MBO, it is more important to obtain a good maximum score than a good median score, and we therefore prioritize the Best@128 and pairwise diversity metrics in our evaluation of DynAMO. We highlight this potential limitation of DynAMO in our revised Limitations section, included at [this link](https://postimg.cc/YjfWXyf4).\n\n2. **(ChEMBL and UTR Tasks)** We would be happy to move the results associated with these two tasks to the Appendix in accordance with this feedback. (Notably, DynAMO's average optimality gap consistently *increases* and its average ranking stays the same or improves after excluding the ChEMBL and UTR tasks.)\n\n3. **(Normalization of Results)** All of the quality metrics reported have already been max-min normalized (0,1) according to the equation $\\hat{y}=\\frac{y-y_{\\text{min}}}{y_{\\text{max}}-y_{\\text{min}}}$ in alignment with prior work in the offline MBO literature. We multiplied the normalized metrics $\\hat{y}$ by 100 to improve the legibility of our results, as highlighted in the table captions.\n\n4. **(Good and Bad Examples in Offline RL)** We note that KL constraints are also commonly used in offline RL, where the dataset can include non-expert trajectories. Intuitively, the direction of the KL term keeps the generated samples inside the support of the offline dataset, with failing to match the offline dataset outside its support being penalized less. We can further tune the tradeoff by using the temperature hyperparameter $\\tau$ defined in Equation (6). Using a $\\tau>0$, designs that are good are upweighted in the reference distribution $p^\\tau(x)$ compared to bad designs - see **Fig. A1** for examples. As a result, we are able to use the standard intuition and techniques derived from imitation learning, and show in our work that our approach works both theoretically and empirically.\n\n5. **(Figure 1)** We thank the Reviewer for this comment. Indeed, DynAMO generates a diversity of designs, many of which are not concentrated around the global maxima. However, the point of DynAMO (and **Figure 1**) is that at least *a few* of the proposed designs are located around the different global maxima. This is in contrast with the baseline method, which proposed a batch of designs that might all be near-optimal, but are all clustered around the same optima and region of the input space. Our primary motivation of DynAMO is that obtaining a small number of diverse and high-quality designs is better than obtaining a large number of high-quality but near-identical designs. If a particular application of offline MBO does not suit this use case, then DynAMO may not be well-suited for the application at hand. We have better clarified this point in **Figure 1** of our revised manuscript.\n\n6. **(Diversity for Secondary Objectives)** To demonstrate how diversity can enable better downstream exploration of secondary objectives, we compare DynAMO-enhanced optimization methods against their baseline counterparts and report the results [here](https://postimg.cc/yDbHbzVP). We find that if an optimization method achieves a higher pairwise diversity score, it also consistently achieves a larger variance in the values of the secondary objectives, supporting the idea that secondary objectives can benefit from diversity.\n\nTo be clear, we do not claim that DynAMO will discover designs that maximize multiple objectives while only optimizating against a single objective. The goal is that by increasing diversity, we can better interrogate other design properties, which will hopefully be different from one another.\n\n7. **(Additional Baselines)** We thank the Reviewer for sharing these additional baselines with us. We have included all 5 referenced methods in our updated design [quality](https://postimg.cc/m17L4ZKS) and [diversity](https://postimg.cc/jwpxBn35) results. Our results do not significantly change with the inclusion of these additional baselines.\n\n8. **(Choice of Terminology)** Thank you for this comment. We initially chose to leverage terminology from both the offline MBO and RL literature to more explicitly highlight how DynAMO adapts techniques inspired from offline RL to the MBO setting. We have revised our manuscript to remove references to reward functions and other RL-centric terminology to better align with existing offline MBO literature.", "claims_and_evidence": "- DynAMO improves both design diversity and quality compared to baseline methods:\n\nWhile DynAMO consistently improves pairwise diversity and achieves competitive results against baselines, concerns remain regarding the actual performance improvements. For example, in the D’Kitty evaluation, DynAMO does not surpass the best designs already present in the offline dataset, contradicting the fundamental goal of offline MBO—to improve upon the given data.\nAdditionally, normalizing the results, as is standard in offline MBO literature, would likely show that DynAMO’s improvements are marginal and possibly indistinguishable from other methods. Even without normalization, results on tasks like ChEMBL are too close across methods, making it difficult to determine whether DynAMO truly provides a performance advantage.\nIf the goal is to discover multiple high-quality design modes, then a slight sacrifice in top performance might be expected. However, when analyzing Median@128, which should reflect a more consistent quality across diverse designs, DynAMO lags behind other methods. This raises concerns about whether diversity is translating into useful variety or merely scattering designs without maintaining quality. Since the goal of offline MBO is to improve upon the dataset, the results question whether DynAMO's diversity actually contributes to better design candidates or simply increases search space coverage without meaningful gains.\n\n- DynAMO is optimizer-agnostic and improves performance across all optimization strategies:   \n\nExtensive results comfirm that dynamo is compatiblewith different optimization strategies.", "methods_and_evaluation_criteria": "- Pairwise Diversity (PD) and the other discussed metrics are reasonable measures to evaluate whether DynAMO increases diversity in design generation.\n\n- Best@128 and Median@128 are standard offline MBO performance metrics and are appropriately used.\n\n- ChEMBL results are too close across methods, making it unsuitable for comparing baselines. Similarly, UTR is known to produce similar results across methods, as shown in prior work: Bidirectional Learning for Offline Infinite-width Model-based Optimization (https://arxiv.org/pdf/2209.07507) and Conservative Objective Models for Effective Offline Model-Based Optimization (https://arxiv.org/pdf/2107.06882v1).\n\n- Non-normalized results may exaggerate reported gains—preferably, results should be normalized as done in offline MBO literature.\n\n- In offline RL, KL-divergence prevents OOD actions as a regularization constraint, keeping optimization within the dataset. This setup may enhance diversity but not necessarily high-quality diverse designs, as seen in Median@128 results. The intuition from imitation learning is not valid since the offline dataset contains both good and bad examples, not only expert designs.\n\n- Figure 1 is intended to highlight DynAMO selecting diverse designs, but most appear scattered, raising questions about the trade-off between diversity and maintaining quality. The figure suggests that DynAMO generates varied designs, but they are not concentrated around the different global maxima as intended.\n\n- The paper states that secondary objectives (e.g., manufacturing cost, drug toxicity) can benefit from diversity, yet the chosen tasks do not include any secondary objectives. Would it be more appropriate to validate this claim using offline multi-objective optimization tasks, such as those discussed in Offline Multi-Objective Optimization (https://arxiv.org/pdf/2406.03722)", "theoretical_claims": "No.", "experimental_designs_or_analyses": "Yes. Please check the previous sections.", "relation_to_broader_scientific_literature": "DynAMO builds on offline MBO methods, incorporating an adversarial critic to mitigate overestimation in surrogate models. Its KL-divergence regularization aligns with offline RL techniques, which constrain policies to remain within the offline dataset distribution to prevent OOD actions. The method also relates to diversity-driven optimization, sharing similarities with quality-diversity algorithms and generative modeling approaches that promote multi-modal exploration. Additionally, DynAMO leverages distribution matching through KL-divergence, guiding optimization to maintain design diversity while staying close to the offline data distribution.", "essential_references_not_discussed": "The authors provide a strong coverage of literature across various topics; however, some works in offline MBO are missing, including:\n1. Parallel-mentoring for Offline Model-based Optimization (https://arxiv.org/pdf/2309.11592).\n2. Importance-aware Co-teaching for Offline Model-based Optimization (https://arxiv.org/pdf/2309.11600).\n3. Offline Model-Based Optimization via Policy-Guided Gradient Search (https://arxiv.org/pdf/2405.05349).\n4. Learning Surrogates for Offline Black-Box Optimization via Gradient Matching (https://arxiv.org/pdf/2503.01883).\n5. Robust Guided Diffusion for Offline Black-Box Optimization (https://arxiv.org/pdf/2410.00983).\n\n(3) is particularly relevant to the reinforcement learning section in related work as it leverages offline RL techniques to guide the search for quality designs..", "other_strengths_and_weaknesses": "Strengths:\n- Diversity is an important factor often ignored in offline MBO literature, and this paper explicitly integrates it into the optimization process.\n- Extensive discussion of the method and its components.\n- The authors shared the code.", "other_comments_or_suggestions": "- Why frame the offline optimization problem using RL terminology and reward functions instead of the standard terminology used in offline model-based optimization literature?", "questions_for_authors": "- Please check the previous sections.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EjS2erqj6e", "reviewer": "Reviewer_scBu", "summary": "This paper presents a novel approach to incorporating design diversity as an explicit objective in the offline model-based optimization problem. Specifically, the original optimization objective is modified to enhance the diversity of generated samples using a distribution matching technique inspired by recent advances in imitation learning. Additionally, a source critic constraint is introduced to mitigate out-of-distribution evaluations of the reward surrogate model. Extensive experiments demonstrate the effectiveness of DynAMO in promoting diversity while maintaining the discovery of high-quality candidates.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate Reviewer scBu for their insightful feedback and thorough evaluation of our work. We have provided our responses to your comments below and would be happy to answer any follow-up questions. Thank you!\n\n1. **(Presentation of Results)** To improve the legibility of our results, we summarize the main findings on the task-averaged Rank and Optimality Gap metrics in the [main table](https://postimg.cc/ygg6fyPt) in our revised manuscript, and move the detailed breakdown of algorithm performance by task to the Appendix for the interested reader.\n\n2. **(Diversity versus Quality)** To be clear, **we do not claim that DynAMO will propose designs of higher *quality* than other baseline methods across all settings.** (In fact, *no baseline method* is state-of-the-art across all tasks.) The goal of DynAMO is to increase the diversity of designs without significantly reducing the quality of best design proposed. Indeed, across multiple offline MBO tasks evaluated, there is often no statistically significant difference between the quality of designs proposed by DynAMO and other baseline methods according to the Best@128 evaluation metric. **However, DynAMO is consistantly state-of-the-art in producing diverse sets of designs across all settings.** We have better clarified this primary goal and use case of DynAMO in our manuscript.\n\n3. **(Equation (5) for Diversity)** To clarify, we do not want to sample designs that are truly OOD, as we cannot guarantee the correctness of these designs using an arbitrary surrogate $r_\\theta(x)$. Ideally, it would be great to discover designs that are both novel and significantly OOD, but this is not possible while guaranteeing correctness in the offline setting. Consistent with prior work (e.g., [Yao et al. NeurIPS (2024).](https://arxiv.org/abs/2402.06532)), DynAMO seeks to generate **interpolated, in-distribution points** to balance diversity with naive reward maximization. Through the regularization in Eq. (5), we aim to learn a generative policy that proposes designs with coverage of the search space similar to the offline dataset $\\mathcal{D}$. If $\\mathcal{D}$ is too small or has poor coverage, then our method would likely be less effective - we discuss this in further detail in our Limitations section of our revised manuscript.\n\n4. **(Source Critic Constraint)** The primary goal of the source critic is to penalize designs that are OOD with regards to the offline dataset $\\mathcal{D}$. If $c^*(x)$ is large, then $x$ is likely from the same distribution as $\\mathcal{D}$ and so we can trust that the prediction $r_\\theta(x)$ is a good estimate of $r(x)$ with reasonable confidence. If $c^*(x)$ is small, then $x$ is likely not from $\\mathcal{D}$ and so we penalize the reward associated with $x$ to avoid choosing the design as a proposed candidate. Note that this source critic is different from the forward surrogate $r_\\theta$ (approximating oracle $r$), and so we cannot replace $c^*$ with the forward surrogate.\n\nWe note that our use of $c^*(x)$ builds on prior literature from [Yao et al. Proc NeurIPS (2024)](https://arxiv.org/abs/2402.06532) and [Arjovsky et al. Proc ICML (2017)](https://arxiv.org/abs/1701.07875) - we refer the Reviewer to these prior works for additional discussion.\n\n5.  **(Fairness in Comparisons)** To ensure fairness in comparisons, we tuned the hyperparameters of DynAMO according to *only on its performance using the Grad. optimizer*, following the same paradigm as COMs, ROMO, and GAMBO. All hyperparameter values for all methods (including DynAMO) were then held constant for all of the other optimizers. While each of the methods (including DynAMO) might benefit from further optimizer-specific and task-specific hyperparameter fine-tuning, such an approach is not feasible in real-world applications. All of the quality metrics reported have already been max-min normalized to a (0, 1) scale according to the equation $\\hat{y}=\\frac{y-y_{\\text{min}}}{y_{\\text{max}}-y_{\\text{min}}}$ in alignment with prior work. However, we then multiplied the normalized metrics $\\hat{y}$ by 100 to improve the legibility of our results, as highlighted in the table captions where relevant.\n\n6. **(Optimization Initialization Strategy and Distribution of Oracle Objective Values)** We thank the Reviewer for suggesting an alternative strategy to improve the diversity of designs through random initialization. We compared top-$k$ and random-$k$ initialization strategies for first-order optimization methods in this [table](https://postimg.cc/SndKjsRT). For most methods, neither initialization strategy results in consistently more diverse designs (except for DynAMO, which benefits from random-$k$ initialization). Furthermore, both COMs and RoMA suffer from *lower quality of designs* proposed when a random-$k$ initialization strategy is used. **These results highlight the significance of DynAMO as a novel algorithm to consistently increase the diversity of final candidates.**", "claims_and_evidence": "The claims in this paper are well-supported by clear and compelling evidence.", "methods_and_evaluation_criteria": "The proposed method and evaluation criteria, including benchmark tasks and baselines, are well-suited to the problem at hand.", "theoretical_claims": "I have verified the correctness of the proofs for Lemma 3.1 and Lemma 3.3.", "experimental_designs_or_analyses": "I have verified the soundness and validity of the experimental designs and analyses presented in the main text, as well as the additional results in the appendix.", "relation_to_broader_scientific_literature": "This paper presents a promising direction in offline optimization with potential applications in designing more effective drugs, engineering new materials with desirable properties, and solving various scientific challenges. While previous works (COMs, ExPT, BOSS) focus solely on identifying the best designs, this paper introduces a method to balance the trade-off between optimal quality and diversity.", "essential_references_not_discussed": "No, there are no essential related works missing that are crucial for understanding the key contributions of this paper.", "other_strengths_and_weaknesses": "Strengths:\n- This paper proposes a novel approach to explicitly balancing the trade-off between reward optimality and design diversity in offline optimization.\n- The writing is clear and well-structured.\n- The results demonstrate that DynAMO enhances diversity while maintaining high-reward candidates to some extent.\n\nWeaknesses:\n- The experimental results in both the main text and appendix are poorly presented, primarily as large tables with multiple bold elements, making it difficult for readers to analyze the data.\n- While the paper shows an improvement in diversity, it comes at the cost of reward optimality. Specifically, although the mean rank appears low, DynAMO does not achieve state-of-the-art performance across all settings.", "other_comments_or_suggestions": "I suggest presenting the experimental results in a clearer and more structured manner to improve readability and analysis.", "questions_for_authors": "1. How does the proposed regularization in Eq. (5) enhance the diversity of candidate designs? According to line 159, $\\tau$-weighted probability distribution has nonzero support only at offline data points, meaning $p_D^\\tau(x)=0$ for $x \\not \\in D$. Consequently, the KL regularization does not ensure diversity of samples in out-of-distribution (OOD) regions, regardless of the choice of $\\tau$.\n\n2. How does the introduction of the source critic constraint prevent the forward surrogate model from evaluating wildly out-of-distribution inputs? Additionally, could $c^*$ in this constraint be replaced by the forward surrogate $r_\\theta$?\n\n3. There may be potential unfairness in the comparisons presented in Table 1. First, the reported baseline results using non-original optimizers may not be optimal. For instance, the COMs baseline was originally designed with the Grad optimizer, meaning its hyperparameters were tuned accordingly. When integrated with other optimizers such as Adam or CMA-ES, did the authors retune its hyperparameters? Second, since each dataset has its own value range, normalizing results to a (0,1) scale is necessary to ensure fair comparisons, especially for metrics like the Optimality Gap (Opt. Gap).\n\n4. Although DynAMO achieves a lower Pairwise Diversity score and competitive Best@128 performance compared to other methods, it exhibits poor Median@128 performance in Table A1. Does the lower Pairwise Diversity score stem from weaker performance at other percentile levels? If so, this suggests that the reward distribution of candidates has a wide range, which contradicts the goal of offline optimization—finding a distribution concentrated on high-value designs. The authors should conduct additional experiments to validate this hypothesis. Furthermore, I am curious about a straightforward approach to improving the diversity of existing baselines. For instance, COMs initializes the K-best designs from the offline dataset and then applies gradient ascent to refine them. Instead, could selecting K random designs as initialization increase the diversity of the final candidates?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "w80w0JhZbn", "reviewer": "Reviewer_DFtV", "summary": "This paper proposes a distributed matching based adversarial optimization framework (DynAMO) aimed at addressing the issue of insufficient design diversity in offline model optimization (MBO) tasks. By explicitly modeling diversity objectives as a matching problem between generative design and offline dataset distribution, this method significantly improves diversity while ensuring the quality of candidate designs, and validates its effectiveness in multiple scientific fields such as biological sequence design.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer DFtV for their thoughtful feedback and careful consideration of our work. Please find our responses to your comments below.\n\n1. **(Combination of KL Divergence and Adversarial Constraints)** Thank you for this comment - indeed, we believe that it is a *strength* of our method that DynAMO is able to leverage existing individual techniques--such as KL divergence penalization from imitation learning and adversarial constraints from the GAN literature--and combine them together in a unique way to solve a new problem (offline MBO). Namely, leveraging KL divergence penalization has not been shown in prior MBO literature, and leveraging adversarial regularization has only recently been explored by [Yao et al. NeurIPS (2024)](https://arxiv.org/abs/2402.06532). **Combining them together in a meaningful and principled way to solve the problem of offline MBO is the main novelty and contribution of our work.**\n\nRegarding the unique value of combining both techniques, we refer the Reviewer to **Supp. Tables A8-A10** in **Appendix E.2**, which shows that using both KL-divergence and adversarial regularization **together in combination** significantly improves the performance of DynAMO than when using either technique alone.\n\n2. **(Theoretical Guarantee in Section D.4)** To clarify, our primary contribution of this work is the DynAMO algorithm. The goal of the theoretical bound presented in Section D.4 is to use standard, validated techniques from the literature to show theoretically what we already observe in our presented experiments: DynAMO can improve the diversity of designs obtained using different generative optimization methods. Indeed, we use fairly standard methods and techniques to arrive at our result in Section D.4 - for example, see [Ma et al. Proc NeurIPS (2022)](https://arxiv.org/abs/2206.03023), [Huang et al. (2024)](https://arxiv.org/abs/2407.13399), and [Liang (2016)](https://web.stanford.edu/class/cs229t/notes.pdf) for additional background.\n\n3. **(Method Limitations)** Thank you for this comment. We highlight that many of our discrete MBO tasks (i.e., ChEMBL, Molecule) are problems over high-dimensional design spaces as discussed in prior work ([Maus et al. NeurIPS (2022)](https://arxiv.org/abs/2201.11872), [Yao et al. NeurIPS (2024)](https://arxiv.org/abs/2201.11872), [Trabucco et al. ICML (2021)](https://proceedings.mlr.press/v139/trabucco21a.html)). Furthermore, the DKitty task in our work includes a sparse and highly-sensitive oracle reward model as discussed in [Trabucco et al. ICML (2022)](https://proceedings.mlr.press/v162/trabucco22a.html). We have better highlighted these points in our revised manuscript, and have also included an additional paragraph explicitly discussing the challenges with optimization in these settings in alignment with the Reviewer's feedback.\n\n4. **(Setting Hyperparameters)** Thank you for this comment. We highlight that **all experimental results presented in the main text use the same hyperparameter values (e.g., $\\tau=1.0$ and $\\beta=1.0$) across *all optimizers and MBO tasks***. Furthermore, **Supp. Figures A4-A6** show that the performance of our method is robust to the choice of hyperparameters. In short, while we domain expertise and tuning of hyperparameters might improve our method further, it is by no means necessary to achieve good experimental results using DynAMO.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Yes.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "* By reparameterizing the optimization objective as a combination of weighted entropy and divergence using Lemma 3.3, a closed form solution (Lemma 3.4) is provided, which solves the algebraic complexity problem that previous studies (such as those related to chi square divergence) could not directly apply. This theoretical contribution expands the application scope of f-divergence in offline optimization, complementing broader probability distribution alignment studies such as divergence optimization in generative models.\n* D. Section 6 validated the computational feasibility of the method in large-scale design spaces through empirical comparisons (such as with the model proposed by Yu et al., 2021), filling the practical gap of model independent methods (such as Fu&Levine, 2021).", "essential_references_not_discussed": "NAN.", "other_strengths_and_weaknesses": "Strengths:\n* The experiment covers multiple scientific fields (such as Table A10) to verify the generalization ability of the method.\n* Table A10 clearly distinguishes the performance of different methods (DynMO, AMO, DynAMO) and enhances readability through confidence intervals and bold annotations.\n\n\nWeaknesses:\n* The method relies on existing techniques such as KL divergence penalty and adversarial constraints, and further evidence is needed to demonstrate the unique value of their combination.\n* The theoretical guarantee section (D.4) was not elaborated in the abstract, which may lack original breakthroughs in theoretical depth.\n* The limitations of methods in ultra-high dimensional design spaces or sparse reward scenarios have not been discussed.\n* The trade-off parameters between diversity objectives and quality objectives (such as τ value) may rely on domain experience tuning and lack universal guidance.", "other_comments_or_suggestions": "NAN.", "questions_for_authors": "See weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "afFpzqmaxA", "reviewer": "Reviewer_sKFn", "summary": "This paper introduces DynAMO - a method for offline model-based optimization whose objective is to produce a diverse distribution of various designs. The objective is clearly formulated, the method is carefully derived, and empirically evaluated. The empirical evaluation is very extensive and it is certainly impressive. The results show that the method is very strong in terms of the reward of the designs, and unusually strong in terms of design diversity. Some elements of the paper pertaining the technical clarity and the rigor of the mathematical statements could be slightly improved, but it definitely qualifies as a good paper.  \n\n## Update after rebuttal  \nThe authors addressed my comments and did extra work to communicate that to me. The proposed method is now well-, and correctly-motivated theoretically. It showcases the desired properties (performance + diversity), the empirical evaluation is extensive, and the limitations are described. I strongly advocate for accepting this paper.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer sKFn for their helpful and constructive feedback on our manuscript, and appreciate their efforts in helping us improve our work. Please find our responses below - we would be happy to discuss further and answer any follow-up questions as necessary. Thank you!\n\n1. **(Assumption in Lemma 3.1)** Thank you for this comment. The inequality in Equation (16) follows directly from monotonicity of the integral and holds even if $q^\\pi(x)$ is non-zero in a measure-zero set. If we have misunderstood your concern, please let us know.\n\n2. **(Reward Term in Eq. (5))** Thank you for this comment. We include a pure reward term in (5) because the forward surrogate model $r_\\theta(x)$ provides an important (albeit sometimes inaccurate) signal of design quality outside the dataset $\\mathcal{D}$. This strategy builds off prior work in offline MBO (e.g., [Trabucco et al. ICML (2021).](https://proceedings.mlr.press/v139/trabucco21a.html), [Yu et al. NeurIPS (2021).](https://arxiv.org/abs/2110.14188), [Yao et al. NeurIPS (2024).](https://arxiv.org/abs/2402.06532)). Without this reward term, the Shannon entropy term in **Lemma 3.3** would not appear in our derivation and so the generative policy is not explicitly encouraged to find designs that maximize against the $r_\\theta(x)$ surrogate signal.\n\n3. **(Clarification of \"Baseline\")** Thank you for this question. In general, there are two \"optimizers\" used in our experiments: (1) a ***model-training*** optimizer used to train a surrogate reward function on the offline dataset; and (2) a ***generative*** optimizer used to generate new designs by optimizing against the trained surrogate function from step (1). \"Baseline\" only refers to the latter **generative** optimizer.\n\nThe reward functions used for all the experiments in **Table 1** were trained to approximate the oracle reward function using Adam as the model-training optimizer.\n\nWe have better clarified this point in our revised manuscript.\n\n4. **(Limitations of DynAMO)** We thank the Reviewer for this comment. You are correct - our method does indeed trade reward for diversity. In many applications of offline MBO, we argue it is most important to obtain a good maximum score over a good median score. We highlight this potential limitation of DynAMO in our revised Limitations section, included at [this link](https://postimg.cc/YjfWXyf4).\n\n5. **(Algorithm 1 Presentation)** Thank you for this comment. We have revised **Algorithm 1** [here](https://postimg.cc/5jGz48wM) to improve its legibility. Regarding the while loops, the outer while loop terminates when the optimizer $a^b$ no longer improves, and is implemented as lines 281-308 of our `main.py` source code file in our Supplementary Material. The inner while loop terminates when the optimal Lagrange multiplier is found, and is implemented in lines 226-256 in the `src/dogambo/models/mlp.py` file (which is invoked in line 304 of our `main.py` source code file).\n\nWe would be happy to make any additional changes to the presentation of Algorithm 1 to improve its legibility.\n\n6. **(Table 1 Formatting)** We appreciate your careful eye to detail! For the TFBind8 optimization task using the Adam optimizer, the top performing method in terms of design quality was RoMA+ with a score of $96.5\\pm 0.0$ (lower bound on 95% CI is 96.5). However, ROMO scores $95.6\\pm 0.0$ (upper bound on 95% CI is 95.6) and GAMBO scores $94.0\\pm 2.2$ (upper bound on 95% CI is 96.2). Because the 95% CI intervals of ROMO and RoMA+ (and also GAMBO and RoMA+) are non-intersecting, ROMO and GAMBO are correctly left unbolded.\n\n7. **(Meaning of \"Myopic\")** We follow [Deisenroth et al.](https://dl.acm.org/doi/10.1561/2300000021), [Ngo](https://www.alignmentforum.org/posts/GqxuDtZvfgL2bEQ5v/arguments-against-myopic-training), and others to refer to policies that are trained according a value function with an discount factor of $\\gamma=0$ in reinforcement learning as \"myopic\". Intuitively, this means that the surrogate reward function used by the REINFORCE algorithm only considers the current input design in computing the reward, and not future possible states like in traditional RL applications. We have better clarified this point in our revised manuscript.", "claims_and_evidence": "The claims are supported by evidence. There are several theoretical claims, all of which come with mathematical proofs. Some things like a bit problematic:\n\n> Lemma 3.1: You need to make some assumptions about q(x) or loosen the statement to make it hold. q(x) could be non-zero on some measure-zero set of designs, and then your Inequality (16) in the proof does not hold.\n\n> The formulation of the objective in Equation (5) is neat. But we can prove that its solution is an EBM distribution like the one in Equation (6), but with a different temperature parameter. Thus, I do not understand why to have a separate pure reward term. Lemma 3.3 makes my point even more explicit.", "methods_and_evaluation_criteria": "The experimental design is very reasonable, and broad. The authors cover enough tasks and many baselines. The results are satisfying. I would, however, ask to clarify what is happening in the experiments: \n\n> Do I understand correctly? Does \"Baseline\" (e.g. Adam) mean the reward is trained with the Baseline optimizer and then the designs are trained with that optimizer too?", "theoretical_claims": "I have read the theorems and the proofs. Read my comments about their rigor in \"Claims and Evidence\".", "experimental_designs_or_analyses": "I confirm the experimental design is valid. I had one question about clarity (see \"Methods and Evaluation Criteria\"). I have one qualm though: \n\n> As your Table A1 shows, your method trades off reward for diversity. Maybe you get a good argmax but the quality of remaining designs, like the median, can be unimpressive. Your results in A1, especially in comparison to baseline Adam, show that you generally produce suboptimal, but very diverse, designs. It is not a deal breaker but it is an important limitation. I offer a deal: if you write it in Limitations, I will raise your score, and if you address my other concerns, I will raise your score further. But the Limitations paragraph is a condition.", "relation_to_broader_scientific_literature": "The paper is relevant for the field of offline MBO. It addresses an important problem of design diversity that is often omitted in the MBO literature.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "Algorithm 1. Could you please make it more legible? Right now it looks messy. And I doubt you have while loops inside while loops. I haven’t found it in the code either.\n\nTable 1. Adam. TFBind8. Shouldn’t ROMO and GAMBO be in bold?\n\nLine 403 (right). “Trabucco et al. (2022) used REINFORCE-style methods similar to Williams (1992) to learn a myopic sampling policy…” - what do you mean by myopic here?", "questions_for_authors": "If the limitation of the trade-off between diversity and quality is acknowledged and discussed, I will raise my score. Then, if my other concerns, regarding clarity and rigor, are addressed, I will consider raising my score further.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: **Summary**: The paper considers a new setup for offline optimization problems where the typical goal is to find designs maximizing an objective/property of interest using only a static offline dataset. The paper introduces a criterion for including diversity of generated designs as an explicit criterion in the offline optimization problem setup. The key idea is to formulate adding diversity explicitly as a distribution matching problem, using a KL-divergence penalty to encourage the distribution of generated designs to match the offline dataset. Building on top of Yao et al, the paper  incorporates an adversarial source critic constraint as well. The paper provides a theoretical analysis that leads  to a tractable Lagrangian dual formulation for the overall constrained optimization objective. 

The paper received mixed reviews. All reviewers recognized the importance of the problem setup. (Reviewers kDhU, scBu, sKFn) raised concern about the potential trade-off between diversity and quality, particularly highlighted by the Median@128 metric where the proposed method’s (DyNAMO) performance was a bit on the lower side. I found the author's rebuttal very convincing in addressing all the concerns. They acknowledged the diversity-quality trade-off and expanded the limitation section.  Reviewer sKFn even increased their scores after the rebuttal. 

Overall, I believe this paper tackles an important aspect relevant for offline optimization: generating a diverse set of high-quality solutions. The proposed DynAMO method is technically sound, combining ideas from distribution matching and adversarial learning. The empirical performance is solid and the authors share well-written code for reproducibility. **Therefore, I recommend accepting the paper.** However, I request the authors to include all the points discussed in the rebuttal period in the final camera-ready version, especially including all relevant baselines mentioned by reviewer kDhU and detailed description of hyperparameter tuning as mentioned by reviewer scBu.
Decision: Accept (poster)