Offline Model-Based Optimization by Learning to Rank
Authors: Rong-Xi Tan
Ke Xue
Shen-Huan Lyu
Haopu Shang
yaowang
Yaoyuan Wang
Fu Sheng
Chao Qian
Venue: ICLR 2025 Poster (2025)
Keywords: Offline model-based optimization
black-box optimization
learning to rank
learning to optimize
Primary area: optimization

Abstract: Offline model-based optimization (MBO) aims to identify a design that maximizes a black-box function using only a fixed, pre-collected dataset of designs and their corresponding scores. This problem has garnered significant attention from both scientific and industrial domains. A common approach in offline MBO is to train a regression-based surrogate model by minimizing mean squared error (MSE) and then find the best design within this surrogate model by different optimizers (e.g., gradient ascent). However, a critical challenge is the risk of out-of-distribution errors, i.e., the surrogate model may typically overestimate the scores and mislead the optimizers into suboptimal regions. Prior works have attempted to address this issue in various ways, such as using regularization techniques and ensemble learning to enhance the robustness of the model, but it still remains. In this paper, we argue that regression models trained with MSE are not well-aligned with the primary goal of offline MBO, which is to \textit{select} promising designs rather than to predict their scores precisely. Notably, if a surrogate model can maintain the order of candidate designs based on their relative score relationships, it can produce the best designs even without precise predictions. To validate it, we conduct experiments to compare the relationship between the quality of the final designs and MSE, finding that the correlation is really very weak. In contrast, a metric that measures order-maintaining quality shows a significantly stronger correlation. Based on this observation, we propose learning a ranking-based model that leverages learning to rank techniques to prioritize promising designs based on their relative scores. We show that the generalization error on ranking loss can be well bounded. Empirical results across diverse tasks demonstrate the superior performance of our proposed ranking-based method than twenty existing methods. Our implementation is available at \url{https://github.com/lamda-bbo/Offline-RaM}.

Published as a conference paper at ICLR 2025
OFFLINE MODEL-BASED OPTIMIZATION BY LEARNING TO RANK
Rong-Xi Xi Tan .2, Ke Xue Sunn--Huan Lyu Haopu Shang" 1.2
Yao Wang*, Yaoyuan Wang', Sheng Fu Chao Qian
1.2-
National Key Laboratory for Novel Software Technology, Nanjing University, China
2 School of Artificial Intelligence, Nanjing University, China
3 Key Laboratory of Water Big Data Technology of Ministry of Water Resources, Hohai University, China
4 College of Computer Science and Software Engineering, Hohai University, China
5 Advanced Computing and Storage Lab, Huawei Technologies Co., Ltd., China
ABSTRACT
Offline model-based optimization (MBO) aims to identify a design that maximizes black-box function using only fixed, pre-collected dataset of designs and their corresponding scores. This problem has garnered significant attention from both scientific and industrial domains. A common approach in offline MBO is to train regression-basee -based surrogate model by minimizing mean squared error (MSE) and then find the best design within this surrogate model by different gradient ascent). However, a critical challenge is the risk of optimizers (e.g., errors, i.e., the surrogate model may typically overestimate the scores and mislead the optimizers into suboptimal regions. Prior works have attempted to address this issue in various ways, such as using regularization techniques and ensemble learning to enhance the robustness of the model, but it still remains. In this paper, we argue that regression models trained with MSE are not well-aligned with the primary goal of offline MBO, which is to select promising designs rather than to predict their scores precisely. Notably, if surrogate model can maintain the order of candidate designs based on their relative score relationships, it can produce the best designs even without precise predictions. To validate it, we conduct experiments compare the relationship between the quality of the final designs and MSE, finding that the correlation is really very weak. In contrast, metric that measures order-maintaniing quality shows significantly stronger correlation. Based on this observation, we propose learning a ranking- model that leverages learning to rank techniques to prioritize promising designs based on their relative scores. We show that the generalization error on ranking loss can be well bounded. Empirical results across diverse tasks demonstrate the superior performance of our proposed ranking-based method than twenty existing methods. Our implementation available hhtttps: //t.thub..... con/ lamda bbo/ oofffffllii leleline-aee RaM
INTRODUCTION
The task of creating new designs optimize specific properties represents a significant challenge across scientific and industrial domains, including real-world engineering design (Kumar et al.
2022. Shi al., 2023), protein design (Khan et al., 2023 Kolli 2023 Chen et al. [2023b; Kim et al. 2023), and molecule design (Gaulton et al. [012 Stanton et al. (022). Numerous methods facilitate the generation of new designs by iteratively querying an unknown objective function that correlates design with its property score. Nonetheless, in practical scenarios, the evaluation of the objective function can be time-consuming, costly, or even pose safety risks (Dara et al. 2022). To identify the next candidate design using only accumulated data, offline model-based optimization (MBO: Trabucco et al.) 2022) has emerged as a widely adopted approach. This method restricts access to offline dataset and does not allow for iterative online evaluation, which, however,
*Correspondence Chao Qian <ggianc@nju.edu.cnn

---

Published as a conference paper at ICLR 2025
(6)
Offiie Delovtt Design Candidate
(b)
Offllie Datzeet Design Candidate
400 -6or)
Ha's r0")
Soore
Sooee
Sumogade Moddl fo Cround-rrth Punction
Suntogate Model fo Ground-uuth Function
P1
Design
Design
Figure I: Illustration of (a) OOD issue of regression-baseee models and (b) urder-preserving ranking- based models. In (a), the regression-based method searches into suboptimal regions. Prior works focus on high OOD-MSE, while in this work, we point out that is caused by the OOD error in preserving order. In (b), although the surrogate model also has high OOD-MSE, can maintain the order, thus resulting in good design candidates.
also results in significant challenges. A common strategy, referred to as the forward method, entails the development of a regression- based surrogate model by minimizing mean squared error (MSE), which is subsequently utilized to identify the optimal designs by various ways (e.g., gradient ascent).
The main challenge of offline MBO is the risk of out-of-distribution (OOD) errors (Kim et al. 2025). i.e., the scores in OOD regions may be overestimated and mislead the gradient-ascent optimizer into suboptimal regions, as shown in Figure [a). Thus, overcoming the OOD issue has been the focus of recent works, such as using regularization techniques (Trabucco et al. 2021 Fu & Levine [202]] Yu et al. 2021 Chen et al. 2022 Qi et al. 2022] Dao et al. (022bb) and ensemble learning Yuaan et al.
2023
Chen al.
2023a to enhance the robustness of the model, but it still remains.
Recent studies (Hoang et al. 2024) have pointed out that value matching alone is inadequate for offline MBO. In this paper, we conduct a more thorough and systematic analysis on this view. We aim to answer the key question: "Is MSE good metric for offline MBO?" Consequently, we find through experiments that the relationship between the quality of the final designs and MSE in the OOD region (denoted as OOD-MSE) is weak, which underscores the need for more reliable evaluation metric.
Next, we reconsider the primary goal of offline MBO, which seeks to identify the optimal design over the entire design space. Intuitively, this process does not require exact score predictions from the surrogate model; rather, it demands that the model accurately discerns the partial ordering of designs. As shown in Figure [[b), if surrogate model can maintain the order of candidate designs based on their relative score relationships, it can produce the best designs even without precise predictions. We prove the equivalence of optima for order-preserving surrogates, and introduce ranking-related metric, Area Under the Precision-Cove Coveerae Curve (AUPCC), for offiin MBO, which shows a significantly stronger correlation with the final performance.
Based on this observation, we propose learning a Ranking- Model (RaM) that leverages learning to rank (LTR) techniques to prioritize promising designs based their relative scores. Our proposed method has three components: 1) data augmentation to make the offline dataset align with LTR techniques; 2) LTR loss learning train the RaM: 3) output adaptation to make gradient ascent optimizers work well in RAM. We show that the generalization eттоr on ranking loss can be well bounded, and conduct experiments on the widely used benchmark Resign-Bence (Trabuceo) et al., (022). Equipped with two popular ranking losses, i.e., RankCosine (Qin et al.) 22088) and ListNet (Cao et al. (007), our proposed method, RaM, performs better than state-of-the- MBO methods. Ablation studies highlight the effectiveness of the main modules of RaM. We also examine the influence of different ranking loss, and demonstrate the versatility of ranking loss, bringing improvement even by simply replacing the MSE loss of existing methods with ranking loss.
offline
The contributions of this work are highlighted in three key points:
1) To the best of our knowledge, we are the first to indicate that MSE is not suitable for offline MBO.
2) We show that the ranking-related metric AUPCC is well-aligned with the primary goal of offline MBO, and propose a ranking-based method for offline MBO.
3) We conduct comprehensive experiments across diverse tasks, showing the superiority of our proposed ranking- based method over a large variety of state-of-the-ee art offline MBO methods.
2

---

Published as a conference paper at ICLR 2025
2
BACKGROUND
2.1
OFFLINE MODEL-BASED OPTIMIZATION
Given the design space X' C R where d is the design dimension, offline MBO (Trabucco et al. Kim et al. 2025 Qian et al. 2025 Xue et al. 2024) aims to find a design x" that maximizes arg x fxxx,, using only a pre-collected offline dataset DD without access to online evaluations. That is, an offline MBO algorithm is provided only access to the static dataset D [(x,, 9;),, where x, represents specific design (e.g.. a superconductor material), and y, = fxx,) represents the target property score that needs to be maximized (e.g., the critical temperature of the superconductor material).
2022
black-box objective function S, i.e.,
The mainstream approach for offline MBO is the forward approach, which fits a surrogate model, typically deep neural network so X - R, parameterized by 0, to approximate the objective function f in a supervised manner. Prior works (Trabucco et al. 2022 Fu & Levine 2021 Yu et al. 2021: Qi et al.) [0222] Yuan et al.) 2023) Chen et al.) [002aa] Hoang et al. 2024 Dao et al. 2024b)
learn the surrogate model by minimizing MSE between the predictions and the true scores:
.V arg min Σxn, (fo(x.) - wi) /N.
With the trained model fe. the final design can be obtained by various ways, typically gradient ascent:
X Xx + 17 Vxfo(x) x=x; for E {0, I........ 1}.
(I)
where 1) the search step size. T is the number of steps, and XT serves as the final design candidate to output. However, this method is limited by its poor performance in duitrf-distribution (OOD)
regions, where the surrogate model fe may erroncously overestimate objective scores and mislead the gradient-ascent optimizer into sub-optimal regions. There have been many recent efforts devoted
to addressing this issue, such using regularization techniques (Fu & Levine 2021 Trabucco et al.
2021; Yu et al. 2021] Dao et al., 20244 a) and ensemble learning (Yuan et al. 2023 Chen et al. 2023a)
enhance the robustness of the model.
Another type of approach for offline MBO is the backward approach, which typically involves training conditioned generative model pe (xly)) and sampling from it conditioned on a high score. For example, MINs [Kumar& & Levine 2020) trains an inverse mapping using a conditioned GAN-like model (Goodfellow et al. 2014): DDOM (Krishnamoorthy
al. (0223) directly parameterizes the inverse mapping with conditioned diffusion model (Ho et al.] 2020); BONET (Mashkaria et al.. 2023) uses trajectories train an autoregressive model, and samples them using heuristic.
A comprehensive review of offline MBO methods is provided in Appendix A due to space limitation. In this paper, we point out that the regression- based models trained with MSE are not well-aligned with offline MBO's primary goal, which is to select promising designs rather than predict exact scores. Intuitively, off MBO does not require exact score predictions from the surrogate model; rather, demands that the model accurately discerns the partial ordering of designs, which naturally aligns with the learning to rank (LTR) framework introduced in Section 2.2
2.2 LEARNING TO RANK
LTR aims to learn an optimal ordering for a given set of objects (e.g., designs in offline MBO), and has applications across various domains, including information retrieval (Liu) 2010 Li| 2011).
recommendation systems (Karatzoglou et al.] 2013), and language model alignment (ong et al.
supervised learning task. Given the training E x x R"), }, where X' is the object space, X is a list of n objects to be ranked, each denoted by x; E X, and y a list of n corresponding relevance labels yi R, the goal of LTR is to learn a ranking function that assigns scores to individual objects and then arranges these scores in descending order to produce a ranking. Formally, LTR aims to identify a ranking score function sg: X + I., parameterized by 0. Let (X) [se(x]) s0(2)).... ..., sx(x))) and
2024; Liu al. 2024). It is typically formulated data DR = ((X,)) y) (X.y)
we can optimize the model by minimizing the empirical loss:
[(so) -EX 1
3

---

Published as a conference paper at ICLR 2025
where I(.) is the loss function applied to each list of labels and predictions. Depending on their approach to handling ranking loss, LTR algorithms are categorized into three types: 1) Point- wise (Crammer & Singer (200]): Treat ranking as regression or classification problem individual objects; 2) Pairwise (Köppel et al. 2019): Transform ranking into a binary classification problem on
object pairs; 3) Listwise (Xia al. 2008): Directly optimize the ranking of the entire list of objects.
METHOD
In this section, introduce our ranking- aunking-based surrogate models for offline MBO. We first analyze in detail the goal of offline MBO and aim to answer the critical question, "Is MSE good metric for offline MBO?" in Section 3.1 Consequently, we find that MSE is not suitable metric, and thus introduce a better one in Section[3.2] i.c., Area Under the Precision-Coverage Curve (AUPCC), which related to ranking. This motivates us propose a framework based on LTR solve offline MBO in Section 3.3, Furthermore, we show that the surrogate model based on LTR methods can have a good generalization error bound, which will be shown in Section 3.4
3.1 Is MSE A GOOD METRIC FOR OFFLINE MBO?
An ideal metric should be able to accurately assess the goodness of a surrogate model, i.e., the better the metric, the better the quality of the final design obtained using the surrogate model. As shown in Eq. (1). Xy, which approximately maximizes the surrogate model fo by gradient ascent, serves as the final design to ouiput. During the optimization process, it will inevitably traverse the OOD region. Therefore, the performance of the surrogate model in the OOD region will significantly impact the performance of offline MBO. Unfortunately, previous works (Trabucco et al. 202 2022) have shown that the regression- based models optimized using MSE often result in poor predictions in the OOD region. i.e., the MSE value in the OOD region (denoted as OOD-MSE) can be very high, and thus many methods have been proposed to decrease OOD-MSE (Fu & I Levine 2021 Chen et al.
2023a| Yuan et al., (023) or avoid getting into OOD regions (Trabueco
[021] Yu et al. Yao et al. 20244. In this paper, however, we indicate that even if OOD-MSE is small, the final
1
performance of offline MBO can still be bad. That is, the relationship between the quality of the final
designs and OOD-MSE is weak. In the following, we will validate this through experiments
To analyze the correlation between the OOD-MSE of a surrogate model and the score of the final de- sign candidate obtained by conducting gradient ascent on the surrogate model, we select five surrogate models: gradient-ascent baseline and four sffte-of-the-aee forward approaches. COMs (Trabucco)
et al. 202T). 1OM (Qi et al. 2022). ICT (Yuan et al. 2023), and Tri-Mentoring (Chen et al., (023a)) We follow the default setting as in Chen et al. (2023a); Yuan al, (2023) for data preparation and model-inner search procedures. To construct an OOD dataset, we follow the approach outlined in Chen et al. (2023a), selecting high-scoring designs that are excluded from the training data in Design-Bench (Trabucco al. (022). Detailed information regarding model selection, training and search configurations, and OOD dataset construction can be found in Appendix [E.I] We train the surrogate models, evaluate their performance using various metrics (e.g., MSE) on the OOD dataset, and obtain the final design with its corresponding ground-truth score under eight different seeds. Subsequently, we rank the OOD-MSE values in ascending order, and rank the 100th percentile scores of the final designs in descending order. To show the correlation between OOD-MSE and the final score, we create scatter plots of the two rankings and calculate their Spearman correlation coefficient.
The left two subfigures of Figure show the scatter plots on a continuous task, D' D'Kitty (Ahn et al. 2020), and discrete task, TF-Bind-8 8 (Barrera et al., 2016). Both scatter plots exhibit highly dispersed data points, with no clear overall trend or strong clustering. showing no consistent pattern in their distribution. This scattered nature of the data points is also reflected in the low Spearman correlation coefficients (0.23 for D' D'Kitty and -0.24 for TF-Bind-8). indicating weak correlations between OOD-MSE rank and score rank in both tasks. These results demonstrate that OOD-MSE is not a good metric for offline MBO, underscoring the need for more reliable evaluation metric.
3.2
WHAT IS THE APPROPRIATE METRIC FOR OFFLINE MBO?
As mentioned before, an intuition of offline MBO is that the goodness of surrogate model may depend on its ability to preserve the score ordering of designs dictated by the ground- -tuth function. We substantiate this intuition through the following theorem.
4

---

Published as a conference paper at ICLR 2025
D' (Spearman 0.23)
D'Kitty (Spearman corr: 0.64)
30
Score Rank
Score Rank
20-
20-
0-
10
20
30
40
10
20
50
40
OOD-MSE Rank
OOD-AUPCC Rank
Goddett-Aass
COMs
J0M
ICT
Tei Merkurigg
Ordiin-Ascm
COMs
10M
ICT
Te-Momorinn
TF-Bind-8 (Spearman corr: -0.24)
TF-Biad-8 (pearmon 0.52)
40
30-
30
Score Rank
30-
20
10
10
20
30
40
30
OOD-MSE Rank
OOD-AUFOC
Graäets-Aac COCL 10M KCT
Tri-eentrrix
Grndenn-Ascet
COMs
Mentoringg
Figure 2: Scatter plots of five surrogate models (each trained using eight seeds) on the two tasks of D' D'Kitty and TF-Bind-8, where the y-axis denotes the rank of the 100th percentile score, and the r-axis denotes the rank of the metric in the OOD region, i.c., OOD-MSE or OOD-AUPCH The Spearman correlation coefficients are also calculated, as shown in the title of each subfigure.
Theorem (Equivalence of Optima for Order-Preservee Preserving Surrogates). Let fe be surrogate model and f the ground-truth function. A function h. TR TR. is order-preserving, if U22 E R. y1 < y22 iff h(y)) < h(yz). If there exists an pnder-preserving h such that fo(x) = (x)) Vx, then finding the maximum of f is equivalent to finding that of fo. ie., arg max f(x) = arg maxxex fo(x).
Proof. Suppose x* E arg maxx fxx). For any x, we have f(x') 2 f(x). Since h is order- preserving, we have h(f(x))) 2 M(f(x) for all x. Thus, given fo(x) h((()),, we have fo(x") 2 (x) for all x. Therefore, x" arg maxx fe(x), i.e., arg maxx f(x) S arg max (x)). Note that since h is strictly increasing, it is bijective and thus has an inverse function h- which is also strictly increasing. With h-1 the reverse implication follows similarly, proving the equivalence. O
Theorem [ shows that good surrogate model needs to maintain order- wnder-preserving mapping from the ground-truth function. Besides, in the practical setting of offline MBO, the standard procedure is select the top-k designs (e.g.. k 128), which maximize the surrogate model's predictions, for evaluation (Trabucco et al., 2022). Thus. we introduce novel metric. Area Under the Precision- Coverage Curve (AUPCC) Definition [ for offline MBO assess the model's 's capability in identifying the top-k ones from set of candidate designs.
Definition (AUPCC for Offline MBO). Consider surrogate model fo and ground-truth, f. Given a dataset D = {(x;, y:) Y denote {x,)), as fo (Du), and ( as (Do). Let (S) denote the set of the k largest elements in set S. For each k E {1,2, .., N).
Precision @k (fo
top, (f(Do)) topp (f(D))
(top: n top, (f(oa)) k
Coverage @k (fo n Dol/|Pol k/N.
The Precision- Prevesion-Coovragge curve is obtained by plotting Precision @k against Coverage @k for all values of k. Then, the AUPCC defined as the area under this curve:
N-1
(Coverage @(k + 1) - Coverage @k)- Precision @(k: + 1) + Precision @k
AUPCC R
2
5

---

Published as a conference paper at ICLR 2025
The AUPCC metric for offline MBO can effectively evaluate model's ability to identify top-k designs with varying & and thus the ability to preserve order across the entire design space, so naturally serves ranking-related metric. A higher AUPCC value indicates better performance in ranking and selecting better designs. We visualize the correlation between OOD-AUPO (i.e., the AUPCC value in the OOD region) rank and score rank in the right two subfigures of Figure 2 where the rank of OOD-AUPCC is obtained in descending order. In contrast to the OOD-MSE results, the scatter plots of OOD-AUPCC exhibit clear upward trends, with data points clustered more tightly around the diagonal compared to their OOD-MSE counterparts. This improved correlation is also verified by the substantially higher Spearman correlation coefficients, 0.64 for D' Kittty and 0.52 for TF-Bind-8.
To further validate the reliability of AUPCC compared to MSE, we conduct a quantitative analysis. incorporating another three tasks, Superconductor ( (Hamidieh) 2018) and Ant (Brockman et al. 2016) in continuous space, and TF-Bind-10 (Barrera et al. (016) in discrete space. We evaluate the metries in the OOD regions and the final scores, and calculate Spearman correlation coefficients between the two rankings, following the same approach as in our previous analysis. The results in Table [] demonstrate the superior performance of OOD-AUPCE compared to OOD-MSE in correlating with the 100th percentile score across various offline MBO tasks. OOD-AUPCE consistently shows stronger correlations than OOD-MSE, with an average improvement of 0.364 in correlation strength. Notably, OOD-AUPCC achieves positive or significantly improved correlations even in the tasks where OOD-MSE shows negative correlations, such as Superconductor and TF-Bind-8 tasks. Coupled with Theorem D which establishes the relationship between a model's order- snder-preeervig capability and its final performance, the consistently stronger empirical correlations confirm that OOD-AUPCC is indeed a more effective and reliable metric than DOD-MSD for evaluating the performance of a surrogate model in offline MBO. In the next section, we will discuss how to use LTR techniques to optimize the AUPCC, thus to obtain high-scoring designs.
Table 1: Comparison between Spearman correlation coefficients of OOD-MSE and OOD-AUPCC with respect to the 100th percentile score.
Ant
D' D'Kitty
Superconductor
TF-Bind-8
TF-Bind-10
OOD-Metric
Coef.
Gain
Coef.
Gain
Coef.
Gain
Coef.
Gain
Coef.
Gain
OOD-MSE
0.161
0.243
-0.116 0.101
-0.239 0.520
-0,573
OOD-AUPCC
0.257
+0.096
0.503
+0.260
+0.217
+0.759
-0.087
+0.486
3.3 OFFLINE MBO BY LEARNING TO RANK: A PRACTICAL ALGORITHM
In this section, in order optimize AUPCC for the surrogate model, we design a novel framework for offline MBO based on LTR, as shown in Algorithm [ augmentation; 2) LTR loss learning; 3) output adaptation.
which consists of three parts: 1) data
Data augmentation. In LTR tasks, the training set DR typically requires a list of designs as features. However, the offline dataset D in offline MBO is not directly structured in this manner, thus the LTR loss functions cannot be directly applied. A naïve approach to address this issue is to treat each batch of training data as a list of designs to be ranked, with the batch size determining the list length. However, this method has its limitation since each design in the training data appears in only one list during one single epoch, which is unable to analyze relationship with other designs that are not in the list. To address this limitation, we propose a simple yet effective data augmentation method. We randomly sample m design-score pairs {(x,, yı) m from D, and concatenate them to form a design list X = [x1, X x,. and its score list y = [y1, 92....... yml T then repeat this step for n times to construct dataset DR = yx) }= for LTR modeling. We will discuss the setting of n. and m in Section 4.1, and show the benefit of data augmentation over the naive approach Section 4.2
LTR loss learning. In Section 3.2
we have discussed that AUPCC is ranking-related metric, and thus we can use the well-studied ranking loss (Li) 201 from the field of LTR to optimize the AUPCC on the training distribution, so to generalize to the OOD regions. We study wide range of ranking losses, including pointwise (Crammer & Singer 2001). pairwise (Köppel et al. 2019). and listwise (Xia et al. 2008) losses. Here we take RankCosine (Qin et al. 2008). pairwise loss, and ListNet (Cao et al. 2007). a listwise loss, for example. The idea of RankCosine is to measure the difference between predicted and true rankings using cosine similarity, operating directly in the

---

Published as a conference paper at ICLR 2025
Algorithm 1 Offline MBO by Learning Rank
Input:
Offline dataset D, number of lists in the training data, length m of each list, training steps No. ranking loss I, learning rate A, search steps T, search step size 1).
Output: The final high-scoring design candidate.
1: Initialize fo: Initialize Xn as the design with the highest score in D; Construct training data via data augmentation
2: Initialize DR < 0;
3: for 1 to do
4: Randomly sample m design-score pairs (x, v) from D;
5: Add (X; y) to DR. where X [x1, X2 Xm] and y [y), 02..... ymm
6: for i= === to Ng do
b Use LTR loss to train the surrogate model
7:
Calculate the ranking loss: ((0) = E (),,) (y,
where fe (X) = (..)) .... fx())
8:
Minimize ()) with respect to 0 using gradient update: - -VoC(()
Calculate the in-distribution predictions y = (ū fo(x), (x,8) E ED]; Conduct gradient ascent via oulput adaptation 10: Obtain statistics of the in-distribution predictions: jı = mean(ÿ), ē std(ý);
9:
for 0 to 7 1 do
12: Update X,+1 via gradient ascent: X X + V (x) X=xI
where (x) = (fe(() //)6
Return XT
score space. Formally, given a list X of designs and the list y of their corresponding scores, let be the predicted scores. The RankCosine loss function is: fo(X) = fo(x),, Sx
IRankCosine (y, (x)) = = y y (x) -I.((()))))
The idea of ListNet is to minimize the cross-entropy between the predicted ranking distribution and the true ranking distribution, which is defined as:
m
exp(y;)
exp (xp(fo(x;)) E= cxp(ff(x:))
ListtNe (y, fo())
log 1 exp(y:)
Em m
We provide detailed description of other ranking losses in Appendix and compare their effectiveness We also provide detailed information of model training in Section 4.1
for offline MBO in Section 4.2
Output adaptation. The surrogate model trained with ranking loss has a crucial issue for the hyper- parameter setting of gradient-ascent optimizers. Unlike MSE, which aims for accurate prediction of target scores, ranking losses do not require precise estimation of target scores. This shift in objective may lead to significant changes in the scale of model predictions, and thus impact the magnitude of gradients, making it challenging to determine appropriate values for the search step size yj and the number T' search steps in Eq. ( Moreover, different ranking losses can result different output scales, which necessitate careful pyper-pprameter tuning for a specific loss.
Notably, the scores in the training data for regression-based models have statistical characteristic of zero mean and unit standard deviation after z-score normalization (Trabucco et al., [202T] (022). and the trained regression-buset distribution. Consequently, to mitigate the impact of varying scales across different loss functions and to ensure a fair comparison with the regression- based models, we normalize the predictions of the ranking model after it is trained. Specifically, we first apply the trained model to the entire training set and calculate the mean value ğ and standard deviation ō the resulting predictions. Subsequently, we use jí and ō to apply z-score normalization to the model's prediction. Such normalization enables us to directly use the setting of and T as in regression- based models. That is, we compute the gradient of the normalized predictions with respect to x, and use the default sarem-aaraettess in Chen et al. (2023a); Yuan et al. (2023) to search for the final design candidate. We will examine the effectiveness of using output adaptation in Section 4.2
model will try to preserve these statistical properties within the training

---

Published as a conference paper at ICLR 2025
3.4 THEORETICAL ANALYSIS
In the previous subsections, we have indicated the importance of preserving the score order of designs, and proposed to learn surrogate model by optimizing ranking losses. Here, we further point out that the generalization error can be well bounded in the context of LTR. Note that the generalization of LTR has been well studied (Agarwal al., 2005 Lan et al.) 2009 Chen et al. 2010; Tewari & Chaudhuri 2013), which is mainly analyzed by the Probably Approximately Correct (PAC) learning theory (Cucker Smale, (00T) and Rademacher Complexity (Bartlett & Mendelson, 2003). By
leveraging these existing generalization error bounds, we provide theoretical support for our approach of applying LTR techniques for offline MBO.
Formally, assume that we have an i.i.d. training data DR = (()., where X; E Xmm consisting of m designs, and y; E IR"". Given a ranking algorithm A (e.g., RankCosine or ListNet), its loss function (a(f: X, y) is normalized by y)/Zx. where ZA is normal- ization constant (c.g., RankCooine = 1). The expected risk with respect to the algorithm A is defined as (f) = Jxm Rm (a(fi P(d,X, dy), and the empirical risk is defined as (R; = #/4 Ex (aff;X,, Let F be the ranking function class, and Theorem 2 gives an upper bound on the generalization error sup (PPPE=((R,())
Rd, (x))..
Theorem 2 (Generalization Error Bound for LTR (Lan et al. 2009)). Let Φ be an increasing and
strictly positive transformation function (e.g., ((2) (z)( Assume that: 1) V/X X,
|||| S M; | < B}.
2) the ranking model f to be learned is from the linear function class FF = {x + + w x
Then with probability 1-6 8, the following inequality holds:
SuPJEr (fi (f)) S 4BM Col 21n (2/8)/n.
where: 1) A stands for specific LTR algorithm; 2) N()) = SuPsEI-BM.. BACEl-BMMBBB.. (=), which is an factor measuring the smoothness of o; 3) CΛ(g) an factor; e.g., Citak Cosine (0) = /m/ (2 BM)).
We will introduce some settings of ó and the corresponding (ō) and CΛ(ò) in Appendix B We can observe from the inequality in Theorem 2 that the generalization error bound vanishes at the rate O11 / /n), since (d) and N(Ø) are independent of the size of training set. In Appendix C
discuss probable approaches and difficulties in extending the theoretical analysis, identify a special case where the pairwise ranking loss is more robust than MSE, and analyze it via experiments.
4 EXPERIMENTS
In this section, we empirically compare the proposed method with a large variety of previous offline MBO methods on various tasks. First, we introduce our experimental settings, including five tasks, twenty compared methods, training settings, and evaluation metrics. Then, we present the results to show the superiority of our method. We also examine the influence of using different ranking losses, and conduct ablation studies to investigate the effectiveness of each module of our method. Furthermore, we simply replace MSE of existing methods with the best-performing ranking loss, to demonstrate the versatility of the ranking loss for offline MBO. Finally, we provide the metries, OOD- MSE and AUPCC,, in the OOD regions to validate their relationship with the final performance. Our implementation is available at wtlhttps://giteeee
com lamda -bbbbbbbboo oofffffiinne RaM
4.1 EXPERIMENTAL SETTINGS
Benchmark and tasks. We benchmark our method on Design-Bench tasks (Trabucco et al. 2022).
including three continuous tasks and two discrete tasks The continuous tasks include: 1) Ant Morphology (Brockman et al.] (016): identify an ant morphology with 60 parameters to crawl quickly. 2) D'Kitty Morphology (Ahn et al., 2020): optimize D' Kitty morphology with 56 parameters to crawl quickly. 3) Superconductor (Hamidich) 2018): design 86-dimensional superconducting material to maximize the critical temperature. The two discrete tasks are TF-Bind-8 and TF-Bind- 10 (Barrera et al., 2016): find DNA sequence of length 8 and 10, respectively, maximizing binding affinity with a particular transcription factor.
Following recent works (Yun et al. 2024 Yu et al. 2024 we exclude three tasks from Bench,, and provide detailed explanations in Appendix
8

---

Published as a conference paper at ICLR 2025
Compared methods. We mainly consider three categories of methods to solve offline MBO. The first
category involves baselines that optimize a trained regression- based model, such as BO-qEI Garnett
2023; Shahhiari et al. 2016), CMA-ES (Hansen 2016), REINFORCE (Williams 1992). Gradient
Ascent and its variants of mean ensemble and min ensemble. The second category encompasses
backward approaches, including CbAS (Brookes al., 2019). MINs (Kumar & Levine 2020).
DDOM (Krishnamoorthy et al. 2003), BONET (Mashkaria et al. (202)) and GTG (Yun et al.) 2024).
The third category comprises recently proposed forward approaches, which contain COMs (Trabucco)
et al.
2021). RoMA (Yu et al., 2021). IOM (Qi et al. 2022). BDI (Chen et al. 2022), ICT ( (Yuan
ct al.
2023). Tri-Mentoring (Chen et al. 2023a). PGS (Chemingui et al. 2024). FGM (Kuba et al.
2024b), and Match-OPT (Hoang et al. 2024)
Training settings. We set the size n of training dataset to 10, 000, and following LETOR 4.0 QQn
& Liu 2013 Qin et al., 2010b), a prevalent benchmark for LTR, we set the list length m 1000. To make fair comparison to regression-
methods, following [Trabucco et al. (2021 2022):
Chen et al. (2033a); Yuan et al. (2023). we model the surrogate model fo as a simple multilayer perceptron with two hidden layers of size 2048 using PyTorch (Paszke al.] 2019). We use ReLU as activation functions. RankCosine (Qin et al. 2008 and ListNet (Cao et al. (007) will be used as Two main loss functions in our experiments. The model is optimized using Adam (Kingma & Ba 2015) with a learning rate of 3 x 10 and a weight decay coefficient of 1 x 10-5. After the model trained, following Chen et al. (023a): Yuan al. (2023), we set 1) = 10-3 and T = 200 for continuous tasks, and 17 = x 10 and T' 100 for discrete tasks to search for the final design. All experiments are conducted using eight different seeds. Additional training details are provided in Appendix E.4
Evaluation and metrics. For evaluation. we use the oracle from Design- Bench and follow the protocol of prior works (Trabucco et al. 202T] 2022). That is, we identify k = 128 most promising designs selected by an algorithm and report the T00th percentile normalized ground design score y is normalized via computing (y /mii)/((mnn) - ymmax), where ymin and ymaxx denote the lowest and the highest scores in the full unobserved dataset from Design-Bence We also provide the 50th percentile normalized ground-trute -tuth results in Appendix F.1
score. A
4.2 EXPERIMENTAL RESULTS
Main results. In Table 2 we report the results of our experiments, where our method based on Ranking Model is denoted as RaM appended with the name of the employed ranking loss. Among the compared 22 methods, Mann-een and RaM-ListNet achieve the two best average ranks, 2.7 and 2.2, respectively, while the third best method, BDI, only obtains an average rank of 5.9. We can observe that RaM-RanCCosine performs best on one task, TF-Bind- 10, and is runner-up on two tasks. Superconductor and TF-Bind-8; and RaM-ListNet performs best on two tasks, D Kitty and Superconductor. These results clearly demonstrate the superior performance of our proposed method.
Influence of different ranking loss. We compare RaM with various ranking losses: Sigmoid- CrossEntropy (SCE), BinaryCrossEntropy (BCE), and MSE for pointwise loss; RankNet (Burges
2018). and RankCosine (Qin et al. 2008 for pairwise loss: Softmax (Cao et al. 2007 Bruch et al. [019a] ListNet (Cao et al. 2007). ListML. (Xia et al., 2008), and ApproxNDCC CG (Qin et al. 2010a Bruch et al., (019b) for listwise loss. The results in Table 8 in Appendix [22 show that ListNet is the best-performing loss with an average rank of 2.0 over 10 losses, and RankCosine is the runner-up with an average rank of 3.2.
et al. 2005). LambdaRank (Burges et al., 2006 Wang et al.,
Ablation of main modules. To better validate the effectiveness of the two moduels, data augmenta- tion and output adaptation, of our method, we perform ablation studies based on the top-performing loss functions shown in Table X MSE for pointwise loss, RankCosine for pairwise loss, and ListNet for listwise loss. The results in Table 9 in Appendix F.3 show that for each considered loss, RaM with data augmentation performs better than the na've approach which treats a batch of the dataset as list to rank. The results in Table show the benefit of using output adaptation. We also examine the influence of the list length m, as illustrated in Appendix F.4
DDue the lack of open-source implementations or inapplicability for comparison, we exclude NEMO (Fu Levine 2021), BOSS (Dao et al. 20246), DEMO Yuuan et al. 2024) and I.F.O Yu et al. 2024). Detailed
explanations are provided in Appendix E.3
Note that MSE is regression loss, which thus be viewed as pointwise ranking loss.

---

Published as a conference paper at ICLR 2025
Table 2: 100th percentile normalized score in Design-Bench, where the best and runner-up results on each task are Blue and Violet. D(best) denotes the best score in the offline dataset.
Method
Ant
D'Kiity
Supereonductor
TF-Bind-8
T--Bind-ll 10
Mean Rank
D(best)
0.565
0.884
0.400
0,439
0.467
BO-qEI
0.812 = 0.000
0.896
0.000
0.382 = 0.013
0.802 = 0.081
0.628 = 0.036
18.0/
CMA-ES
1.712 + 0.754
0.725
0.002
01.663 4 0.042
0.944 0.017
0.641 a 0.036
11,4/ 22
REINFORCE
0.248 + 0.039
0.541
= 0.19
0.478 : 0.017
0.935 + 0.049
0.673 + 0.074
14.0/ 22
Grad. Ascent
0.273 = 0.023
0
0.510 = 0.028
0,969 = 0.021
0.646 = 0.037
11.6/22
Grad. Axcent Mean
0.306 + 0.053
0.875
1755=00.00 0.024
0.500 + 0.019
0,985 0.008
0.633 + 0.030
11.2/22
Grad. Ascent Min
0.282 + 0.033
0.884 1 0.018
0.514 + 0.020
0.979 + 0.014
0.632 +: 0.027
11.5/22
CbAS
0.846 + 0.032
0.896 0.009
0.421 * 0.049
0.921 0.046
0.630 + 0.039
15.5/
MINS
0.906 0.024
0.007 0.939
0.464 1: 0.023
0,910 0.051
0.633 0.034
13.0/22
DDOM
0.908 = 0.024
0.930=0.000
0,452 = 0.028
0.913 = 0.047
0,616 0.018
14.6/
BONET
0.921 = 0.031
0.949 = 0.016
0.390 + 0.022
0.798 + 0.123
0.575 * 0.039
15.1/22
GTG
0.855 0.044
0.942 0.017
0.480 +: 0.055
0.910 0.040
0.619 0.029
13.9/22
COMs
0.916 1 0.026
0,949
0.016
0.460 1 0.040
0,953 0.038
0.644 0.052
9,5
RoMA
0.430 = 0.048
0.767 0.031
0,494 = 0.025
0.665 0.000
0,553 0,000
18.3/22
1OM
0.889 # 0.034
0.928 0.008
0.491 * 0.034
0.925 + 0.054
0.628 + 0.036
13.1/22
BDI
0.963 = 0.000
0,000
0.508 1 0.013
0.973 0.000
0.658 0.000
5.9/22
ICT
0.915 = 0.024
0.947=0.000
0.494 = 0.026
0.897 2 0.050
0.659
9.4
Tri-Mentoring
0.891 = 0.011
0.947
=
0.503 # 0.013
0.956 + 0.000
0.662
1.7/22
POS
0.715 + 0.046
0.954
0.444 0.020
0.889 0.061
0.634
13
FGM
0.923 = 0.023
0.044
0,481 + 0.024
0,811 + 0.079
0.611 0.008
13
Match-OPT
0.933 = 0.016
0.952
0.504 # 0.021
0.824 + 0.067
0.655 # 0.050
8.0
RaM-Rank Cosine (Ours)
0.940 # 0.028
0 = 0.017
0.514 # 0.026
0,982 0.012
0.675 # 0.049
22
RaM-ListNet (Ours)
0.949 1 0.025
0.962 0.015
0.517 a 0.029
0.981 0.012
0.670 1: 0.035
2
Table 3: 100th percentile normalized score of different methods combined with the MSE or ListNet loss in Design-Bence Bench, where positive and negative gain rates are Blue and Red.
Method
Type
Ani
D' Kilty
Supercunäuctor
TF-Bind-8
TF-Bind-10
Scure
Gain
Score
Guin
Score
Gain
Score
Gain
Scure
Gain
BD-gE1
ListNet
MSF
0.812
0.812 0,000
+0.0%
0.896 0.000
0.896
0.382 0.001
0.509 a 0.013
0,912
0.802 0.081
+13.7%
0.628 0.036
0.653 0.055
CMA-ES
ListNet
MSE
1.712
1,923
+12.3%
0.722 à 0.001
0.723 e 0.002
+0.1%
0.463
0.486 0.000
+5.0%
D:900
0.017
+1.7%
0,611
0.061
01044
REINFORCE
ListNet
MSE
0.248
0.318
0.156
0.039
+28.2%
0.366 a 0.091
0.559 # 0.139
+4.3%
0.478 a 0.017
0.501 : 0.013
+4.8%
0.935 0.049
0,935
= 0.049
+0.0%
0.673 a: 0.074
0.673 e 0.074
+0.0%
God. Ascent
LisNa
MSE
0.273 a 0.022
0.280
0.021
+2.6%
0853 + 0.017
0.890 e 0.019
+4.3%
0.510
0.521 a 0.012
+2.0%
0 0.020
0.011 0,985
+1.7%
0.646 # 0.037
0.660 e: 0,049
+2.2%
ChAS
MSE LisNN
0.546
0.854
0,030
0,037
+0.9%
0.9% #: 0.009
0.898 + 0.009
+0.2%
0.421 = (1,46
0:425 e 0.036
+1.0%
0.956 0.033
0,921 0.046
+3.8%
0A300 C/139
0.642 e 0.034
+1.9%
MINs
ListNet
MSE
0911
0.906
0.024
0,025
+0.5%
0.939 a 0.007
0,941
+0.2%
0.464 a 0.02
0,477 0.009
+2.8%
0,910 0.029
0.910 0.051
+0.0%
0.633 0.032
0.638 0.037
+0.8%
Mentoring
ListNet
MSE
0.915
0.801
+2.7%
0.947 0.005
0.943 à 0.004
0.503
0.503 0.010
0.956
0.005
+1.7%
0.710
0.012
PGS
MSE LisNar
0.715
0.723
0.832
+1.1%
0.944 à
0.962 + 0.018
+0.8%
0.452 0.042
0,020
+1.8%
0.886
.0.3%
0,604
0.643 0.030
Monch-4ee'
ListNer
MSE
0.933 a: 0.016
0.936 0.027
+0.3%
0.952 # 0.008
0.956 + 0.018
+0.4%
0.504 a 0.021
0.515
-1.8%
0.824 = 0.067
0.829 0.009
+0.6%
0.635 :: 0.050
0.659 + 0.037 +0.6%
Versatility of ranking loss. We examine whether simply replacing the MSE loss of some regression- based methods with a ranking loss can even bring improvement. Specifically, we substitute MSE with the best- rest-performng ranking loss, ListNet, and incorporate output adaptation. The results in Table show that the gains are always positive except two cases, clearly demonstrating the versatility of ranking loss. Details regarding method selection and implementation are provided in Appendix E.5
Results on OOD-MSE and OOD-AUPCC. We also present the OOD-MSE and OOD-AUPCC values of some methods in Appendix F55 where RaM performs well in OOD-AUPCC while poor in OOD-MSE, further demonstrating that ranking loss is more suitable than MSE for offline MBO.
5 CONCLUSION
Offline MBO methods often learn a surrogate model by minimizing MSE. In this paper, we question this practice. We empirically show that MSE has a low correlation with the final performance of the surrogate model. Instead, we show that the ranking-related metric AUPCC is well-aligned with the primary goal of offline MBO, and propose ranking- based model for offline MBO. Extensive experimental results show the superiority of our proposed ranking- of state-of-the--at offline MBO methods. We hope this work can open a new line of offline MBO.
baaeeng--asedd based model over a large variety
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
The authors would like to thank Yi-Xiao He for insightful discussions. This work was supported by the National Science and Technology Major Project (2022ZD01 16600), the National Science Foundation China (62276122 24, 624B1022 624B2069), the Fundamental Research Funds for the Central Universities (14380020), and Young Elite Scientists Sponsorship Program by CAST for PhD Students. Shen-Huan Lyu was supported by the National Natural Science Foundation of China (62306104), Hong Kong Scholars Program (XJ2024010), Jiangsu Science Foundation (BK20230949), China Postdoctoral Science Foundation (2023TQ0104), and Jiangsu Excellent Postdoctoral Program (2023ZB140) The authors want to acknowledge support from the Huawei Technology cooperation Project.
REFERENCES
Shivani Agarwal, Thore Graepel, Ralf Herbrich, Sariel Har-Peled, and Dan Roth. Generalization bounds for the area under the ROC curve. Journal of Machine Learning Research, 1.393-45,
2005.
Michael Ahn, Henry Zhu, Kristian Hartikainen, Hugo Ponte, Abhishek Gupta, Sergey Levine, and Vikash Kumar. ROBEL: Robotics benchmarks for learning with low-cost robots. In Proceedings of the 4th Conference Robot Learning (CoRL), pp. 1300-1313, Virtual, 2020.
Luis A. Barrera, Anastasia Vedenko, Jesse V. Kurland, Julia M. Rogers, Stephen S. Gisselbrecht, Elizabeth J. Rossin, Jaie C. Woodard, Luca Mariani, Kian Hong Kock, Sachi Inukai, Trevor Siggers, Leila Shokri, Raluca Gordân, Nidhi Sahni, Chris Cotsapas, Tong Hao. S. Stephen Yi, Manolis Kellis, Mark J. Daly, Mare Vidal, David E. Hill, and Martha Bulyk. Survey of variation in human transcription factors reveals prevalent DNA binding changes. Science, 351(6280):1500-144 1454, 2016.
Peter L. Bartlett and Shahar Mendelson. Rademacher and Gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research, 3:463-488 4822, 2003.
Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer- Verlag, Berlin, Heidelberg, 2006.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milnn and Qiao Zhang. JAX: Composable transformations of Python+NumPy programs, 2018. URL
http::/ //gthub. cow cax/jax jommjaaxmm/jjxx ml/jax
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. OpenAl Gym. arXiv: 1606.010000 2016.
David Brookes, Hahnbeom Park, and Jennifer Listgarten. Conditioning by adaptive sampling for robust design. In Proceedings of the 36th International Conference Machine Learning (ICML), pp. 773-782, Long Beach, CA, 2019.
Sebastian Bruch, Xuanhui Wang, Michael Bendersky, and Mare Najork. An analysis of the softmax cross entropy loss for learning-to-rank with binary relevance. In Proceedings of the 42nd ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR), pp. 75-78, Santa Clara, CA, 2019a.
Sebastian Bruch, Masrour Zoghi, Michael Bendersky, and Mare Najork. Revisiting approximate metric optimization in the age of deep neural networks. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pp. 1241-1244, Paris, France, 2019b.
Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. Learning to rank using gradient descent. Proceedings of the 22nd International Conference Machine Learning (ICML), pp. 89-96, Bonn, Germany, 2005.
11

---

Published as a conference paper at ICLR 2025
Christopher J. C. Burges, Robert Ragno, and Quoc Viet Le. Learning to rank with nonsmooth cost functions. In Advances in Neural Information Processing Systems 19 (NeurIPS), pp. 193-200, Vancouver, Canada, 2006.
Zhe Cao. Tao Qin, Tie- Yan Liu, Ming-Feng Tsai, and Hang Li. Learning to rank: From pairwise approach listwise approach. In Proceedings of the 24th International Conference on Machine Learning (ICML), pp. 129-136, Corvalis, OR, 2007.
Olivier Chapelle and Yi Chang. Yahoo! learning to rank challenge overview. In Proceedings of the 2011 International Conference on Yahoo! Learming Rank Challenge (YLRC), pp. 1-24, Haifa, Israel, 2011.
Olivier Chapelle, Yi Chang, and Tie-Yan Liu. Future directions in learning rank. In Proceedings of the 2010 International Conference on Yahoo! Learning to Rank Challenge (YLRC), pp. 91-100, Haifa, Israel, 2010.
Yassine Chemingui, Aryan Deshwal, Trong Nghia Hoang, and Janardhan Rao Doppa. Offline model- based optimization via gulice gradient search. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI), pp. 11230- 1239, Vancouver, Canada, 2024.
Can (Sam) Chen, Yingxue Zhang, Jie Fu, Xue (Steve) Liu, and Mark Coates. Bidirectional learning for offline infinite-width model-based optimization. In Advances in Neural Information Processing Systems 36 (NeurIPS), pp. 29454-29467, New Orleans, LA, 2022.
Can (Sam) Chen, Christopher Beckham, Zixuan Liu, Xue (Steve) Liu, and Christopher Pal. Parallel- mentoring for offline model-based based optimization. In Advances in Neural Information Processing Systems 37 (NeurIPS), pp. 76619-76636, New Orleans, LA, 2023a.
Can (Sam) Chen, Yingxue Zhang, Xue (Steve) Liu, and Mark Coates. Bidirectional learning for offline model-based biological sequence design. In Proceedings of the 40th International Conference on Machine Learning (ICML). pp. 5351 5366,, Honolulu, HI, 2023b.
Can (Sam) Chen, Christopher Beckham, Zixuan Liu, Xue (Steve) Liu, and Christopher Pal. Robust guided diffusion for offline black-box optimization. Transactions on Machine Learning Research, 2024.
Wei Chen, Tie- Yan Liu, and Zhiming Ma. Two-layer generalization analysis for ranking using Rademacher average. In Advances in Neural Information Processing Systems 23 (NeurIPS), pp. 370-378, Vancouver, Canada, 2010.
Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems 31 (NeurIPS), pp. 4302-4310, Long Beach, CA, 2017.
Koby Crammer and Yoram Singer. Pranking with ranking. In Advances in Neural Information Processing Systems 14 (NeurIPS), pp. 641-647, Vancouver, Canada, 2001.
Felipe Cucker and Stephen Smale. On the mathematical foundations of learning. Bulletin of the American Mathematical Sociery, 39:1-49 2001.
Manh Cuong Dao, Phi Le Nguyen, Thao Nguyen Truong, and Trong Nghia Hoang. Incorporating surrogate gradient norm improve offline optimization techniques. In Advances in Neural Information Processing Systems 38 (NeurIPS), Vancouver, Canada, 2024a.
Manh Cuong Dao, Phi Le Nguyen, Thao Nguyen Truong, and Trong Nghia Hoang. Boosting offline optimizers with surrogate sensitivity. In Proceedings of the 41st International Conference on Machine Learning (ICML), pp. 10072-10090, Vienna, Austria, 2024b.
Suresh Dara, Swetha Dhamercherla, Surender Singh Jaday, Ch Madhu Babu, and Mohamed Jawed Ahsan. Machine learning in drug discovery: A review. Artificial Intelligence Review, 55(3): 1947-1999, 2022.
12

---

Published as a conference paper at ICLR 2025
Domenico Dato, Claudio Lucchese, Franco Maria Nardini, Salvatore Orlando, Raffaele Perego, Nicola Tonellotto, and Rossano Venturini. Fast ranking with additive ensembles of oblivious and non-oblivious regression trees. ACM Transactions Information Systems, 35(2): 1-32))1-3,,, 2016.
Clara Fannjiang and Jennifer Listgarten. Autofocused oracles for model-based design. In Advances in Neural Information Processing Systems 33 (NeurIPS), pp. 12945-12956, Virtual, 2020.
Justin Fu and Sergey Levine. Offline model-based optimization via normalized maximum likelihood estimation. Proceedings of the 9th International Conference on Learning Representations (ICLR), Virtual, 2021.
Roman Garnett. Bayesian Optimization. Cambridge University Press, 2023.
Anna Gaulton, Louisa J. Bellis, A. Patrícia Bento, Jon Chambers, Mark Davies, Anne Hersey, Yvonne Light, Shaun McGlinchey, David Michalovich, Bissan Al-Lazikani, and John P. Overington. ChEMBL: A large-scale bioactívity database for drug discovery. Nucleic Acids Research, 40(D1): D1100-D1107, 2012.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville. and Yoshua Bengio. Generative adversarial networks. In Advances in Neural Information Processing Systems 27 (NeurIPS), pp. 139-144, Montreal, Canada, 2014.
Kam Hamidieh. A data-driven statistical model for predicting the critical temperature of a supercon- ductor. Computational Materials Science, 154:346-354, 2018
Nikolaus Hansen. The CMA evolution strategy: tutorial. 1rXi::1604.00772, 2016.
Geoffrey F. Hinton, Nitish Srivastava, Alex Krizhevsky, llya Sutskever, and Ruslan Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv: 1207.058. 2012.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems 33 (NeurIPS), pp. 6840-6851, Virtual, 2020.
Minh Hoang, Azza Fadhel, Aryan Deshwal, Jana Doppa, and Trong Nghia Hoang. Learning surrogates for offline black-box optimization via gradient matching. In Proceedings of the 41 International Conference on Machine Learning (ICML), pp. 18374-18393, Vienna, Austria, 2024.
Kalervo Järvelin and Jaana Kekäläinen. IR evaluation methods for retrieving highly relevant doc- uments. In Proceedings of the 23rd ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pp. 41 41-48, Athens, Greece, 2000.
Kalervo Järvelin and Jaana Kekäläinen. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4):422 046, 2002.
Alexandros Karatzoglou, Linas Baltrunas, and Yue Shi. Learning to rank for recommender systems. In Proceedings of the 7th ACM Conference on Recommender Systems (RecSys), pp. 493-494, Hong Kong, China, 2013.
Asif Khan, Alexander I. Cowen-Rivers, Antoine Grosnit, Derrick-Goh-Xin Deik, Philippe A. Robert, Victor Greiff, Eva Smorodina, Puneet Rawat, Kamil Dreczkowski, Rahmad Akbar, Rasul Tutunov, Dany Bou-Ammar, Jun Wang, Amos Storkey, and Haitham Bou-Ammar. Toward real-world automated antibody design with combinatorial Bayesian optimization. Cell Reports Methods, 3(1), 2023.
Minsu Kim, Federico Berto, Sungsoo Ahn, and Jinkyoo Park. Bootstrapped training of score- conditioned generator for offline design of biological sequences. In Advances in Neural Information Processing Systems 36 (NeurIPS), pp. (7661,6766, New Orleans, LA, 2023.
Minsu Kim, Jiayao Gu, Ye Yuan, Tacyoung Yun, Zixuan Liu, Yoshua Bengio, and Can Chen. Offline model-based optimization: Comprehensive review. arXiv:2503.1788, 2025.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference Learning Representations (ICIR), San Diego, CA, 2015.
13

---

Published as a conference paper at ICLR 2025
Diederik P. Kingma and Max Welling. Auto-encoding variational Bayes. In Proceedings of the 2nd International Conference on Learning Representations (ICLR), Banff, Canada, 2014.
Sathvik Kolli. Conservative objective models for biological sequence design. Master's thesis, EECS Department, University of California, Berkeley, May 2023.
Marius Köppel, Alexander Segner, Martin Wagener, Lukas Pensel, Andreas Karwath, and Stefan Kramer. Pairwise learning to rank by neural networks revisited: Reconstruction, theoretical analysis and practical performance. In Proceedings of the 19th European Conference Machine Learning and Knowledge Discovery in Databases (ECML. PKDD), pp. 237-252, Würzburg, Germany, 2019.
Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, and Aditya Grover. Diffusion models for black- box optimization. In Proceedings of the 40th International Conference on Machine Learning (ICML), pp- 17842-17857, Honolulu, HI, 2023.
Jakub Grudzien Kuba, Pieter Abbeel, and Sergey Levine. Cliqueformer: Model-based optimization with structured Transformers. arXiv:2410.13 2024a.
Jakub Grudzien Kuba, Masatoshi Uehara, Sergey Levine, and Pieter Abbeel. Functional graphical models: Structure enables offline data-driven optimization. In Proceedings of the 27th International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 2449-2457, Valencia, Spain, 2024b.
Aviral Kumar and Sergey Levine. Model inversion networks for model-based optimization. In Advances in Neural Information Processing Systems 33 (NeurIPS), pp. 5126-5137, Virtual, 2020.
Aviral Kumar, Amir Yazdanbakhsh, Milad Hashemi, Kevin Swersky, and Sergey Levine. Data-driven offline optimization for architecting hardware accelerators. Proceedings of the 10th International Conference on Learning Representations (ICLR), Virtual, 2022.
Yanyan Lan, Tie- Yan Liu, Zhiming Ma, and Hang Li. Generalization analysis of listwise learning- to-rank algorithms. In Proceedings of the 26th International Conference on Machine Learning (ICML), pp. 577-584, Montreal, Canada, 2009.
Hang Li. Learning to rank for information retrieval and natural language processing. Synthesis Lectures on Human Language Technologies, 4:1 4:1-113, 2011.
Tianqi Liu, Zhen Qin, Junru Wu, Jiaming Shen, Misha Khalman, Rishabh Joshi, Yao Zhao, Mo- hammad Saleh, Simon Baumgartner, Jialu Liu, Peter Liu, and Xuanhui Wang. LiPO: Listwise preference optimization through learning-to-rann arXiv:2402.01888, 2024.
Tie-Ya Yan Liu. Learning to rank for information retrieval. Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pp. 904, Geneva, Switzerland, 2010.
Huakang Lu, Hong Qian, Yupeng Wu, Ziqi Liu, Ya-Lin Zhang, Aimin Zhou, and Yang Yu. Degradation-resisttan offline optimization via accumulative risk control. Proceedings of the 26th European Conference on Artificial Intelligence (ECAI). pp. 1609-1616, Kraków, Poland, 2023.
Jayanta Mandi, Victor Bucarey, Maxime Mulamba Ke Tchomba, and Tias Guns. Decision-focused learning: Through the lens of learning rank. In Proceedings of the 39th International Conference on Machine Learning (ICML), pp. 14935-14944 Baltimore, MD, 2022.
Jayanta Mandi, James Kotary, Senne Berden, Maxime Mulamba, Victor Bucarey, Tias Guns, and Ferdinando Fioretto. Decision-focunt learning: Foundations, state of the art, benchmark and future opportunities. Journal of Artificial Intelligence Research, 80, 2024.
John I. Marden. Analyzing and modeling rank data. London: Chapman and Hall, 1995.
Satvik Mehul Mashkaria, Siddarth Krishnamoorthy, and Aditya Grover. Generative pretraining for blxck-box optimization. In Proceedings of the 40th International Conference on Machine Learning (ICML), pp. 249997 Honolulu, HI, 2023.
14

---

Published as a conference paper at ICLR 2025
Farzan Memarian, Wonjoon Goo, Rudolf Lioutikov, Scott Nickum, and Ufuk Topcu. Self Self-super online reward shaping in sparse-reward environments. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2369-2375, Prague, Czech Republic, 2021.
Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv: 2014.
Tung Nguyen, Sudhanshu Agrawal, and Aditya Grover. ExPT: Synthetie pretraining for few-shot experimental design. In Advances in Neural Information Processing Systems 36 (NeurIPS), pp. 45856-45869 New Orleans, LA, 2023.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach Vito,, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, high-performance deep learning library. In Advances in neural information processing systems 32 (NeurIPS), pp. 8024-8035, Vancouver, Canada, 2019.
Przemyslaw Pobrotyn and Radoslaw Bialobrzeski. NeuraINDCG: Direct optimisation of a ranking metric via differentiable relaxation of sorting. In the 5th SIGIR Workshop eConmerce at SIGIR 21, Virtual, 2021.
Przemyslaw Pobrotyn, Tomasz Bartezak. Mikolaj Synowiec, Radoslaw Bialobrzeski, and Jaroslaw Bojar. Context-aware learning to rank with self self-attention. In the 4th SIGIR Workshop on eCommerce at SIGIR"20, Virtual, 2020.
Han Qi, Yi Su, Aviral Kumar, and Sergey Levine. Data-driven offline decision- via invariant representation learning. In Advances in Neural Information Processing Systems 36 (NeurIPS), pp. 13226-13237, New Orleans, LA, 2022.
Hong Qian, Yiyi Zhu, Xiang Shu, Xin An, Yaolin Wen, Shuo Liu, Huakang Lu, Aimin Zhou, Ke Tang, and Yang Yu. SOO-Bench: Benchmarks for evaluating the stability of offline black-bol optimization. In Proceedings of the 13th International Conference on Learning Representations (ICLR), Singapore. 2025.
Tao Qin and Tie- Yan Liu. Introducing LETOR 4.0 datasets. arXiv:1306.2597 2013.
Tao Qin, Xudong Zhang, Ming-Feng Tsai, De-Sheng Wang, Tie-Yan Liu, and Hang Li. Query-level loss functions for information retrieval. Information Processing & Management, 44:838-855, 2008.
Tao Qin, Tie- Yan Liu, and Hang Li. A general approximation framework for direct optimization of information retrieval measures. Information Retrieval, 13(4):375-39) 2010a.
Tao Qin, Tie- Yan Liu, Jun Xu, and Hang Li. LETOR: A benchmark collection for research on learning rank for information retrieval. Information Retrieval, 13:346-374, 2010b.
Zhen Qin, Le Yan, Honglei Zhuang, Yi Tay, Rama Kumar Pasumarthi, Xuanhui Wang, Mike Bendersky, and Mare Najork. Are neural rankers still outperformed by gradient boosted decision trees? In Proceedings of the 9th International Conference on Learning Representations (ICLR), Virtual, 2021.
Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P. Adams, and Nando de Freitas. Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE, 104(1): 148-175, 2016.
Tianhao Shen, Renren Jin, Yufei Huang, Chuang Liu, Weilong Dong, Zishan Guo, Xinwei Wu, Yan Liu, and Deyi Xiong. Large language model alignment: survey. arXiv:2309. 15025, 2023.
Yunqi Shi, Ke Xue, Lei Song, and Chao Qian. Macro placement by wire-mask-guide black-box optimization. In Advances in Neural Information Processing Systems 37 (NeurIPS), pp. 6825-6843, New Orleans, LA, 2023.
15

---

Published as a conference paper at ICLR 2025
Feifan Song, Bowen Yu, Minghao Li, Haiyang Yu, Fei Huang. Yongbin Li, and Houfeng Wang. Preference ranking optimization for human aligmment. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI), pp. 119900-1999, Vancouver, Canada, 2024.
Samuel Stanton, Wesley J. Maddox, Nate Gruver, Phillip Maffettone, Emily Delaney, Peyton Green- side, and Andrew Gordon Wilson. Accelerating Bayesian optimization for biological sequence design with denoising autoencoders. In Proceedings of the 39th International Conference on Machine Learning (ICML), pp. 20459-20478, Baltimore, MD, 2022.
Ambuj Tewari and Sougata Chaudhuri. Generalization error bounds for learning to rank: Does the length of document lists matter? In Proceedings of the 32nd International Conference on International Conference on Machine Learning (ICML), pp. 315-323, Lille, France, 2015.
Brandon Trabucco, Aviral Kumar, Xinyang Geng, and Sergey Levine. Conservative objective models for effective offline model-based optimization. In Proceedings of the 38th International Conference on Machine Learning (ICML), pp. 10358-10368, Virtual, 2021.
Brandon Trabucco, Xinyang Geng, Aviral Kumar, and Sergey Levine. Design-Bench: Benchmarks for data-driven offline model-based optimization. In Proceedings of the 39th International Conference on Machine Learning (ICML), pp. 21658-21676, Baltimore, MD, 2022.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems 31 (NeurlPS), pp. 6000-6010, Long Beach, CA, 2017.
Vikas Verma, Kenji Kawaguchi, Alex Lamb, Juho Kannala, Arno Solin, Yoshua Bengio, and David Lopez-Paz. Interpolation consistency training for semi- suneiiupereieed learning. Neural Networks, 145: 90-106, 2022.
Xuanhui Wang, Cheng Li, Nadav Golbandi, Michael Bendersky, and Marc Najork. The Lambdaloss framework for ranking metric optimization. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (CIKM), pp. 1313-1322 Torino, Italy, 2018.
Ronald Williams. Simple statistical gradient- following algorithms for connectionist reinforcement learning. Machine Learning. 8:229-24 2-229-256, 1992.
Fen Xia, Yan Liu, Jue Wang, Wensheng Zhang, and Hang Li. Listwise approach to learning to rank: Theory and algorithm. In Proceedings of the 25th International Conference on Machine Learning (ICML), pp. 1192-1199, Helsinki, Finland, 2008.
Ke Xue, Rong-Xi Tan, Xiaobin Huang, and Chao Qian. Offline multi-objective optimization. In Proceedings of the 41 International Conference on Machine Learning (ICML), pp. 55595-55624, Vienna, Austria, 2024.
Michael S. Yao, Yimeng Zeng, Hamsa Bastani, Jacob Gardner, James C. Gee, and Osbert Bastani. Generative adversarial model-based optimization via source critic regularization. Advances in Neural Information Processing Systems 38 (NeurIPS), Vancouver, Canada, 2024.
Peiyu Yu, Dinghuai Zhang, Hengzhi He, Xiaojian Ma, Ruiyao Miao, Yifan Lu, Yasi Zhang, De- qian Kong, Ruiqi Gao, Jianwen Xie, Guang Cheng, and Ying Nian Wu. Latent energy-based Odyssey: Black-box optimization via expanded exploration in the energy-based latent space. arXiv:240S.16730, 2024.
Sihyun Yu, Sungsoo Ahn, Le Song, and Jinwoo Shin. RoMA: Robust model adaptation for offline model-based optimization. In Advances in Neural Information Processing Systems 34 (NeurIPS), pp. 4619-666333 Virtual, 2021.
Ye Yuan, Can (Sam) Chen, Zixuan Liu, Willie Neiswanger, and Xue (Steve) Liu. Importance-awe aware co-teaching for offline model-based optimization. In Advances in Neural Information Processing Systems 37 (NeurPPS), pp. 55718-55733, New Orleans, LA, 2023.
Ye Yuan, Youyuan Zhang, Can (Sam) Chen, Haolun Wu, Zixuan Li, Jianmo Li, James Clark, and Xue (Steve) Liu. Design editing for offline model-based optimization. arXiv:2405.1 13964, 2024.
16

---

Published as a conference paper at ICLR 2025
Taeyoung Yun, Sujin Yun, Jaewoo Lee, and Jinkyoo Park. Guided trajectory generation with diffusion models for offline model-based optimization. In Advances in Neural Information Processing Systems 38 (NeurIPS), Vancouver, Canada, 2024.
Zhi-Hua Zhou and Ming Li. Tri-training: Exploiting unlabeled data using three classifiers. IEEE Transactions on Knowledge and Data Engineering, 17(1 2005.
17

---

Published as a conference paper at ICLR 2025
A RELATED WORK
A.1 OFFLINE MODEL-BASED OPTIMIZATION
Offline MBO methods (Trabucco et al. 2022 Kim al., 2025 Qian et al. 2025 Xue et al. 2024)
can be generally categorized into two types of approaches. The mainstream approach for offiine MBO is the forward approach, which first trains a forward surrogate model So X + IR and then employs gradient ascent to optimize the learned surrogate to output candidate solutions, as introduced in Section 2 A crucial challenge of this approach is how to improve the surrogate model's general- ization ability in the OOD regions, which can significantly affect the performance. Prior works of forward approach mainly add regularization items to: 1) regulate the nature of the surrogate model: NEMO (Fu & Levine| ( optimizes the gap between the surrogate model and the ground-truth function via normalized maximum likelihood, BOSS (Dao et al. 2024b) and IGNITE (Dao et al., 2024 regulate the sensitivity of the surrogate model against perturbation on model weights from different perspectives, while RoMA (Yu et al., 2021) enhances the smoothness of the model in pre-trained and adaptation manner; 2) regulate surrogate model's predictions directly: COMs (Tra bucco et al. 2021) penalize identified outliers via a GAN-like procedure (Goodfellow et al. 2014). whereas 1OM (Qi et al. 2022) maintains representation invariance between the training dataset and design candidate. Given that an ensemble of surrogate models can bring an improvement (Trabucco iet al. 2022), ICT (Yuan et al. 2023) and Tri-Mentoring (Chen et al. 2023a) train three symmetric surrogate models and ensemble them, where ICT uses a semb-supervised learning via pseudo-label procedure (Verma et al., 2022) and Tri-Mentoring employs a strategy similar to Tri-training (Zhou &
Li 2005) from pairwise perspective. Besides, recent works have tried to uncovering the structural information of the dataset for better learning. BDI (Chen et al.) 2022] utilizes both forward and backward mappings to distill knowledge from the offline dataset to the design. Both FGM (Kuba)
ct al. 2024b) and Cliqueformer (Kuba et al. 2024a) consider novel modeling, which splits the design space into cliques on dimension- level,, to approximate scores. Both PGS (Chemingui et al. 2024) and Match-OPP (Hoang et al. 2024) construct trajectories from the dataset, while PGS uses ofiline reinforcement learning to learn a policy that predicts the search step size of the gradient ascent optimizer and Match-OPT enforces the model to match the ground-truth gradient. Recent works also consider regulating the model-inne, search procedure. For example. DEMO (Yuan et al. 2024) edits the designs obtained by gradient ascent via a diffusion prior, GAMBO Yao al. 2024) regulates the optimize trajectory via modeling the search procedure as a constrained optimization problem, while ARCOO (Lu et al.) (023) guides the search step size via trained energy model. However, all prior works in forward approach train the surrogate model based on a regression-bussd model using MSE as a base term in the loss function. In this work, we challenge this practice and train the surrogate model in ranking suite, obtaining superior performance, as shown in Section 4
Another type of approach for offline MBO is the backward approach, which typically involves training a conditioned generative model pa (xly) and sampling from it conditioned on a high score, for example, MINs (Kumar Levine 2020) trains an inverse mapping using a conditioned GAN-
like model (Goodfellow et al. [014: Mirza& & Osindero 2014). while CbAS (Brookes et al., 2019
Fannjiang & Listgarten 2020) models it as a zero-sum game via a VAE (Kingma & Welling 2014).
Note that generative models show impressive expressiveness and have achieved huge success, works in this field employ powerful generative model to obtain final designs. DDOM (Krishnamoorthy et al.,
2023) and RGD (Chen al. (2044) directly parameterize the inverse mapping with a conditional diffusion model (Ho et al. 2020) in the design space. ExPT (Nguyen et al. (023) learns from synthetic prior and adapt in a few-shot suite using Transformers (Vaswani et al. 2017). LEO YYu et al. 2024) constructs latent space through an energy- mneeg model that does not require MCMC sampling. Recent works in this category also focus on generating designs via constructed trajectories. For example, BONET (Mashkaria et al. 2023) uses trajectories to mimic a black-box optimizer, thus to train an autoregressive model and sample designs using a heuristic; GTG (Yun et al. 2024) considers improving the quality of trajectories via local scarch, and then directly generate trajectories using a context conditioning diffusion model.
In the field of offline MBO, some studies are related to the idea of ranking designs or implicitly use the ranking information: 1) Match-OPT (Hoang et al. 2024). The idea of gradient matching in this paper is related to ranking samples, since model with proper gradient could reflect the relationship in a small neighborhood. 2) Tri-Mentoring (Chen et al. 2023a). In Tri-Mentoring, each proxy uses weak semi-supervised pairwise-ranking-bae voting signals provided by other proxies to fix its
18

---

Published as a conference paper at ICLR 2025
predictions and finetune its weights. 3) BONET (Mashkaria et al. 2023). The trajectories used in BONET are constructed by ranking the collected samples, from which the model may capture some ranking information. Although these methods capture ranking information in some ways, in this work, we explicitly identify the idea of ranking samples, and conduct a systematic analysis on this view. After that, reformulate the objective of the training process by replacing the core MSE loss with a ranking loss, and apply data augmentation and output adaptation for model training and solution search, respectively. The superior experimental results in Section | also indicate the significance to focus on ranking information in offline MBO. In Appendix that leverage LTR techniques into their respective fields to make advances.
A A.2
we also discuss some related works
A.2 LEVERAGING LTR TECHNIQUES INTO SPECIFIC DOMAINS
In this subsection, we also briefly introduce works in three fields that share similar motivation to leverage LTR techniques to advance their respective domains.
Decision-focused learning (DFL: Mandi et al., 2024). DFL, also termed ppredict-then-opeee then-optimize", aims to predict unknown parameters for an optimization problem using MI. model in an end-to- end paradigm. A recent popular work of this field is Mandi et al. (2022), which utilizes LTR losses that preserve the correct order of solutions in the discrete feasible space to train a better parameter-predicting model.
Preference- reinforcement learning (PbRL; Christiano et al. 2017). The goal of PbRL is to infer reward functions from human feedback in the form of preferences rankings over demonstrated behaviors, Memarian et al. (2021) define preference oracle to measure the total order equivalency and use pairwise ranking loss train a reward model for the sparse-reward environments.
Language model alignment (Shen et al. 2023). The objective of language model alignment is to let the models align with human preferences. Song et al. (024)) adopt LTR techniques to process human preference rankings of varying lengths, while Liu et al. (2024) formulate the problem as a listwise ranking problem, which can learn more efficiently from given ranked list response.
However, our work differs from these works in both motivation and methodology. We focus on offline MBO and investigate the root cause of the OOD issue, which widely-studied in this field but still remains. We provide systematic analysis of the OOD issue, propose the AUPCC metric for quantification, develop a ranking-base based framework, and verify its effectiveness through theoretical analysis and comprehensive experiments.
B PREVALENT SETTINGS O, N()), AND Ca(o) IN THEOREM
In this section, we introduce some settings of Φ. (Ø), and Ca(ő) in Theorem shown in Lan et al. 2009).
In Theorem is increasing and strictly positive transformation function, which maps the output of the surrogate model or the score to a positive real number. Recall that B represents the upper bound of the weight norm ||w|| of the linear function class F (x - w x ||w| < B} where the ranking model f to be learned is from, and M the upper bound the norm of designs ||x]] in design space X. It is usually represented as a:
Linear function: (z) = 6, E [BM, BMM, where > 0 and 6 > aBM:
Exponential function: (z(()) exp(az), z E [-BM, BM). where > 0;
Sigmoid function: (2(2)) T=exp(-as) z E BM, BM], where a >> 0.
Following Lan et al. 2009), ) introduce some settings based on the above definition of o in Table
For detailed derivation for (o)),, please refer to Lan et al. 2009)).
C PROBABLE APPROACHES AND DIFFICULTIES FOR THEORETICAL ANALYSIS
In this section, we first further discuss the probable approaches and difficulties for direct theoretical analysis for ranking-based framework for offline MBO. Although it is challenging, still find
19

---

Published as a conference paper at ICLR 2025
Table 4: N (0) and (o( for L.TR algorithms A (e.g., RankCosine QQnn et al. 2008) or ListNet (Coo et al. 2007) different definitions of Φ.
(Φ)
CRankCosine (6)
LiieNNel()
(z) = az +
a
2b-aBBM)
v
(BM m-+log 24a
2m!
6z(2) = exp(az)
a ee
Mm.exp(BBM)
2ml BX)
2
log m 22 BMM
(s(2) = I+exp(-c0 os)
a(I+explaBMM)
(m(1+xx((BBM)
2m +oxpaaax
[toxp(-aBMD]
2
log m
counterexample that shows the robustness of LTR losses over MSE. Then, to enhance understanding of the counterexample, we conduct a quantitative experiment to demonstrate this.
C.1 PROBABLE APPROACHES AND DIFFICULTIES
In this subsection, firstly, we revisit our motivation to leverage LTR techniques for offline MBO. Then, propose some probable approaches and difficulties for theoretical analysis.
Note that learning to rank samples correctly is a weaker condition than learning to minimize MSE, since MSE commands for both order preserving and value matching. Besides, the equivalence in Theorem shows that the weaker condition, order preserving, is sufficient for offline MBO, which motivates the proposal of directly learn the ranking information by leveraging LTR techniques.
Thus, a intuitive question according to generalization analysis for offline MBO is: In which scenarios does the model learned with LTR generalize better on some ranking measures than that learned with MSE on OOD regions? Unfortunately, such theoretical support or evidence cannot be found even in the field of LTR, which is also illustrated in Section of Chapelle et al. (2010). Below we briefly present the most promising approach we explored and the difficulties we face.
Try to find a special function class F, from which the ranking model f to be learned is, such that models learned with LTR techniques have an upper bound guarantee on some ranking measure while models trained with MSE do not. Formally, let be ranking measure (which can be the expected risk of a specific ranking loss or ranking metric. e.g., NDCG). and denote the empirical risk of model trained with LTR and that trained with MSE as RETR and RUSE, respectively. For ease of exposition, here refers to the expected risk in Theorem From Theorem the upper bound of R and RLTR R has convergence rate of O ) Then, if we could find function class F such that
RM always has a slower convergence rate, i.e., R - RMSE 2 O( (. we can show that models learned with MSF are worse than that learned with I.TR. However, such an analysis can be difficult because: 1) There no theoretical evidence to show the generalization bound on ranking by optimizing MSE. 2) Most generalization bound analysis in LTR assume i..d (as Theorem 2 in our paper), while OOD analysis in LTR is quite limited.
Identify a special case that supports this intuition. Assume that the function class F is linear function class and the offline data drawn from ground-truth function I with long-tailed noise on the objective value. Models trained with MSE are susceptible to heavy- tailed noise, as the mean of y is heavily influenced in regions with such noise. contrast, models trained with pairwise ranking loss demonstrate greater stability in such scenarios. An illustrative example could be as follows. Assume that the ground
function is f(x) = 2 and the offline dataset D = {(1, 1). (1.9, 3.7), (2.1, 4.5). (2, - 12)} where (2, -22) suffers from the heavy-tailed noise. Models trained with MSE and a representative of the pairwise ranking loss, RankCosine (Qin et al. 2008). are shown in Figure 3 From Figure 3 the model trained with MSE would exhibit negative correlation, while that trained with LTR would demonstrate positive correlation, which shows that the model trained with LTR is more robust. However, such counterexamples are still based on strong assumptions. A well-constructed example with theoretical support remains unexplored.
20

---

Published as a conference paper at ICLR 2025
Linear model obtained by different losses under heavy-tailed noise
10
0
-5
Ground Truth: f(x)
Training Data
10
MSE: y -2.10x + 2.98
RankCosine: 0.79x
0.85
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Figure 3: Plot of the ground-truth function f(z) = x², the training data suffered from heavy-tailed noise, the linear model learned with MSE (green), and the linear model learned with RankCosine. Here the model trained with MSE exhibits negative correlation, while that trained with LTR demonstrates positive correlation, which shows that the model trained with LTR more robust.
C.2 ADDITIONAL EXPERIMENTS IN HEAVY- HEAVY-TAILED NOISY SCENARIOS
In this subsection, we conduct additional quantitative experiments to support the counterexample mentioned in the above subsection.
Following the assumption in Appendix C.1 the ranking model to be learned from the linear function class. Specifically, given dataset D {(ri, v:) want train linear model
f(x) war + b based on two loss functions, e.g., MSE and RankCosine QQin al. 2008), the representative of pairwise ranking losses. Details of how to obtain the linear model trained with these two losses are as follows.
MSE. The linear model trained with MSE has formeddformed solution using the Least Squares Method, Formally, let an augmented matrix X = [1. [21 Z2. ... Z y =
(y1, 1/. 1/........... ... : .yN] and 0 (w. b)T, and
obtain that = (X (X - y (see
Chapter 3.1.1 Bishop 2006
RankCosine: There is no closed-formed solution due to non-linear operations (specifically, vector normalization operator when calculating RankCosine). Hence, we use Adam opti- mizer (Kingma & Ba 2015) with learning rate 10-3 3 to search 1000 epochs for the optimal value for w and b.
We set the ground-truth function to be quadratic function f(x) = x for case of demonstration, which is increasing and requires Í having a positive w. We assume that the training data is drawn from [0, for better visualization. As for the noise, initiate the heavy-tailed noises from Student's
t-distribution g(t) r(*t))
with the degrees of freedom 2, and change their magnitude controlled by scale 15. Besides, to influence the increasing trend, assume that the heavy-tailed noise is positive for points with z [0, 1.3] and negative for points with æ E (1.3,3]. Each training point has probability of p 0.2 to suffer from the noise.
(
We first present the detailed results of the illustrative example mentioned in Appendix C.1 Figure 3 visualize the ground-truth truth function, training data, and the linear models trained with MSE and RankCosine. We can observe that the model learned from MSE exhibits negative correlation, but the model learned from RankCosine demonstrate positive correlation.
21

---

Published as a conference paper at ICLR 2025
To further verify the robustness of ranking losses, increase the dataset size to 100, and vary the scale of noise a E {10, 15, 20, 50, 100) while the probability of adding noise is fixed at = 0.2. We report the calculated values of wo learned with MSE (denoted WMSE) and that learned with RankCosine (denoted as RankCosine) according to different as Table C.2 From Table C.2 all
Table 5: Values of weight w obtained by learning MSE (denoted as WMSE) and those obtained by learning RankCosine (denoted as WaankCosime) with varying noise scale a. Here, Violet denote positive weights, which satisfies the requirements of the ground-truth function f(x) = x² for increasing linear ranking model.
Noise scale a
WMSE
W RankCosine
10
-1.68
0.88
15
-0.75
0.90
20
-2.98
0.74
50
-14.99
0.94
100
-10.05
0.98
values of HonkCosine are positive while those UMSE are all negative and become substantially worse when the scale of noise goes larger, which demonstrates the stronger stability of the LTR loss against heavy-tailed noise with different strengths.
We also vary the probability of adding noise p E {0.1, 0.2, 1.0) while the scale of noise is fixed at a = 15. The corresponding values of w are shown Table
Table 6: Values of weight w obtained by learning MSE (denoted as WMSE) and those obtained by learning RankCosine (denoted as WRankCosine) with varying noise probability p. Here, Violet denote positive weights, which satisfies the requirements of the ground-truth function f(x) = x² for increasing linear ranking model.
Noise probability
WMSE
WRnnCoosine
0.1
1.61
0.86
0.2
-0.75
0.90
0.3
-4.53
1.01
0.4
-8.46
0.88
0.5
-7.76
1.02
0.6
-10.73
0.95
0.7
-12.87
0.84
0.8
-17.28
0.98
0.9
-20.21
0.98
1.0
-22.66
0.95
From the results in Table only when the noise probability p 0.1, WMSE positive, while in other situations it is negative and it becomes quite bad as p increases. In contrast, WaankCosine remains a positive value near the noise probability p increases from 0.1 to showing impressive robustness against such heavy-tailed noise with wide coverage.
stronlyy Results from both Table C.2 and Table strongly demonstrate the robustness of pairwise ranking loss (i.e., RankCosine) over MSE the ranking performance in scenario where suffers from heavy-tailed noise, which delivers better understanding on the advantage of LTR losses in OOD ranking performance. Combining with the stated equivalence of an order-preserving surrogate model shown in Theorem I performance.
the ranking loss suitable for offline MBO due to its more robust ranking
D DETAILS OF DIFFERENT RANKING LOSSES
In this section, introduce details of the different ranking losses in this paper, including traditional and recently prevalent losses. We study different types of ranking losses in this paper, including pointwise (Crammer & Singer 2001), pairwise (Köppel et al. 2019), and listwise losses (Xia
22

---

Published as a conference paper at ICLR 2025
al. 2008
Formally, given a list X of designs and the list y of their corresponding scores, let (x)(),) (x be the predicted scores.
fo (X)
For the pointwise losses, we consider:
SigmoidCrosEEntroy (SCE): widely used pointwise loss: I(y, fo())
Σ (x,f(xx)) + log(1 + exp(fo(x:))))
BinaryCrosSEntropy (BCE): a common pointwise loss considering in binary clas- sification manner, and we consider its variant with logits input: ((y, fx()) =
>>= 1== Ui log(o(fo(xs))) + yi) log(1 (fo(x)))) where o(-) is the sigmoid function.
Mean Square Error (MSE): a popular pointwise loss aiming to fit the target values: m l(y, fXX)) = - (it - ? Note that the difference of RaM combined with model mainly reflects in the different modeling of
MSE from the regression training data.
For the pairwise losses, consider:
RankNet (Burges et al. 2005 : a popular pairwise
loss:
I(y, foy)))(X)
=
En,y, log exp(.f((x)) fxx)))
LambdaRank [Burges et al. 2006 Wang al. 2018 pairwise loss with ANDCG weight:
I(y, (X))) Σ Ee A ANDCG(i,))) loga cxp(=a(foxx)) fxx))))) where is a smooth parameter and ANDCG is the absolute difference between the values of the Normalized Discounted Cumulative Gain (NDCG), widely used metric in LTR (Järvelin & Kekäläinen 2000 2002). when the surrogate model swap the predictions of the two designs, X; and Xj, and thus swap their positions in the ranked list.
RankCosine (Qin et al. 2008): a classical pairwise loss based on cosine similarity: Iyy, fe())) = 1 y fe (X)((l))) IX)
For the pairwise losses, we consider:
* Softmax (Cao et al. 2007 Bruch al. 2019at: popular listwise loss:
fo fo()
-Em log exp(fo(x.)
Em (x:()((x:))
ListNet (Cao et al.
2007): classical listwise loss minimizing the cross-entropy be-
tween the predicted ranking distribution and the true ranking distribution: 1(y, fo fe())
Ei=1 -E
E (xp( log (x,()
exp(y;)
exp(fo(xx))
ListMLE (Xia et al.
2008 widely used listwise loss based on the Plackett-Luce explfe(x=(o))
Em log Ly, exploxxo where is the permutation derived from the true ranking labels y, Xx(e) represents the item at the i-th position in the true ranking.
model Marden. ): ((y, fe())
2010a Bruch et al. 2019b): listwise that is a differentiable ApproxNDCG (Qin et al.
approximation of NDCG: Ily, fe (X)) DCG .y)
m 24:-1 Σxr= log, (8) where x² is
the optimal permutation that ranks items by y. DCG y represents the Discounted Cumulative Gain (DCG; Järvelin & Kekäläinen 2000 2002) of the ideal ranking given y, and म (i) = 1 + Σ, Sigmoid fo(x,) (x())) with T a smooth parameter. 7
We excluded NeCGaINDCG (Pobrotyn & Bialobrzeski 2021) D a recently proposed listwise loss using neural sort techniques approximate NDCG, due to its high memory requirements.
23

---

Published as a conference paper at ICLR 2025
E DETAILED EXPERIMENTAL SETTINGS
E. DETAILED EXPERIMENTAL SETTINGS FIGURE
In this experiment, select five surrogate models: gradient-ascent baseline and four state-of-the- art approaches, COMs (Trabucco et al. 202T). IOM (Qi et al. 2022), ICT (Yuan et al. (023). and Tri-Mentoring (Chen al. 2023a). These models are chosen due to their common characteristic of employing standard gradient-ascent to obtain the final design. While BDI (Chen et al. (202) and Match-OPT (Hoang et af, 2024) also utilize gradient-ascent for design generation, we exclude BDI for its intractable model, which is built with JAX (Bradbury et al. 2018), and Match-OPT for its time-intensive training procedure.
We follow the default setting as in Chen et al. (2023a); Yuan et al. 2023 to prepare training data and set the syper-parameters Equation | to search inside the model. For discrete tasks, in order to map the design space to continuous one, transform the discrete designs into real-valued logits of categorical distribution, which is provided in Trabucco et al. (2021) 2022). We use z-score method to normalize both the designs and scores for better training. After the model is trained,
Adam optimizer Kingma& & Ba 2015) to conduct gradient ascent. For discrete tasks, we set 1j = 1 x 10-1 and T 100, and for continuous tasks, we 1) 1 10-3 and T 200.
Following Chen et al. (023a)), we construct an OOD dataset by selecting the high-scoring designs that are excluded for the training data in Design-Bent (Trabucco et al. 2022). In Design-Bench, the training dataset is selected as the bottom performing x% in the entire collected dataset, (i.e.,
proovide 40, 50, 60). Note that the open-source repository provides an API to access the entire dataset. We identify the excluded (100 x 1% high-scoring data to comprise the OOD dataset for analysis, except for TF-Bind- 10 (Barrera et al. 2016) task, whose excluded (100 x) % high-scoring data contains 4161482 samples and is too large for AUPRC evaluation. Thus, we randomly sample 30000 samples from the (100- 3 data to construct the OOD dataset for TF-Bind-10 task.
F.2 EXCLUDED DESIGN- -ENCH TASKS
Following prior works KKishhamoorthyy al. 2023 Mashkaria et al. 2023 Yun al. 2024 Yu
et al. 2024). we exclude three tasks in Benigh-Bench (Trbbucco et al. 2022) for evaluation, including Hopper (Brockman et al. 2016), ChEMBL (Gaulton et al.) 2012). 2). and synthetic NAS tasks on CIFARIO (Hinton et al. 2012). As noted in prior works, this is a bug for the implementation of Hop- per in Design- Bench (see https: /ttp//gttii http:/uwww.........aaeoeeaeaeeee
www.://bannnntttoeo abucco/ wawpucoo//eiiinbeeeneeeeeeeeeeee
issues http://iissuucoommntii 1086 088 758113 for details). For the ChEMBL task, exclude because almost all methods produce the same results, as shown in Mashkaria et al. (2023); Krish namoorthy al. (2023). which is not suitable for comparison. We also exclude NAS due to its high computation cost for exact evaluation over multiple seeds, which is beyond our budget.
E.3 EXCLUDED OFFLINE MBO ALGORITIMMS
We exclude NEMO Fuu & Levine 2021) ) since there is no open-source implementation. We also exclude concurrent works, DEMO Yuan al. 2024) and LEO (Yu al., 2024), since they not yet peer-reviewed and lack an open-source implementation at the time of our imitial submission. For BOSS (Dao et al., 2024b), we exclude it since it is a general trick that can be applied any regression- based forward method, instead of a single proposed methods.
E.4 DETAILED EXPERIMENTAL SETTINGS OF MAIN RESULTS IN TABLE 2
We set the size n of training dataset to 10000, and the list length m = 1000. To make a fair comparison to regression- based methods, following Trabucco et al. 22021 2022); ;) Chen et al. (2023a); Yuan et al. (2023). , we model the surrogate model fe as a simple multilayer perceptron (MLP) with two hidden layers of size 2048 using PyTorch (Paszke et al.) 2019). We use ReLU as activation functions. RankCosine (Qin et al. 2008) and ListNet (Cao et al. 2007) is used as two main loss
24

---

Published as a conference paper at ICLR 2025
functions in experiments. Our implementation of different loss functions is either inherited from Pobrotyn et al. 2020 or implemented by ourselves.
We split the dataset into training set and validation set of the ratio 8 2. The model is trained for .N. = 200 epochs and is optimized using Adam (Kingma & Ba, 2015) with a learning rate of 3 x 10-4 and weight decay coefficient of x 10 and the model with minimal validation loss among No epochs serves as the final model.
After the model is trained.
fix the model parameters and normalize the output values, then following Chen et al. (2023)); Yuan et al. (202)), we set 1) = 1 x 10-3 and T = 200 for continuous 10 and T 100 for discrete tasks search for the final design.
tasks, and 1)
For baselines methods and CbAS (Brookes et al. 2019). MINs (Kumar & Levine 2020). COMs Tra- bucco et al. (222)) in Table 2 we use the open-source baselines implementations from the source
code of design-Bench
For other offline MBO methods (DDOM (Krishnamoorthy et al. 2023
BONET Mashkaria et al.
2023
$ GTG (Yun et al. 2024
9 RoMA (Yu 2
2021
10M
et a
2022
BDI (Chen et al.
2022
12
ICT Yuaan et al. 2023
13
Tri-Mentoring (Chen et al. 2023a
14
PGS Chemingui et al. 2024 FGM (Kuba et al. 2024b 16
Match-OPT (Hoang et al. 2024 0. we
use the open-source implementation provided in their papers and use their hlyper-parameter settings, except for DDOM and BONET, where we modify the evaluation budget l from 256 to 128 following the protocol of other works. A brief review of offline MBO methods is also provided in Appendix A.1
E.5 DETAILED EXPERIMENTAL. SETTINGS OF TABLE 3
In this experiment, for fair comparison of MSE and ListNet, do not adopt the data augmentation method, instead, we use the naive approach introduced Section 3.3 viewing a batch of designs as a list be ranked.
We choose baselines methods that optimize a trained model, BO-qEI GGrnnttt 2023). CMA-
ES (Hansen 2016), REINFORCE (Williams] 1992), and Gradient Ascent, two backward approach provided in Trabucco et al. 2022). CbAS (Brookes et al. 2019) and MINs (Kumar & Levine 2020). and three state-of-the-ee forward methods that can replace MSE with ListNet, Tri-Mentoring (Chnn
et al. 2023a). PGS (Chemingui et al. (024). and Match-OPI (Hoang et al. 2024). Note that the model trained with ranking loss has different prediction scales as regression- based models, as dis- cussed in 3.3 We exclude many forward methods due to the inapplicability of directly replacing
MSE with ListNet. For example, COMs (Trabucco et al. (02I), RoMA (Yu et al.) 2021). IOM (Qi et al. 2022) use the prediction values to calculate the loss function, where the changing scales of predictions could influence the scales of the loss values, while BDI (Chen et al. (022) and ICT (Yuan)
ett al.] (2023) assign weight to each sample, thus MSE in these methods cannot be directly replaced with a ranking loss like ListNet.
In order adapt the same parameters of the online optimizers (e.g., BO-qEI, Gradient Ascent) that optimize the trained model for a fair comparison, we also perform an output adaptation for ranking-basc based model after it is trained.
All the replacements are conducted fixing their open-source codes by replacing MSE with ListNet when training the forward model.
USp
asppshharig
25

---

Published as a conference paper at ICLR 2025
Table 7: 50th percentile normalized score in Design-Bench, where the best and runner-up results on each task are Blue and Violet. D(best) denotes the best score in the offline dataset.
Method
Ant
D'K Kitty
Superconductor
TF-Bind-8
TF-Bind-10
Mean Rank
D(best)
0.565
0.884
0.400
0.439
0.467
BO-yEI
0.568 0.000
0.883 0.000
0.311 0.019
0.439 0,000
0.467 0.000
12.6/
CMA-ES
-0.041 = 0.004
0.684 0.017
0.377 = 0.009
0.539 = 0.017
0.482 = 0.009
12.6/ 22
REINFORCE
0.124 + 0.042
0.460 + 0.209
0.457 = 0.020
0,466 + 0.023
0.464 + 0.009
15.7/
Grad. Ascent
0.016
0.581 0.128
0.471 0.017
0,582 0.027
0.470 0.004
11.2//2
Grad. Ascent Mean
0.185 + 0.012
0.718 = 0.037
0,481 = 0.023
0.630 0.033
0.470 = 0.005
9.2/22
Grad. Ascent Min
0.187 = 0.012
0.714 0.040
0,480 = 0.022
0.628 + 0.025
0.470 0.004
9.6/22
CbAS
0.385 0.027
0.740 = 0.023
0.121 = 0.014
0.422 i 0.022
0.457 = 0.006
18.4/22
MINs
0.640 = 0.029
0.886 0.006
0.332 0.014
0,407 = 0.014
0.465 = 0.006
12.0/22
DIXM
0.598 + 0.030
0.829 + 0.050
0.313 0.017
0.416 0.023
0.464 + 0.006
14.4/
BONET
0.795 = 0.039
0.906 + 0.008
0.334 0.032
0.476 0.149
0.452 1: 0.050
10.6/ 22
GTG
0.593 + 0.022
0.889 0.002
0.350 0.023
0.542 0.038
0.458 + 0.008
9.4/22
COM&
0.532 a: 0.020
0.882 a: 0.002
0.376 0.067
0,513 1 0.019
0.474 1: 0.014
9,3/22
RoMA
0.193 = 0.017
0.344 = 0.097
0.368 = 0.012
0.520 + 0.074
0,516 = 0.004
12.0/22
IOM
0,459 + 0.024
0.829 + 0.022
0.291 0.059
0.490 0.055
0.467 + 0.000
14.9/
BDI
0.569 = 0.000
0.876 = 0.000
0.389 0.022
0,595 0.000
0.429 = 0.000
9.4/22
ICT
0,550 + 0.028
0.875 + 0.006
0.333 = 0.018
0.547 * 0.041
0.499 = 0.012
9.2/22
Tri-Mentoring
0.548 0.013
0.870 0.002
0.363 0.019
0.491
8.0/22
PGS
0.190
0.885 = 0.001
0.233
0,503 = 0.041
0.386 0.177
16.0/ 22
FGM
0.532 + 0.039
0.871 + 0.017
0,353 = 0.058
0,540 + 0.117
0.466 + 0.004
12.1/22
Match-OPT
0.587 0.008
0.887 0.001
0.381 0.038
0.471
8,0/22
CoMin (Ours)
0.566 0.012
0,881 0.003
0.356 0.013
0.544 0.043
0,462 + 0.006
11.0/22
RaM-ListNet (Ours)
0.579 +: 0.014
0.888 + 0.003
0.359 0.013
0,552 0.032
0.467 a 0.009
7.4/22
F ADDITIONAL EXPERIMENTS
In this section, we provide additional experimental results mentioned in Section
F.1 50TH PERCENTILE RESULTS ON DESIGN-BENCH
Following the evaluation protocol in Trabucco et al. (2022). to validate the robustness of our proposed method, we also provide the detailed results of 50th percentile results Table
In Table 7, we can observe although RaM combined with RankCosine performs not so well on 50th percentile results, RaM combined with ListNet, which is the best methods in our main experimental results (Table 2). also obtains best average rank of 7.4 among 22 methods.
F.2 RESULTS OF DIFFERENT RANKING LOSSES
We compare a wide range of ranking losses that combined with RaM in the context of offline MBO, including three types of pointwise, pairwise, and listwise losses. Details of these ranking losses are provided in Appendix [D and experimental results of 100th percentile normalized score in Design-Bench are provided in Table
Table 8: 100th percentile normalized score of RaM combined with different ranking losses in Design- Bench. The best and runner-up results on each task are Blue and Violet. D(best) denotes the best score in the offline dataset.
Type
Method
Ant
Kitty
Superconductor
TF-Bind-8
TF-Bind-10
Mean Rank
D(best)
0,565
0.884
0.400
0.439
0.467
RaM-SCE
0.928 = 0.012
0.953 = 0.012
0.502 + 0.013
0.820 + 0.065
0.662 + 0.026
6.9/10
Pointwise
RaM-BCE
0.014
0.950 0.009
0.501
0.012
0.825 0.065
0.656
0.021
8
RaM-MSE
0.933
0.957 = 0.013
0.507 1 0.028
0.962 = 0.031
0.674 a 0.044
3.9/ 10
RaM-RankNet
0.921 0.033
0.955 0.008
0.510 0.032
0.962 0.030
0.676 0.037
Pairwise
RaM-LamblaRank
0.918 = 0.020
0.949 = 0.010
0.528 0.020
0.962 = 0.020
0.650 + 0.039
6./ 10
RaM-RankCosine
0.940 = 0.028
0.951 = 0.017
0,514 : 0.026
0.982 = 0.012
0.675 a 0.049
3.2/
RaM-Softmax
0.014
0.954 0.011
0.509 0.028
0.918 0.039
0.489 0.1
6.2/10
RaM-ListNet
0.949 = 0.025
0.962 = 0.015
0.517 a 0.029
0,981 = 0.012
0.670 a: 0.035
200/10
Listwise
RisMLi
0.032
0.953 0.012
0.022
0.020
0.656 0.041
RaM- ADM-AgproNDDCG
0.926 + 0.031
0.952 + 0.004
0.507 + 0.010
0.936 + 0.069
0 + 0.058
7
26

---

Published as a conference paper at ICLR 2025
We find that MSE performs the best in all of 3 pointwise losses, RankCosine (Qin al. 2008 outperforms other pairwise losses, and ListNet (Cao et al.] 2007) obtains the highest average rank among listwise losses. Note that prevalent ranking losses such as ApproxNDCG (Bruch et al. 2019b) do not perform well in RaM. This might due to the simplicity of MLP, which cannot absorb complex information of conveyed by the trending powerful loss functions (Qin et al.) [202T] Pobrotyn al.) 2020). However in this work, we parameterize the surrogate model as a simple MLP for a fair comparison to the regression- baaee methods, and we will consider more complex modeling in our future work.
F.3 ABLATION STUDIES RESULTS ON MAIN MODULES
To better validate the effectiveness of the two moduels, data augmentation and output adaptation, of our method, we perform ablation studies based on the top-performing loss functions shown in Table MSE for pointwise loss, RankCosine for pairwise loss, and ListNet for listwise loss. The results in Table show that for each considered loss, RaM with data augmentation performs better than the na've approach which treats a batch of the dataset as a list to rank. The results in Table TO] show the benefit of using output adaptation. All of these ablation studies provide strongly positive support to the effectiveness of these two modules.
Table 9: Ablation studies data augmentation, considering learning with MSE, RankCosine, and ListNet, which are the rest-performing pointwise, pairwise, and listwise loss, respectively, as shown For each combination of loss and task, the better performance is Bolded. D(best) denotes
in Table &
the best score in the offline dataset.
Method
Ant
D' D'Kitty
Superconductor
TF-Bind-8
TF-Bind-10
Mean Rank
D(best)
0,565
0.884
0.400
0.439
0.467
RaM MSE (w/ Aug.)
0.933 a 0.032
0.957 + 0.013
0.507 e: 0.028
0.962 a: 0.031
0.674 a 0.044
3.7 6
RaM-MSE (wlo Aug.)
0.928 0.022
0.944 ::: 0.017
0.502 : 0.015
0.983 2 0.012
0.652 0.045
4.7/6
RuM-RankCosing (w/ Aug.)
0.940 0.028
0.951 = 0.017
0.514 = 0.026
0.982 : 0.012
0.675 = 0,049
2.2/6
RaM- RankCosine (w/o Aug.)
0.929 * 0.019
0.944 + 0.005
0.504 + 0.018
0.980 + 0.016
0.654 = 0.038
4.7/6
RaM-ListNet (w/ Aug.)
0.949 = 0.025
0,962 = 0.015
0.517 = 0.029
0,981 # 0.012
0.670 = 0.035
2.0/6
RaM-ListNe (w/o Aug.)
0.938 0.025
0.964 = 0.011
0.507 0.007
0.975 a 0.010
0.640 a 0.037
3
Table 10: Ablation studies on output adaptation, considering learning with MSE, RankCosine, and ListNet, which are the best-performing pointwise, pairwise, and listwise loss, respectively, as shown in Table 8 For each combination of loss and task, the better performance is Bolded. D(best) denotes
the best score in the offline dataset.
Method
Aat
D' D'Kitty
Superconductor
TF-Bind-8
TF-Bind-10
Mean Rank
D(best)
0.565
0.884
0.400
0.439
0.467
RaM MSE (w/ Adapt.)
0.933 + 0.032
0.957 # 0.013
0.507 = 0.028
0.962 0.031
0.674
0,044
3.8/
RaM-MSE (w/o Adupt.)
0,913 0.028
0.953
0.012
0.506
0.024
0.966
0.023
0.653 0.030
5.2/6
RaM- RankCosine (w/ Adapt.)
0.940 = 0.028
0.951 + 0.017
0.514 = 0.1126
0.982 0.012
0.675 0.049
2.7/6
RaM- RankCosine (wlo Adapt.)
0.908 1: 0.023
0.955 a: 0.015
0.514 =: 0.025
0.970 1 0.016
0.649 0.019
4.5/6
RaM-ListNet Adapt.)
0.949 0.025
0.962 a: 0.015
0.517 0.029
0.981 0.012
0.670
0.035
1.6/6
RaM ListNet (w/ A
0,932 a: 0.034
0,961 a: 0.013
0,516 a 0.029
0.968 0.016
0.655 a 0.015
3.2/6
F.4 ABLATION OF THE LIST LENGTH
Note that the list length m in the training data could have a impact on the generalization ability of the model (Lan et al.) 2009; Tewari & Chaudhuri] 2015)) and its impact on OOD generalization ability of LTR algorithms is undiscovered (Chapelle et al. 2010). Besides, popular benchmarks in LTR QQin
al. 2010b Qin & Liu 2013 Chapelle & Chang 2011 Dato et al., 2016) have different settings of the list length, ranging from 5 to 1000.
Hence, meet the settings of different LTR benchmarks and to better understand the sensitivity of RaM-ListNet with respect to m, we conduct a careful ablation study of the setting of list length m, with values varying in (10, 20, 50, 100, 200, 500, 1000, 1500, 2000), shown in Table 11
27

---

Published as a conference paper at ICLR 2025
Table 11: 100th percentile normalized score in Design-Bench of RaM-ListNet with varying values of m. where the best and runner-up results on each task are Blue and Violet. D(best) denotes the best score in the offline dataset.
m
Ant
D'Kitty
Superconductor
TF-Bind-8
TF-Bind-l
Mcan Rank
((best)
0.565
0.884
0.400
0.439
0.467
10
0.916
0.014
0,953 # 0.016
0.508 0.017
0.984 # 0.007
0.655 # 0.063
6.6/9
20
0.913 a 0.026
0.954 0.011
0,518 + 0.022
0,972 # 0.016
0.670 = 0.043
6,2/9
50
0,930 0.035
0.963 = 0.014
0.524 0.021
0.978 # 0.014
0.653 = 0.046
3,5/9
0.922 0.024
0.963 = 0.015
0.525 0.020
# 0.016
0.702 = 0.129
3.4/9
200
0.927 0.023
0.960
0.526 0.020
0.977 0.015
0.050
4.6/9
500
0.920 0.028
0.962 = 0.007
0.518 + 0.028
0.975 + 0.011
0.699 0.128
4.0/9
1000
0.949 0.025
0.962 + 0.015
0.517 + 0.029
0.981 # 0.012
+670
3,4/9
1500
0,918 + 0.030
0.961 0.002
0,511 + 0.025
0,971 # 0.019
0.691 0.127
5,8/9
2000
0,905 0.035
0.958 0.016
0.515 0.027
0.967 # 0.020
0.664 = 0.043
7.5/9
we observe that RaM-ListNet obtains the best performance when 100 1000, and will undergo score drop gets relatively large, which demonstrates the need for careful tuning of the list length.
From the results in Table 11
F.5 RESULTS ON OOD METRICS
In this subsection, we also present the OOD-MSE results in Table 12 and OOD-AUPCC values in Table 13 As the results deliver, RaM combined with RankCosine or I ListNet perform poor in OOD-MSE, while they rank the best two in OOD-AUPCC. Coupled with the fact that RaM obtains the best performance in our main results (Table results on OOD-MSE and OOD-AUPCC further demonstrate that: 1) an algorithm with better OOD-AUPCC could result in better performance in offline MBO, no matter what its OOD-MSE is; 2) ranking loss is more suitable than MSE for offline MBO, since RaM obtains a better OOD-AUPCC compared to other regression- based methods.
Table 12: OOD-MSE of different methods in Design- Design-Bench, where the best and runner-up results on each task are Blue and Violet.
Method
Ant
D Kitty
Supercenductorr
TF-Bind-8
TF-Bind-10
Mean Rank
Ascent
0.444 0.123
0.229
+ 0.171
3
COMs
9.084 1 0.514
0,303 : 0.104
0.930 0.043
5,941 : 0.201
2,541 1 0.047
2.6/7
10M
9.520 0.948
0,299 0.062
0,798 0.041
8.779 : 0.364
2.594 0.069
3
ICT
160468.883
71634.341
8444.529 60,098
0.060
4.67
TriMMentoring
160641,446 146: 111.371
015557860222
25.979
8190.870 4 1.785
0.115 + 0.000
0.381 a: 0.030
4.4/7
RaM-RankCosine (Ours)
17.113 0.174
1,503 0.092
11.170 0.293
18.799 1.345
2.632 0.129
5.2/7
RaM-LlstNet (Ours)
25.011
2.689 0.249
8,999 2.449
11,546 3.295
0.315
4.677
Table 13: AUP-CUPCC of different methods in Design-Bench, where the best and runner-up results on each task are Blue and Violet.
Method
Ant
Kittty
Superconductor
TF-Bind-8
TF-Bind-10
Meun Rank
Grad. Ascent
0.363 = 0.028
0.403 =003=00 0.002
0.731 = 0.006
0.670 = 0.017
0.518 = 0.009
4.7/7
COMs
0.744 : 0.015
0.727 0.005
0.391 4: 0.028
0.433 0.002
0.505 4 0.001
4.4/7
IOM
0.649 = 0.044
0,648 : 0,085
0.436 : 0.057
0.428 + 0.007
0.515 : 0.056
4.6/7
ICT
0.443 = 0.095
0.549 0.089
0.596 = 0.100
0.658 = 0.015
0.545 =
4.6/7
Tri-Mentoring
0.346 0.000
0.403 0.000
0.740 0.003
0.690 0.000
0.571 + 0.000
3.7/7
Rank-eneCCoeee (Ours)
0.492 0.013
0.437
0.013
0.713 0.003
0.714 0.004
0.551 0.009
3
RaM-ListNet (Ours)
0.474 = 0.045
0.628 0.017
0.723 # 0.004
0.709 + 0.005
0.562 + 0.008
28/7
28


Reviews:
[{"review_id": "Af13NM3Xsg", "reviewer": "Reviewer_9jkq", "summary": "Offline model-based optimization (MBO) is concerned with the goal of maximizing an unknown objective function using an offline dataset. \n$\\min_{x \\in X} f(x)$ with unknown $f(.)$. The goal is to learn a proxy function $\\hat{f} (.)$. \nA dataset from past observations is available $[  (x_i, f(x_i) ]_{i=1}^n$. The training has to be done completely offline, because online evaluation is not possible in this setup.\nThe rudimentary approach is to train $\\hat{f} (.)$ by minimizing the MSE loss between $\\hat{f} (x_i)$ and $f (x_i)$. This vanilla approach has the problem of out-of-distribution issue, because $\\hat{f} (.)$ does not approximate well on unseen space within $X$.\nThis paper proposes to minimize a learning-to-rank (LTR) loss instead of the MSE loss and experimentally compare the proposed RankCosine and ListNet losses with the existing approaches.", "questions": "- In MBO problems, the exact value of the objective function is often critical. For example, when optimizing GPU performance, the goal is not only to minimize latency but also to know the precise latency value to assess the suitability of the design. Using LTR losses, however, does not guarantee that the output will match the true objective. Therefore, a potential drawback of the proposed approach is its failure to accurately estimate the objective function, which is highly relevant in real-world applications.\n- I am a bit puzzled by the definitions of Precision and Recall in Definition 1. In machine learning, Precision typically measures the ratio of true positives to all instances classified as positive, while Recall measures the ratio of true positives to the total number of ground-truth positives.  If a design is included in the top-k set, it should be considered a true positive. This suggests the numerator should be the same for both Precision and Recall, the dimension of the intersection and the denominator for both Precision and Recall should be k.\n- The out-of-distribution issue in MBO has been well studied. Adversarial weight perturbation [1] is one approach. Perturbing and minimizing the worst case performance among the performance is crucial to have better out-of-distribution issue and to improve generalization. Do you claim that the proposed approach do not require such perturbations?\n- To achieve generalization, methods like regularization, ensembling weak learners, and training under adversarial inputs are commonly used. It’s unclear how using an LTR loss alone could eliminate the need for such approaches. Even when training with an LTR loss on a static offline dataset, how you can guarantee that the ranking will generalize to out-of-distribution data.\n\n[1]: Yu, Tianhe, et al. \"Combo: Conservative offline model-based policy optimization.\" Advances in neural information processing systems 34 (2021): 28954-28967.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Originality**: This paper utilizes the existing LTR loss functions for offline MBO problems. This is a novel way to approach the offline MBO problem.\n\n**Quality**: I found some concerns which I will raise in the weakness section.\n\n**Clarity**: Overall, the paper has clarity.\n\n**Significance**: This new approach to offline MBO could be of significance, if the following questions are answered.", "weaknesses": "Although the approach is novel, there are concerns regarding how it can generalize performance and alleviate the out-of-distribution issue. It is not convincing that just plugging an LTR loss would address would solve the problem.\n(See Questions)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aZFbOQNVUi", "reviewer": "Reviewer_ChpN", "summary": "This paper proposes a new approach for offline optimization which focuses on training a surrogate to rank samples rather than to accurately predict their values. This surrogate is trained using the LTR framework with ListNet loss.", "questions": "Please see above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Paper was clearly written and original. To the best of my knowledge, the LTR framework was not previously used for offline optimization. Empirical results show positive improvement over previous baselines.", "weaknesses": "I think the idea that MSE is not always optimal for offline opt was recently pointed out in the MATCH-OPT paper (Hoang et al., 2024) (in which they use the term \"value-matching surrogates\" to describe model trained with MSE loss). Personally, I think that MSE is not worse than any other metric for optimization, but it could be harder to be estimated accurately for OOD samples. This seems to have been acknowledged in the MATCH-OPT paper. Section 3.1 and 3.2 of this paper seems to focus on justifying this insight using empirical evidence. 3.3 describes the LTR algorithm (which is not original), and 3.4 quotes some generalization result of LTR.\n\nRegarding 3.1 and 3.2, it makes no sense to compare the two metrics MSE and AUPRC. MSE evaluation does not use info about the top-k design (which has very strong correlation with the rank of the 100th percentile candidate), whereas AUPRC explicitly uses this info. It seems obvious to me that the latter would have higher Pearson correlation. Additionally, one cannot use the top-k info during training, so this would have little relevance to the actual method (which uses the ListNet loss). Most offline opt frameworks don't simply rely on the MSE loss though. As pointed out in the intro, there are some that use generative modeling or regularization to condition the surrogate model. How do those techniques relate to the idea of ranking samples?\n \nThe theory is, unfortunately, quite shallow and needs some extension. Theorem 1 (which is more of a proposition), states an equivalence, so the reverse argument could also be put forward -- why not MSE instead of ranking? As mentioned, I think the key challenge of offline opt is that we cannot control the surrogate's behavior OOD. Early error can lead the gradient search astray, so one should focus on quantifying this compounding effect.\n\nTheorem 2 is a direct quote from previous work, but it seems to add no value to the current setting. Does LTR generalize better than MSE on OOD data? I doubt that we can conclude anything general like that without a particular set of assumptions. So, the right question is, in which scenario will LTR generalize better than MSE on OOD data? I think this is important to answer this, especially when this paper is trying to advocate for a paradigm shift in offline opt. \n\nEmpirical results seem reasonable. However, given the landscape of offline opt research, in which every recent paper seems to report a different set of results on the Design Bench (typically suggesting that it is best), I find it hard to conclude that any method is more significant and robust than the others without proper theoretical backing. The paper should also compare with the recent DDOM method (Krisnamoorthy et al., 2023) which is representative of the generative modeling line of work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3UKfRwqF4O", "reviewer": "Reviewer_wgm9", "summary": "This paper challenges the use of MSE for training surrogate models in offline MBO, demonstrating its weak correlation with final performance. Instead, the authors propose a ranking-based model leveraging AUPRC, which achieves superior results compared to existing methods and suggests promising new directions for offline MBO.", "questions": "1. As mentioned by the authors in the introduction, one of the key challenges in offline optimization is Out-of-Distribution (OOD) data. While this ranking approach offers a better indication than the traditional OOD-MSE and aligns well with the offline optimization context, how do the authors address the OOD issue in the ranking setting? Could you provide a detailed analysis of the OOD problem in this context?\n\n2. In Definition 1, regarding Recall@k, why does the Recall@k value depend solely on k and N, with no relation to the model's predictions or true ranking? Additionally, this definition does not seem to reflect the model’s recall performance. Should the authors consider revising the definition, or reconsider using the term \"recall\" to refer to this variable?\n\n3. In Table 2, the results for backward methods like BONET[1] and GTG[2] appear to be much lower than those in the original paper (Table 2 in [1] and Table 1 in [2]). Could this be due to differences in experimental settings? If so, why don't the authors use the same configuration?\n\n[1] Generative pretraining for black-box optimization. ICML 2023. \n\n[2] Guided trajectory generation with diffusion models for offline model-based optimization. NeurIPS 2024.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Utilizing ranking scores in place of MSE to train the offline optimization surrogate model is an innovative approach. It aligns more intuitively with the objectives of offline optimization, making it a fitting metric for this context.\n\n- The algorithm is very clear and easy to understand. At the same time, this paper conducts a theoretical analysis of generalization error bound.\n\n- The authors conducte a wide range of experiments, providing comprehensive comparisons with the latest methods. The experimental results show the feasibility and effectiveness of the proposed approach.\n\n- This article is well-structured, with clear logic and easy-to-follow explanations that enhance readability and comprehension. \n\n- The supplementary material is sufficient and the algorithm can be fully reproduced without any problems. The appendices contain ample additional experiments and ablation studies. The appendices also contain a very comprehensive review of related work.", "weaknesses": "- Some concepts need to be clarified, such as OOD in the ranking model and Recall@k in Definition 1. These are raised in the \"questions\" below.\n\n- Typos. For example, \"e.g\" in line 48 should be \"e.g.\", \"caculated\" in line 241 should be \"calculated\". A thorough proofreading is recommended.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)