Offline Model-Based Optimization by Learning to Rank

## Abstract

Abstract Offline model-based optimization (MBO) aims to identify a design that maximizes a black-box function using only a fixed, pre-collected dataset of designs and their corresponding scores. This problem has garnered significant attention from both scientific and industrial domains. A common approach in offline MBO is to train a regression-based surrogate model by minimizing mean squared error (MSE) and then find the best design within this surrogate model by different optimizers (e.g., gradient ascent). However, a critical challenge is the risk of out-of-distribution errors, i.e., the surrogate model may typically overestimate the scores and mislead the optimizers into suboptimal regions. Prior works have attempted to address this issue in various ways, such as using regularization techniques and ensemble learning to enhance the robustness of the model, but it still remains. In this paper, we argue that regression models trained with MSE are not well-aligned with the primary goal of offline MBO, which is to select promising designs rather than to predict their scores precisely. Notably, if a surrogate model can maintain the order of candidate designs based on their relative score relationships, it can produce the best designs even without precise predictions. To validate it, we conduct experiments to compare the relationship between the quality of the final designs and MSE, finding that the correlation is really very weak. In contrast, a metric that measures order-maintaining quality shows a significantly stronger correlation. Based on this observation, we propose learning a ranking-based model that leverages learning to rank techniques to prioritize promising designs based on their relative scores. We show that the generalization error on ranking loss can be well bounded. Empirical results across diverse tasks demonstrate the superior performance of our proposed ranking-based models than twenty existing methods.

## 1 Introduction

The task of creating new designs to optimize specific properties represents a significant challenge across scientific and industrial domains, including real-world engineering design (Tanabe & Ishibuchi, ; Kumar et al., ), protein design (Khan et al., ; Kolli, ; Chen et al., ; Kim et al., ), and molecule design (Gaulton et al., ; Stanton et al., ). Numerous methods facilitate the generation of new designs by iteratively querying an unknown objective function that correlates a design with its property score. Nonetheless, in practical scenarios, the evaluation of the objective function can be time-consuming, costly, or even pose safety risks (Dara et al., ). To identify the next candidate design using only accumulated data, offline model-based optimization (MBO; Trabucco et al., ) has emerged as a widely adopted approach. This method restricts access to an offline dataset and does not allow for iterative online evaluation, which, however, also results in significant challenges. A common strategy, referred to as the forward method, entails the development of a regression-based surrogate model by minimizing mean squared error (MSE), which is subsequently utilized to identify the optimal designs by various ways (e.g, gradient ascent).

The main challenge of offline MBO is the risk of out-of-distribution (OOD) errors, i.e., the scores in OOD regions may be overestimated and mislead the gradient-ascent optimizer into suboptimal regions, as shown in Figure [1](#S1.F1)(a). Thus, overcoming the OOD issue has been the focus of recent works, such as using regularization techniques (Trabucco et al., ; Fu & Levine, ; Yu et al., ; Chen et al., ; Qi et al., ; Hoang et al., ; Dao et al., ) and ensemble learning (Yuan et al., ; Chen et al., ) to enhance the robustness of the model, but it still remains.

In this paper, instead of continuing to decrease the MSE in the OOD region (denoted as OOD-MSE), we first consider whether the currently mainstream offline MBO methods, specifically those based on regression-based surrogate models and optimizing MSE, are appropriate. We aim to answer the key question: “Is MSE a good metric for offline MBO?” Consequently, we find through experiments that the relationship between the quality of the final designs and OOD-MSE is weak, which underscores the need for a more reliable evaluation metric.

Next, we reconsider the primary goal of offline MBO, which seeks to identify the optimal design $\mathbf{x}^{*}$ over the entire design space. Intuitively, this process does not require exact score predictions from the surrogate model; rather, it demands that the model accurately discerns the partial ordering of designs. As shown in Figure [1](#S1.F1)(b), if a surrogate model can maintain the order of candidate designs based on their relative score relationships, it can produce the best designs even without precise predictions. We prove the equivalence of optima for order-preserving surrogates, and introduce a ranking-related metric, Area Under the Precision Recall Curve (AUPRC), for offline BBO, which shows a significantly stronger correlation with the final performance.

Figure: Figure 1: Illustration of (a) OOD issue of regression-based models and (b) order-preserving ranking-based models. In (a), the regression-based method searches into suboptimal regions. Prior works focus on high OOD-MSE, while in this work, we point out that it is caused by the OOD error in preserving order. In (b), although the surrogate model also has high OOD-MSE, it can maintain the order, thus resulting in good design candidates.
Refer to caption: /html/2410.11502/assets/x1.png

Based on this observation, we propose learning a Ranking-based Model (RaM) that leverages learning to rank (LTR) techniques to prioritize promising designs based on their relative scores. Our proposed method has three components: 1) data augmentation to make the offline dataset align with LTR techniques; 2) LTR loss learning to train the RaM; 3) output adaptation to make gradient ascent optimizers work well in RaM. We show that the generalization error on ranking loss can be well bounded, and conduct experiments on the widely used benchmark Design-Bench (Trabucco et al., ). Equipped with two popular ranking losses, i.e., RankCosine (Qin et al., ) and ListNet (Cao et al., ), our proposed method, RaM, performs better than state-of-the-art offline MBO methods. Ablation studies highlight the effectiveness of the main modules of RaM. We also examine the influence of different ranking loss, and demonstrate the versatility of ranking loss, bringing improvement even by simply replacing the MSE loss of existing methods with ranking loss.

The contributions of this work are highlighted in three key points:
1) To the best of our knowledge, we are the first to indicate that MSE is not suitable for offline MBO.
2) We show that the ranking-related metric AUPRC is well-aligned with the primary goal of offline MBO, and propose a ranking-based model for offline MBO.
3) We conduct comprehensive experiments across diverse tasks, showing the superiority of our proposed ranking-based model over a large variety of state-of-the-art offline MBO methods.

## 2 Background

### 2.1 Offline Model-Based Optimization

Given the design space $\mathcal{X}\subseteq\mathbb{R}^{d}$, where $d$ is the design dimension, offline MBO (Trabucco et al., ; Xue et al., ) aims to find a design $\bm{\mathrm{x}}^{*}$ that maximizes a black-box objective function $f$, i.e., $\bm{\mathrm{x}}^{*}=\mathop{\arg\max}\nolimits_{\bm{\mathrm{x}}\in\mathcal{X}}f(\bm{\mathrm{x}})$, using only a pre-collected offline dataset $\mathcal{D}$, without access to online evaluations. That is, an offline MBO algorithm is provided only access to the static dataset $\mathcal{D}=\{(\bm{\mathrm{x}}_{i},y_{i})\}_{i=1}^{N}$, where $\bm{\mathrm{x}}_{i}$ represents a specific design (e.g., a superconductor material), and $y_{i}=f(\bm{\mathrm{x}}_{i})$ represents the target property score that needs to be maximized (e.g., the critical temperature of the superconductor material).

The mainstream approach for offline MBO is the forward approach, which fits a surrogate model, typically a deep neural network $\hat{f}_{\bm{\theta}}:\mathcal{X}\to\mathbb{R}$, parameterized by $\bm{\theta}$, to approximate the objective function $f$ in a supervised manner.
Prior works (Trabucco et al., ; Fu & Levine, ; Yu et al., ; Qi et al., ; Chen et al., ; Yuan et al., ; Chen et al., ; Chemingui et al., ; Hoang et al., ; Dao et al., ) learn the surrogate model by minimizing the MSE between the predictions and the true scores:

$$ $\mathop{\arg\min}_{\bm{\theta}}\sum\nolimits_{i=1}^{N}\left(\hat{f}_{\bm{\theta}}(\bm{\mathrm{x}}_{i})-y_{i}\right)^{2}/N.$ $$

With the trained model $\hat{f}_{\bm{\theta}}$, the final design can be obtained by various ways, typically gradient ascent:

$$ $\bm{\mathrm{x}}_{t+1}=\bm{\mathrm{x}}_{t}+\eta\left.\nabla_{\bm{\mathrm{x}}}\hat{f}_{\bm{\theta}}(\bm{\mathrm{x}})\right|_{\bm{\mathrm{x}}=\bm{\mathrm{x}}_{t}},~{}~{}\text{for }t\in\{0,1,\ldots,T-1\}$ (1) $$

where $\eta$ is the search step size, $T$ is the number of steps, and $\bm{\mathrm{x}}_{T}$ serves as the final design candidate to output. However, this method is limited by its poor performance in out-of-distribution (OOD) regions, where the surrogate model $\hat{f}_{\bm{\theta}}$ may erroneously overestimate objective scores and mislead the gradient-ascent optimizer into sub-optimal regions. There have been many recent efforts devoted to addressing this issue, such as using regularization techniques(Trabucco et al., ; Fu & Levine, ; Yu et al., ; Chen et al., ; Qi et al., ; Hoang et al., ; Dao et al., ) and ensemble learning (Yuan et al., ; Chen et al., ) to enhance the robustness of the model.

Another type of approach for offline MBO is the backward approach, which typically involves training a conditioned generative model $p_{\bm{\theta}}(\mathbf{x}|y)$ and sampling from it conditioned on a high score. For example, MINs (Kumar & Levine, ) trains an inverse mapping using a conditioned GAN-like model (Goodfellow et al., ; Mirza & Osindero, );
DDOM (Krishnamoorthy et al., ) directly parameterizes the inverse mapping with a conditional diffusion model (Ho et al., ) in the design space; BONET (Krishnamoorthy et al., ) uses trajectories to train an autoregressive model, and samples them using a heuristic.

A comprehensive review of the offline MBO methods is provided in Appendix [A](#A1) due to space limitation. However, we point out in this paper that the regression-based models trained with MSE are not well-aligned with the primary goal of offline MBO, which is to select promising designs rather than to predict their scores precisely. Intuitively, offline MBO does not require exact score predictions from the surrogate model; rather, it demands that the model accurately discerns the partial ordering of designs, which aligns well with the learning to rank (LTR) framework introduced in Section [2.2](#S2.SS2).

### 2.2 Learning to Rank

LTR aims to learn an optimal ordering for a given set of objects (e.g., designs in offline MBO), and has applications across various domains, including information retrieval (Liu, ; Li, ), recommendation systems (Karatzoglou et al., ), and language model alignment (Song et al., ; Liu et al., ). It is typically formulated as a supervised learning task. Given the training data $\mathcal{D}_{\text{R}}=\{(\mathbf{X},\mathbf{y})\mid(\mathbf{X},\mathbf{y})\in\mathcal{X}^{m}\times\mathbb{R}^{m}\}$, where $\mathcal{X}$ is the object space, $\mathbf{X}$ is a list of $n$ objects to be ranked, each denoted by $\mathbf{x}_{i}\in\mathcal{X}$, and $\mathbf{y}$ is a list of $n$ corresponding relevance labels $y_{i}\in\mathbb{R}$, the goal of LTR is to learn a ranking function that assigns scores to individual objects and then arranges these scores in descending order to produce a ranking.
Formally, LTR aims to identify a ranking score function $s_{\bm{\theta}}:\mathcal{X}\to\mathbb{R}$, parameterized by $\bm{\theta}$. Let $s_{\bm{\theta}}(\mathbf{X})=[s_{\bm{\theta}}(\mathbf{x}_{1}),s_{\bm{\theta}}(\mathbf{x}_{2}),\ldots,s_{\bm{\theta}}(\mathbf{x}_{m})]^{\top}$, and we can optimize the model by minimizing the empirical loss:

$$ $\mathcal{L}(s_{\bm{\theta}})=\sum\nolimits_{(\mathbf{X},\mathbf{y})\in\mathcal{D}_{\text{R}}}l\left(\mathbf{y},s_{\bm{\theta}}(\mathbf{X})\right)/|\mathcal{D}_{\text{R}}|,$ $$

where $l(\cdot)$ is the loss function applied to each list of objects.
Depending on their approach to handling ranking loss, LTR algorithms are generally categorized into three paradigms: 1) Pointwise (Crammer & Singer, ): Treat ranking as a regression or classification problem on individual objects; 2) Pairwise (Köppel et al., ): Transform ranking into a binary classification problem on object pairs;
3) Listwise (Xia et al., ): Directly optimize the ranking of the entire list of objects.

## 3 Method

In this section, we introduce our ranking-based surrogate models for offline MBO. We first analyze in detail the goal of offline MBO and aim to answer the critical question, “Is MSE a good metric for offline MBO?” in Section [3.1](#S3.SS1). Consequently, we find that MSE is not a suitable metric, and thus introduce a better one in Section [3.2](#S3.SS2), i.e., Area Under the Precision-Recall Curve (AUPRC) which is related to ranking. This motivates us to propose a framework based on LTR to solve offline MBO in Section [3.3](#S3.SS3). Furthermore, we show that the surrogate model based on LTR methods can have a good generalization error bound, which will be shown in Section [3.4](#S3.SS4).

### 3.1 Is MSE A Good Metric for Offline MBO?

An ideal metric should be able to accurately assess the goodness of a surrogate model, i.e., the better the metric, the better the quality of the final design obtained using the surrogate model. As shown in Eq. ([1](#S2.E1)), $\mathbf{x}_{T}$, which approximately maximizes the surrogate model $\hat{f}_{\bm{\theta}}$ by gradient ascent, serves as the final design to output. During the optimization process, it will inevitably traverse the OOD region. Therefore, the performance of the surrogate model in the OOD region will significantly impact the performance of offline MBO. Unfortunately, previous works (Trabucco et al., ; ) have shown that the regression-based models optimized using MSE often result in poor predictions in the OOD region, i.e., the MSE value in the OOD region (denoted as OOD-MSE) can be very high, and thus many methods have been proposed to decrease OOD-MSE (Fu & Levine, ; Chen et al., ; Yuan et al., ) or avoid getting into OOD regions (Trabucco et al., ; Yu et al., ; Qi et al., ). In this paper, however, we indicate that even if OOD-MSE is small, the final performance of offline MBO can still be bad. That is, the relationship between the quality of the final designs and OOD-MSE is weak. In the following, we will validate this through experiments.

To analyze the correlation between the OOD-MSE of a surrogate model and the score of the final design candidate obtained by conducting gradient ascent on the surrogate model, we select five surrogate models: a gradient-ascent baseline and four state-of-the-art forward approaches, COMs (Trabucco et al., ), IOM (Qi et al., ), ICT (Yuan et al., ), and Tri-Mentoring (Chen et al., ).
We follow the default setting as in (Chen et al., ; Yuan et al., ) for data preparation and model-internal search procedures.
To construct an OOD dataset, we follow the approach outlined in (Chen et al., ), selecting high-scoring designs that are excluded from the training data in Design-Bench (Trabucco et al., ).
Detailed information regarding model selection, training and search configurations, and OOD dataset construction can be found in Appendix [D.1](#A4.SS1).
We train the surrogate models, evaluate their performance using various metrics (e.g., MSE) on the OOD dataset, and obtain the final design with its corresponding ground-truth score under eight different seeds.
Subsequently, we rank the OOD-MSE values in ascending order, and rank the 100th percentile scores of the final designs in descending order.
To show the correlation between OOD-MSE and the final score, we create scatter plots of the two rankings and calculate their Spearman correlation coefficient.

The left two subfigures of Figure [2](#S3.F2) show the scatter plots on a continuous task, D’Kitty (Ahn et al., ), and a discrete task, TF-Bind-8 (Barrera et al., ). Both scatter plots exhibit highly dispersed data points, with no clear overall trend or strong clustering, showing no consistent pattern in their distribution. This scattered nature of the data points is also reflected in the low Spearman correlation coefficients ($0.23$ for D’Kitty and $-0.24$ for TF-Bind-8), indicating weak correlations between OOD-MSE rank and score rank in both tasks. These results demonstrate that OOD-MSE is not a good metric for offline MBO, underscoring the need for a more reliable evaluation metric.

Figure: Figure 2: Scatter plots of five surrogate models (each trained using eight seeds) on the two tasks of D’Kitty and TF-Bind-8, where the $y$-axis denotes the rank of the 100th percentile score, and the $x$-axis denotes the rank of the metric in the OOD region, i.e., OOD-MSE or OOD-AUPRC. The Spearman correlation coefficients are also caculated, as shown in the top of each subfigure.
Refer to caption: /html/2410.11502/assets/x2.png

### 3.2 What is the Appropriate Metric for Offline MBO?

As we mentioned before, an intuition of offline MBO is that the goodness of a surrogate model may depend on its ability to preserve the score ordering of designs dictated by the ground-truth function. We substantiate this intuition through the following theorem.

###### Theorem 1 (Equivalence of Optima for Order-Preserving Surrogates) .

Let $\hat{f}_{\bm{\theta}}$ be a surrogate model and $f$ the ground-truth function. A function $h:\mathbb{R}\to\mathbb{R}$ is order-preserving, if $\forall y_{1},y_{2}\in\mathbb{R}$, $y_{1}<y_{2}$ iff $h(y_{1})<h(y_{2})$. If there exists an order-preserving $h$ such that $\hat{f}_{\bm{\theta}}(\mathbf{x})=h(f(\mathbf{x}))\,\forall\mathbf{x}$, then finding the maximum of $f$ is equivalent to finding that of $\hat{f}_{\bm{\theta}}$, i.e., $\mathop{\arg\max}\nolimits_{\mathbf{x}\in\mathcal{X}}f(\mathbf{x})=\mathop{\arg\max}\nolimits_{\mathbf{x}\in\mathcal{X}}\hat{f}_{\bm{\theta}}(\mathbf{x})$.

###### Proof.

Suppose $\mathbf{x}^{*}\in\mathop{\arg\max}_{\mathbf{x}}f(\mathbf{x})$. For any $\mathbf{x}$, we have
$f(\mathbf{x}^{*})\geq f(\mathbf{x})$.
Since $h$ is order-preserving, we have
$h(f(\mathbf{x}^{*}))\geq h(f(\mathbf{x}))$ for all $\mathbf{x}$.
Thus, given $\hat{f}_{\bm{\theta}}(\mathbf{x})=h(f(\mathbf{x}))$, we have $\hat{f}_{\bm{\theta}}(\mathbf{x}^{*})\geq\hat{f}_{\bm{\theta}}(\mathbf{x})$ for all $\mathbf{x}$.
Therefore, $\mathbf{x}^{*}\in\mathop{\arg\max}_{\mathbf{x}}\hat{f}_{\bm{\theta}}(\mathbf{x})$, i.e., $\mathop{\arg\max}_{\mathbf{x}}f(\mathbf{x})\subseteq\mathop{\arg\max}_{\mathbf{x}}\hat{f}_{\bm{\theta}}(\mathbf{x})$.
Note that since $h$ is strictly increasing, it is bijective and thus has an inverse function $h^{-1}$, which is also strictly increasing. With $h^{-1}$, the reverse implication follows similarly, proving the equivalence.
∎

Theorem [1](#Thmtheorem1) shows that a good surrogate model needs to maintain an order-preserving mapping from the ground-truth model. In the practical setting of offline MBO, the standard procedure is to select the top-$k$ designs (e.g., $k=128$), which maximize the surrogate model’s predictions, for evaluation (Trabucco et al., ). Thus, we introduce a novel metric, Area Under the Precision-Recall Curve (AUPRC) in Definition [1](#Thmdefinition1) (Raghavan et al., ; Davis & Goadrich, ; Ricci et al., ), for offline MBO to assess the model’s capability in identifying the top-$k$ ones from a set of candidate designs. Although AUPRC is traditionally associated with imbalanced classification tasks, it can be interpreted as a listwise ranking metric (Wen et al., ), which enables us to employ AUPRC for evaluating a model’s ability to select the top-$k$ designs within the context of offline MBO.

###### Definition 1 (AUPRC for Offline MBO) .

Consider a surrogate model $\hat{f}_{\bm{\theta}}$ and a ground-truth function $f$.
Given a dataset $\mathcal{D}_{0}=\{(\mathbf{x}_{i},y_{i})\}_{i=1}^{N}$, denote $\{\hat{f}_{\bm{\theta}}(\mathbf{x}_{i})\}_{i=1}^{N}$ as $\hat{f}_{\bm{\theta}}(\mathcal{D}_{0})$, and $\{f(\mathbf{x}_{i})\}_{i=1}^{N}$ as $f(\mathcal{D}_{0})$.
Let $\operatorname{top}_{k}(S)$ denote the set of the $k$ largest elements in set $S$. For each $k\in\{1,2,...,N\}$,

$$ $\displaystyle\operatorname{Precision}@k$ $\displaystyle=\frac{|\operatorname{top}_{k}(\hat{f}_{\bm{\theta}}(\mathcal{D}_{0}))\cap\operatorname{top}_{k}(f(\mathcal{D}_{0}))|}{|\operatorname{top}_{k}(f(\mathcal{D}_{0}))|}=\frac{|\operatorname{top}_{k}(\hat{f}_{\bm{\theta}}(\mathcal{D}_{0}))\cap\operatorname{top}_{k}(f(\mathcal{D}_{0}))|}{k},$ $\displaystyle\operatorname{Recall}@k$ $\displaystyle=|\operatorname{top}_{k}(\hat{f}_{\bm{\theta}}(\mathcal{D}_{0}))\cap\mathcal{D}_{0}|/|\mathcal{D}_{0}|=k/N.$ $$

Note that we let $\mathcal{D}_{0}$ be the relevant set in the calculation of $\operatorname{Recall}@k$ for simplification.
The Precision-Recall curve is obtained by plotting $\operatorname{Precision}@k$ against $\operatorname{Recall}@k$ for all values of $k$. Then, the AUPRC is defined as the area under this curve:

$$ $\displaystyle\operatorname{AUPRC}$ $\displaystyle\approx\sum_{k=1}^{N-1}(\operatorname{Recall}@(k+1)-\operatorname{Recall}@k)\cdot\frac{\operatorname{Precision}@(k+1)+\operatorname{Precision}@k}{2}.$ $$

The AUPRC metric for offline MBO can effectively evaluate a model’s ability to identify top-$k$ designs with varying $k$ and thus the ability to preserve order across the entire design space. A higher AUPRC value indicates better performance in ranking and selecting better designs. We visualize the correlation between OOD-AUPRC (i.e., the AUPRC value in the OOD region) rank and score rank in the right two subfigures of Figure [2](#S3.F2), where the rank of OOD-AUPRC is obtained in descending order.
In contrast to the OOD-MSE results, the scatter plots of OOD-AUPRC exhibit clear upward trends, with data points clustered more tightly around the diagonal compared to their OOD-MSE counterparts. This improved correlation is also verified by the substantially higher Spearman correlation coefficients, 0.64 for D’Kitty and 0.52 for TF-Bind-8.

To further validate the reliability of AUPRC compared to MSE, we conduct a quantitative analysis, incorporating another three tasks, Superconductor (Hamidieh, ) and Ant (Brockman et al., ) in continuous space, and TF-Bind-10 (Barrera et al., ) in discrete space. We evaluate the metrics in the OOD regions and the final scores, and calculate Spearman correlation coefficients between the two rankings, following the same approach as in our previous analysis. The results in Table [1](#S3.T1) demonstrate the superior performance of OOD-AUPRC compared to OOD-MSE in correlating with the 100th percentile score across various offline MBO tasks. OOD-AUPRC consistently shows stronger correlations than OOD-MSE, with an average improvement of 0.364 in correlation strength. Notably, OOD-AUPRC achieves positive or significantly improved correlations even in the tasks where OOD-MSE shows negative correlations, such as Superconductor and TF-Bind-8 tasks.
Coupled with Theorem [1](#Thmtheorem1), which establishes the relationship between a model’s order-preserving capability and its final performance, the consistently stronger empirical correlations confirm that OOD-AUPRC is indeed a more effective and reliable metric than OOD-MSE for evaluating the performance of a surrogate model in offline MBO. In the next section, we will discuss how to use LTR techniques to optimize the AUPRC, thus to obtain high-scoring designs.

**Table 1: Comparison between Spearman correlation coefficients of OOD-MSE and OOD-AUPRC with respect to the 100th percentile score.**
| OOD-Metric | Ant | D’Kitty | Superconductor | TF-Bind-8 | TF-Bind-10 |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Coef. | Gain | Coef. | Gain | Coef. | Gain | Coef. | Gain | Coef. | Gain |  |
| OOD-MSE | 0.161 |  | 0.243 |  | -0.116 |  | -0.239 |  | -0.573 |  |
| OOD-AUPRC | 0.257 | +0.096 | 0.503 | +0.260 | 0.101 | +0.217 | 0.520 | +0.759 | -0.087 | +0.486 |

###### Theorem 1 (Equivalence of Optima for Order-Preserving Surrogates) .

###### Proof.

###### Definition 1 (AUPRC for Offline MBO) .

### 3.3 Offline MBO by Learning to Rank: A Practical Algorithm

In this section, in order to optimize AUPRC for the surrogate model, we design a novel framework for offline MBO based on LTR, as shown in Algorithm [1](#alg1), which consists of three parts: 1) data augmentation; 2) LTR loss learning; 3) output adaptation.

Figure: Algorithm 1 Offline MBO by Learning to Rank

Data augmentation.
In LTR tasks, the training set $\mathcal{D}_{\text{R}}$ typically requires a list of designs as features. However, the offline dataset $\mathcal{D}$ in offline MBO is not directly structured in this manner, thus the LTR loss functions cannot be directly applied.
A naïve approach to address this issue is to treat each batch of training data as a list of designs to be ranked, with the batch size determining the list length.
However, this method has its limitation since each design in the training data appears in only one list during one single epoch, which is unable to analyze its relationship with other designs that are not in the list. To address this limitation, we propose a simple yet effective data augmentation method.
We randomly sample $m$ design-score pairs $\{(\mathbf{x}_{i},y_{i})\}_{i=1}^{m}$ from $\mathcal{D}$, and concatenate them to form a design list $\mathbf{X}=[\mathbf{x}_{1},\mathbf{x}_{2},\ldots,\mathbf{x}_{m}]^{\top}$ and its score list $\mathbf{y}=[y_{1},y_{2},\ldots,y_{m}]^{\top}$; then repeat this step for $n$ times to construct a dataset $\mathcal{D}_{\text{R}}=\{(\mathbf{X}_{i},\mathbf{y}_{i})\}_{i=1}^{n}$ for LTR modeling.
We will discuss the setting of $n$ and $m$ in Section [4.1](#S4.SS1), and show the benefit of data augmentation over the naïve approach in Section [4.2](#S4.SS2).

LTR loss learning.
In Section [3.2](#S3.SS2), we have discussed that AUPRC is a listwise metric for ranking, and thus we can use the well-studied ranking loss (Li, ) from the field of LTR to optimize the AUPRC on the training distribution, so as to generalize to the OOD regions. We study a wide range of ranking losses, including pointwise (Crammer & Singer, ), pairwise (Köppel et al., ), and listwise (Xia et al., ) losses.
Here we take RankCosine (Qin et al., ), a pairwise loss, and ListNet (Cao et al., ), a listwise loss, for example. The idea of RankCosine is to measure the difference between predicted and true rankings using cosine similarity, operating directly in the score space.
Formally, given a list $\mathbf{X}$ of designs and the list $\mathbf{y}$ of their corresponding scores, let $\hat{f}_{\bm{\theta}}(\mathbf{X})=[\hat{f}_{\bm{\theta}}(\mathbf{x}_{1}),\hat{f}_{\bm{\theta}}(\mathbf{x}_{2}),\ldots,\hat{f}_{\bm{\theta}}(\mathbf{x}_{m})]^{\top}$ be the predicted scores. The RankCosine loss function is:

$$ $l_{RankCosine}(\mathbf{y},\hat{f}_{\bm{\theta}}(\mathbf{X}))=1-\mathbf{y}\cdot\hat{f}_{\bm{\theta}}(\mathbf{X})/(\|\mathbf{y}\|\cdot\|\hat{f}_{\bm{\theta}}(\mathbf{X})\|).$ $$

The idea of ListNet is to minimize the cross-entropy between the predicted ranking distribution and the true ranking distribution, which is defined as:

$$ $\displaystyle l_{ListNet}(\mathbf{y},\hat{f}_{\bm{\theta}}(\mathbf{X}))=-\sum_{j=1}^{m}\frac{\exp(y_{j})}{\sum_{i=1}^{m}\exp(y_{i})}\log\frac{\exp(\hat{f}_{\bm{\theta}}(\mathbf{x}_{j}))}{\sum_{i=1}^{m}\exp(\hat{f}_{\bm{\theta}}(\mathbf{x}_{i}))}.$ $$

We provide detailed description of other ranking losses in Appendix [C](#A3), and compare their effectiveness for offline MBO in Section [4.2](#S4.SS2). We also provide detailed information of model training in Section [4.1](#S4.SS1).

Output adaptation.
The surrogate model trained with ranking loss has a crucial issue for the hyper-parameter setting of gradient-ascent optimizers.
Unlike MSE, which aims for accurate prediction of target scores, ranking losses do not require precise estimation of target scores.
This shift in objective may lead to significant changes in the scale of model predictions, and thus impact the magnitude of gradients, making it challenging to determine appropriate values for the search step size $\eta$ and the number $T$ of search steps in Eq. ([1](#S2.E1)).
Moreover, different ranking losses can result in different output scales, which necessitate careful hyper-parameter tuning for a specific loss.

Notably, the scores in the training data for regression-based models have a statistical characteristic of zero mean and unit standard deviation after z-score normalization (Trabucco et al., ; ), and the trained regression-based model will try to preserve these statistical properties within the training distribution.
Consequently, to mitigate the impact of varying scales across different loss functions and to ensure a fair comparison with the regression-based models, we normalize the predictions of the ranking model after it is trained.
Specifically, we first apply the trained model to the entire training set and calculate the mean value $\tilde{\mu}$ and standard deviation $\tilde{\sigma}$ of the resulting predictions. Subsequently, we use $\tilde{\mu}$ and $\tilde{\sigma}$ to apply z-score normalization to the model’s prediction. Such normalization enables us to directly use the setting of $\eta$ and $T$ as in regression-based models. That is, we compute the gradient of the normalized predictions with respect to $\mathbf{x}$, and use the default hyper-parameters in (Chen et al., ; Yuan et al., ) to search for the final design candidate. We will examine the effectiveness of using output adaptation in Section [4.2](#S4.SS2).

### 3.4 Theoretical Analysis

In the previous subsections, we have indicated the importance of preserving the score order of designs, and proposed to learn a surrogate model by optimizing ranking losses. Here, we further point out that the generalization error can be well bounded in the context of LTR. Note that the generalization of LTR has been well studied (Agarwal et al., ; Lan et al., ; Chen et al., ; Tewari & Chaudhuri, ), which is mainly analyzed by the Probably Approximately Correct (PAC) learning theory (Cucker & Smale, ) and Rademacher Complexity (Bartlett & Mendelson, ).
By leveraging these existing generalization error bounds, we provide theoretical support for our approach of applying LTR techniques for offline MBO.

Formally, assume that we have an i.i.d. training data $\mathcal{D}_{\text{R}}=\{(\mathbf{X}_{i},\mathbf{y}_{i})\}_{i=1}^{n}$ where $\mathbf{X}_{i}\in\mathcal{X}^{m}$, consisting of $m$ designs, and $\mathbf{y}_{i}\in\mathbb{R}^{m}$. Given a ranking algorithm $\mathcal{A}$ (e.g., RankCosine or ListNet), its loss function $l_{\mathcal{A}}(f;\mathbf{X},\mathbf{y})$ is normalized by $l_{\mathcal{A}}(f;\mathbf{X},\mathbf{y})/Z_{\mathcal{A}}$, where $Z_{\mathcal{A}}$ is a normalization constant (e.g., $Z_{RankCosine}=1$).
The expected risk with respect to the algorithm $\mathcal{A}$ is defined as $R_{l_{\mathcal{A}}}(f)=\int_{\mathcal{X}^{m}\times\mathbb{R}^{m}}l_{\mathcal{A}}(f;\mathbf{X},\mathbf{y})P(\mathrm{d}\mathbf{X},\mathrm{d}\mathbf{y})$, and the empirical risk is defined as $\hat{R}_{l_{\mathcal{A}}}(f;\mathcal{D}_{\text{R}})=\frac{1}{n}\sum_{i=1}^{n}l_{\mathcal{A}}(f;\mathbf{X}_{i},\mathbf{y}_{i})$.
Let $\mathcal{F}$ be the ranking function class, and Theorem [2](#Thmtheorem2) gives an upper bound on the generalization error $\sup_{f\in\mathcal{F}}(R_{l_{\mathcal{A}}}(f)-\hat{R}_{l_{\mathcal{A}}}(f;\mathcal{D}_{\text{R}}))$.

###### Theorem 2 (Generalization Error Bound for LTR (Lan et al., 2009 ) ) .

Let $\phi$ be an increasing and strictly positive transformation function (e.g., $\phi(z)=\exp(z)$).
Assume that: 1) $\forall\mathbf{x}\in\mathcal{X},\,\|\mathbf{x}\|\leq M$; 2) the ranking model $f$ to be learned is from the linear function class $\mathcal{F}=\{\mathbf{x}\to\mathbf{w}^{\top}\mathbf{x}\mid\|\mathbf{w}\|\leq B\}$.
Then with probability $1-\delta$, the following inequality holds:

$$ $\sup\nolimits_{f\in\mathcal{F}}\big{(}R_{l_{\mathcal{A}}}(f)-\hat{R}_{l_{\mathcal{A}}}(f;\mathcal{D}_{\text{R}})\big{)}\leq 4BM\cdot C_{\mathcal{A}}(\phi)N(\phi)/\sqrt{n}+\sqrt{2\ln{(2/\delta)}/n},$ $$

where: 1) $\mathcal{A}$ stands for a specific LTR algorithm; 2) $N(\phi)=\sup_{z\in[-BM,BM]}\phi^{\prime}(z)$, which is an algorithm-independent factor measuring the smoothness of $\phi$; 3) $C_{\mathcal{A}}(\phi)$ is an algorithm-dependent factor, e.g., $C_{RankCosine}(\phi)=\sqrt{m}/(2\phi(-BM))$.

We will introduce some settings of $\phi$ and the corresponding $N(\phi)$ and $C_{\mathcal{A}}(\phi)$ in Appendix [B](#A2). We can observe from the inequality in Theorem [2](#Thmtheorem2) that the generalization error bound vanishes at the rate $\mathcal{O}(1/\sqrt{n})$, because $C_{\mathcal{A}}(\phi)$ and $N(\phi)$ are independent of the size $n$ of training set.

###### Theorem 2 (Generalization Error Bound for LTR (Lan et al., 2009 ) ) .

## 4 Experiments

In this section, we empirically compare the proposed method with a large variety of previous offline MBO methods on various tasks. First, we introduce our experimental settings, including five tasks, twenty compared methods, training settings, and evaluation metrics. Then, we present the results to show the superiority of our method. We also examine the influence of using different ranking losses, and conduct ablation studies to investigate the effectiveness of each module of our method. Furthermore, we simply replace MSE of existing methods with the best-performing ranking loss, to demonstrate the versatility of the ranking loss for offline MBO. Finally, we provide the metrics, OOD-MSE and OOD-AUPRC, in the OOD regions to validate their relationship with the final performance. Our implementation is available at [https://anonymous.4open.science/r/Offline-RaM-7FB1](https://anonymous.4open.science/r/Offline-RaM-7FB1).

### 4.1 Experimental Settings

Benchmark and tasks.
We benchmark our method on Design-Bench tasks (Trabucco et al., ), including three continuous tasks and two discrete tasks (^1^11Following recent works (Yun et al., ; Yu et al., ), we exclude three tasks from Design-Bench, and provide detailed explanations in Appendix [D.2](#A4.SS2).).
The continuous tasks include:
1) Ant Morphology (Brockman et al., ): identify an ant morphology with 60 parameters to crawl quickly.
2) D’Kitty Morphology (Ahn et al., ): optimize a D’Kitty morphology with 56 parameters to crawl quickly.
3) Superconductor (Hamidieh, ): design a 86-dimensional superconducting material to maximize the critical temperature.
The two discrete tasks are TF-Bind-8 and TF-Bind-10 (Barrera et al., ): find a DNA sequence of length 8 and 10, respectively, maximizing binding affinity with a particular transcription factor.

Compared methods. We mainly consider three categories of methods to solve offline MBO.
The first category involves baselines that optimize a trained regression-based model, such as BO-$q$EI (Garnett, ; Shahriari et al., ), CMA-ES (Hansen, ), REINFORCE (Williams, ), Gradient Ascent and its variants of mean ensemble and min ensemble.
The second category encompasses backward approaches, including CbAS (Brookes et al., ), MINs (Kumar & Levine, ), DDOM (Krishnamoorthy et al., ), BONET (Krishnamoorthy et al., ), and GTG (Yun et al., ).
The third category comprises recently proposed forward approaches, which contain COMs (Trabucco et al., ), RoMA (Yu et al., ), IOM (Qi et al., ), BDI (Chen et al., ), ICT (Yuan et al., ), Tri-Mentoring (Chen et al., ), PGS (Chemingui et al., ), FGM (Grudzien et al., ), and Match-OPT (Hoang et al., ) (^2^22Due to the lack of open-source implementations or inapplicability for comparison, we exclude NEMO (Fu & Levine, ), BOSS (Dao et al., ), DEMO (Yuan et al., ) and LEO (Yu et al., ). Detailed explanations are provided in Appendix [D.3](#A4.SS3).).

Training settings.
We set the size $n$ of training dataset to $10000$, and following LETOR 4.0 (Qin & Liu, ; Qin et al., ), a prevalent benchmark for LTR, we set the list length $m=1000$. To make a fair comparison to regression-based methods, following Trabucco et al. (; ); Chen et al. (); Yuan et al. (), we model the surrogate model $\hat{f}_{\bm{\theta}}$ as a simple multilayer perceptron with two hidden layers of size 2048 using PyTorch (Paszke et al., ).
We use ReLU as activation functions. RankCosine (Qin et al., ) and ListNet (Cao et al., ) will be used as two main loss functions in our experiments.
The model is optimized using Adam (Kingma & Ba, ) with a learning rate of $3\times 10^{-4}$ and a weight decay coefficient of $1\times 10^{-5}$.
After the model is trained, following Chen et al. (); Yuan et al. (), we set $\eta=1\times 10^{-3}$ and $T=200$ for continuous tasks, and $\eta=1\times 10^{-1}$ and $T=100$ for discrete tasks to search for the final design.
All experiments are conducted using eight different seeds.
Additional training details are provided in Appendix [D.4](#A4.SS4).

Evaluation and metrics.
For evaluation, we use the oracle from Design-Bench and follow the protocol of prior works (Trabucco et al., ; ). That is,
we identify $k=128$ most promising designs selected by an algorithm and report the 100th percentile normalized ground-truth score.
A design score $y$ is normalized via computing $(y-y_{\min})/(y_{\min}-y_{\max})$, where $y_{\min}$ and $y_{\max}$ denote the lowest and the highest scores in the full unobserved dataset from Design-Bench. We also provide the 50th percentile normalized ground-truth results in Appendix [E.1](#A5.SS1).

### 4.2 Experimental Results

Main results. In Table [2](#S4.T2), we report the results of our experiments, where our method based on Ranking Model is denoted as RaM appended with the name of the employed ranking loss. Among the compared 22 methods, RaM-RankCosine and RaM-ListNet achieve the two best average ranks, 2.7 and 2.2, respectively, while the third best method, BDI, only obtains an average rank of 5.9. We can observe that RaM-RankCosine performs best on one task, TF-Bind-10, and is runner-up on two tasks, Superconductor and TF-Bind-8; and RaM-ListNet performs best on two tasks, D’Kitty and Superconductor. These results clearly demonstrate the superior performance of our proposed method.

**Table 2: 100th percentile normalized score in Design-Bench, where the best and runner-up results on each task are Blue and Violet. $\mathcal{D}$(best) denotes the best score in the offline dataset.**
| Method | Ant | D’Kitty | Superconductor | TF-Bind-8 | TF-Bind-10 | Mean Rank |
| --- | --- | --- | --- | --- | --- | --- |
| $\mathcal{D}$(best) | 0.565 | 0.884 | 0.400 | 0.439 | 0.467 | / |
| BO-$q$EI | 0.812 ± 0.000 | 0.896 ± 0.000 | 0.382 ± 0.013 | 0.802 ± 0.081 | 0.628 ± 0.036 | 18.0 / 22 |
| CMA-ES | 1.712 ± 0.754 | 0.725 ± 0.002 | 0.463 ± 0.042 | 0.944 ± 0.017 | 0.641 ± 0.036 | 11.4 / 22 |
| REINFORCE | 0.248 ± 0.039 | 0.541 ± 0.196 | 0.478 ± 0.017 | 0.935 ± 0.049 | 0.673 ± 0.074 | 14.0 / 22 |
| Grad. Ascent | 0.273 ± 0.023 | 0.853 ± 0.018 | 0.510 ± 0.028 | 0.969 ± 0.021 | 0.646 ± 0.037 | 11.6 / 22 |
| Grad. Ascent Mean | 0.306 ± 0.053 | 0.875 ± 0.024 | 0.508 ± 0.019 | 0.985 ± 0.008 | 0.633 ± 0.030 | 11.2 / 22 |
| Grad. Ascent Min | 0.282 ± 0.033 | 0.884 ± 0.018 | 0.514 ± 0.020 | 0.979 ± 0.014 | 0.632 ± 0.027 | 11.5 / 22 |
| CbAS | 0.846 ± 0.032 | 0.896 ± 0.009 | 0.421 ± 0.049 | 0.921 ± 0.046 | 0.630 ± 0.039 | 15.5 / 22 |
| MINs | 0.906 ± 0.024 | 0.939 ± 0.007 | 0.464 ± 0.023 | 0.910 ± 0.051 | 0.633 ± 0.034 | 13.0 / 22 |
| DDOM | 0.908 ± 0.024 | 0.930 ± 0.005 | 0.452 ± 0.028 | 0.913 ± 0.047 | 0.616 ± 0.018 | 14.6 / 22 |
| BONET | 0.921 ± 0.031 | 0.949 ± 0.016 | 0.390 ± 0.022 | 0.798 ± 0.123 | 0.575 ± 0.039 | 15.1 / 22 |
| GTG | 0.855 ± 0.044 | 0.942 ± 0.017 | 0.480 ± 0.055 | 0.910 ± 0.040 | 0.619 ± 0.029 | 13.9 / 22 |
| COMs | 0.916 ± 0.026 | 0.949 ± 0.016 | 0.460 ± 0.040 | 0.953 ± 0.038 | 0.644 ± 0.052 | 9.5 / 22 |
| RoMA | 0.430 ± 0.048 | 0.767 ± 0.031 | 0.494 ± 0.025 | 0.665 ± 0.000 | 0.553 ± 0.000 | 18.3 / 22 |
| IOM | 0.889 ± 0.034 | 0.928 ± 0.008 | 0.491 ± 0.034 | 0.925 ± 0.054 | 0.628 ± 0.036 | 13.1 / 22 |
| BDI | 0.963 ± 0.000 | 0.941 ± 0.000 | 0.508 ± 0.013 | 0.973 ± 0.000 | 0.658 ± 0.000 | 5.9 / 22 |
| ICT | 0.915 ± 0.024 | 0.947 ± 0.009 | 0.494 ± 0.026 | 0.897 ± 0.050 | 0.659 ± 0.024 | 9.4 / 22 |
| Tri-Mentoring | 0.891 ± 0.011 | 0.947 ± 0.005 | 0.503 ± 0.013 | 0.956 ± 0.000 | 0.662 ± 0.012 | 7.7 / 22 |
| PGS | 0.715 ± 0.046 | 0.954 ± 0.022 | 0.444 ± 0.020 | 0.889 ± 0.061 | 0.634 ± 0.040 | 13.2 / 22 |
| FGM | 0.923 ± 0.023 | 0.944 ± 0.014 | 0.481 ± 0.024 | 0.811 ± 0.079 | 0.611 ± 0.008 | 13.2 / 22 |
| Match-OPT | 0.933 ± 0.016 | 0.952 ± 0.008 | 0.504 ± 0.021 | 0.824 ± 0.067 | 0.655 ± 0.050 | 8.0 / 22 |
| RaM-RankCosine (Ours) | 0.940 ± 0.028 | 0.951 ± 0.017 | 0.514 ± 0.026 | 0.982 ± 0.012 | 0.675 ± 0.049 | 2.7 / 22 |
| RaM-ListNet (Ours) | 0.949 ± 0.025 | 0.962 ± 0.015 | 0.517 ± 0.029 | 0.981 ± 0.012 | 0.670 ± 0.035 | 2.2 / 22 |

Influence of different ranking loss.
We compare RaM with various ranking losses: SigmoidCrossEntropy (SCE), BinaryCrossEntropy (BCE), and MSE (^3^33Note that MSE is a regression loss, thus it can be viewed as a pointwise ranking loss.) for pointwise loss; RankNet (Burges et al., ), LambdaRank (Burges et al., ; Wang et al., ), and RankCosine (Qin et al., ) for pairwise loss; Softmax (Cao et al., ; Bruch et al., ), ListNet (Cao et al., ), ListMLE (Xia et al., ), and ApproxNDCG (Qin et al., ; Bruch et al., ) for listwise loss. The results in Table [6](#A5.T6) in Appendix [E.2](#A5.SS2) show that ListNet is the best-performing loss with an average rank of 2.0 over 10 losses, and RankCosine is the runner-up with an average rank of 3.2.

Ablation of main modules.
To better validate the effectiveness of the two moduels, data augmentation and output adaptation, of our method, we perform ablation studies based on the top-performing loss functions shown in Table [6](#A5.T6): MSE for pointwise loss, RankCosine for pairwise loss, and ListNet for listwise loss. The results in Table [7](#A5.T7) in Appendix [E.3](#A5.SS3) show that for each considered loss, RaM with data augmentation performs better than the naïve approach which treats a batch of the dataset as a list to rank. The results in Table [8](#A5.T8) show the benefit of using output adaptation. We also examine the influence of the list length $m$, which are illustrated in Appendix [E.4](#A5.SS4).

Versatility of ranking loss.
We examine whether simply replacing the MSE loss of some regression-based methods with a ranking loss can even bring improvement.
Specifically, we substitute MSE with the best-performing ranking loss, ListNet, and incorporate output adaptation. The results in Table [3](#S4.T3) show that the gains are always positive except two cases, cleary demonstrating the versatility of ranking loss. Details regarding method selection and implementation are provided in Appendix [D.5](#A4.SS5).

**Table 3: 100th percentile normalized score of different methods combined with the MSE or ListNet loss in Design-Bench, where positive and negative gain rates are Blue and Red.**
| Method | Type | Ant | D’Kitty | Superconductor | TF-Bind-8 | TF-Bind-10 |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Score | Gain | Score | Gain | Score | Gain | Score | Gain | Score | Gain |  |  |
| BO-$q$EI | MSE | 0.812 ± 0.000 |  | 0.896 ± 0.000 |  | 0.382 ± 0.013 |  | 0.802 ± 0.081 |  | 0.628 ± 0.036 |  |
| ListNet | 0.812 ± 0.000 | +0.0% | 0.896 ± 0.000 | +0.0% | 0.509 ± 0.013 | +33.2% | 0.912 ± 0.032 | +13.7% | 0.653 ± 0.056 | +4.0% |  |
| CMA-ES | MSE | 1.712 ± 0.705 |  | 0.722 ± 0.001 |  | 0.463 ± 0.042 |  | 0.944 ± 0.017 |  | 0.641 ± 0.036 |  |
| ListNet | 1.923 ± 0.773 | +12.3% | 0.723 ± 0.002 | +0.1% | 0.486 ± 0.020 | +5.0% | 0.960 ± 0.008 | +1.7% | 0.661 ± 0.044 | +3.1% |  |
| REINFORCE | MSE | 0.248 ± 0.039 |  | 0.344 ± 0.091 |  | 0.478 ± 0.017 |  | 0.935 ± 0.049 |  | 0.673 ± 0.074 |  |
| ListNet | 0.318 ± 0.056 | +28.2% | 0.359 ± 0.139 | +4.3% | 0.501 ± 0.013 | +4.8% | 0.935 ± 0.049 | +0.0% | 0.673 ± 0.074 | +0.0% |  |
| Grad. Ascent | MSE | 0.273 ± 0.022 |  | 0.853 ± 0.017 |  | 0.510 ± 0.028 |  | 0.969 ± 0.020 |  | 0.646 ± 0.037 |  |
| ListNet | 0.280 ± 0.021 | +2.6% | 0.890 ± 0.019 | +4.3% | 0.521 ± 0.012 | +2.0% | 0.985 ± 0.011 | +1.7% | 0.660 ± 0.049 | +2.2% |  |
| CbAS | MSE | 0.846 ± 0.030 |  | 0.896 ± 0.009 |  | 0.421 ± 0.046 |  | 0.921 ± 0.046 |  | 0.630 ± 0.039 |  |
| ListNet | 0.854 ± 0.037 | +0.9% | 0.898 ± 0.009 | +0.2% | 0.425 ± 0.036 | +1.0% | 0.956 ± 0.033 | +3.8% | 0.642 ± 0.034 | +1.9% |  |
| MINs | MSE | 0.906 ± 0.024 |  | 0.939 ± 0.007 |  | 0.464 ± 0.023 |  | 0.910 ± 0.051 |  | 0.633 ± 0.032 |  |
| ListNet | 0.911 ± 0.025 | +0.5% | 0.941 ± 0.009 | +0.2% | 0.477 ± 0.019 | +2.8% | 0.910 ± 0.029 | +0.0% | 0.638 ± 0.037 | +0.8% |  |
| Tri-Mentoring | MSE | 0.891 ± 0.011 |  | 0.947 ± 0.005 |  | 0.503 ± 0.013 |  | 0.956 ± 0.000 |  | 0.662 ± 0.012 |  |
| ListNet | 0.915 ± 0.024 | +2.7% | 0.943 ± 0.004 | -0.4% | 0.503 ± 0.010 | +0.0% | 0.971 ± 0.005 | +1.7% | 0.710 ± 0.020 | +7.3% |  |
| PGS | MSE | 0.715 ± 0.046 |  | 0.954 ± 0.022 |  | 0.444 ± 0.020 |  | 0.889 ± 0.061 |  | 0.634 ± 0.040 |  |
| ListNet | 0.723 ± 0.032 | +1.1% | 0.962 ± 0.018 | +0.8% | 0.452 ± 0.042 | +1.8% | 0.886 ± 0.003 | -0.3% | 0.643 ± 0.030 | +1.4% |  |

Results on OOD-MSE and OOD-AUPRC.
We also present the OOD-MSE and OOD-AUPRC values of some methods in Appendix [E.5](#A5.SS5), where RaM performs well in OOD-AUPRC while poor in OOD-MSE, further demonstrating that ranking loss is more suitable than MSE for offline MBO.

## 5 Conclusion

Offline MBO methods often learn a surrogate model by minimizing MSE. In this paper, we question this practice. We empirically show that MSE has a low correlation with the final performance of the surrogate model. Instead, we show that the ranking-related metric AUPRC is well-aligned with the primary goal of offline MBO, and propose a ranking-based model for offline MBO. Extensive experimental results show the superiority of our proposed ranking-based model over a large variety of state-of-the-art offline MBO methods. We hope this work can open a new line of offline MBO.

## Appendix A Related Work: Offline Model-Based Optimization

Offline MBO methods (Trabucco et al., ; Xue et al., ) can be generally categorized into two types of approaches.
The mainstream approach for offline MBO is the forward approach, which first trains a forward surrogate model $\hat{f}_{\bm{\theta}}:\mathcal{X}\to\mathbb{R}$ and then employs gradient ascent to optimize the learned surrogate to output candidate solutions, as introduced in Section [2](#S2).
A crucial challenge of this approach is how to improve the surrogate model’s generalization ability in the OOD regions, which can significantly affect the performance.
Prior works of forward approach mainly add regularization items to:
1) regulate the nature of the surrogate model:
NEMO (Fu & Levine, ) optimizes the gap between the surrogate model and the ground-truth function via normalized maximum likelihood, while
RoMA (Yu et al., ) enhances the smoothness of the model in a pre-trained and adaptation manner, and
BOSS (Dao et al., ) directly regulates the sensitivity of the surrogate model;
2) regulate surrogate model’s predictions directly:
COMs (Trabucco et al., ) penalize identified outliers via a GAN-like procedure (Goodfellow et al., ), whereas
IOM (Qi et al., ) maintains representation invariance between the training dataset and design candidate.
Given that an ensemble of surrogate models can bring an improvement (Trabucco et al., ), ICT (Yuan et al., ) and Tri-Mentoring (Chen et al., ) train three symmetric surrogate models and ensemble them, where ICT uses a semi-supervised learning via pseudo-label procedure (Verma et al., ) and Tri-Mentoring employs a strategy similar to Tri-training (Zhou & Li, ) from a pairwise perspective.
Recent works also consider uncovering the structural information of the dataset for better learning.
BDI (Chen et al., ) utilizes both forward and backward mappings to distill knowledge from the offline dataset to the design.
FGM (Grudzien et al., ) considers a novel modeling, which splits the design space into cliques on dimension-level, to approximate scores.
Both PGS (Chemingui et al., ) and Match-OPT (Hoang et al., ) construct trajectories from the dataset, while PGS uses offline reinforcement learning to learn a policy that predicts the search step size of the gradient ascent optimizer and Match-OPT enforces the model to match the ground-truth gradient.
Recent works also consider editing the final designs directly, for example, DEMO (Yuan et al., ) edit the designs obtained by gradient ascent via a diffusion prior.
However, all prior works in forward approach train the surrogate model based on regression model using MSE.
In this work, we train the surrogate model in a ranking suite, obtaining superior performance as shown in Section [4](#S4).

Another type of approach for offline MBO is the backward approach, which typically involves training a conditioned generative model $p_{\bm{\theta}}(\mathbf{x}|y)$ and sampling from it conditioned on a high score, for example, MINs (Kumar & Levine, ) trains an inverse mapping using a conditioned GAN-like model (Goodfellow et al., ; Mirza & Osindero, ), while
CbAS (Brookes et al., ; Fannjiang & Listgarten, ) models it as a zero-sum game via a VAE (Kingma & Welling, ).
Note that generative models show powerful expressiveness and have achieved huge success.
DDOM (Krishnamoorthy et al., ) directly parameterizes the inverse mapping with a conditional diffusion model (Ho et al., ) in the design space.
LEO (Yu et al., ) constructs a latent space through an energy-based model that does not require MCMC sampling.
Recent works in this category also focus on generating designs via constructed trajectories. For example, BONET (Krishnamoorthy et al., ) uses trajectories to mimic a black-box optimizer, thus to train an autoregressive model and sample designs using a heuristic;
GTG (Yun et al., ) considers improving the quality of trajectories via local search, and then directly generate trajectories using a context conditioning diffusion model.

## Appendix B Prevalent Settings of ϕ italic-ϕ \phi , N ​ ( ϕ ) 𝑁 italic-ϕ N(\phi) , and C 𝒜 ​ ( ϕ ) subscript 𝐶 𝒜 italic-ϕ C_{\mathcal{A}}(\phi) in Theorem 2

In this section, we introduce some settings of $\phi$, $N(\phi)$, and $C_{\mathcal{A}}(\phi)$ in Theorem [2](#Thmtheorem2), as shown in Lan et al. ().

In Theorem [2](#Thmtheorem2), $\phi$ is an increasing and strictly positive transformation function, which maps the output of the surrogate model or the score to a positive real number.
Recall that $B$ represents the upper bound of the weight norm $\|\mathbf{w}\|$ of the linear function class $\mathcal{F}=\{\mathbf{x}\to\mathbf{w}^{\top}\mathbf{x}\mid\|\mathbf{w}\|\leq B\}$ where the ranking model $f$ to be learned is from, and $M$ is the upper bound of the norm of designs $\|\mathbf{x}\|$ in design space $\mathcal{X}$.
It is usually represented as a:

- •
Linear function: $\phi_{L}(z)=az+b,\,z\in[-BM,BM]$, where $a>0$ and $b>aBM$;
- •
Exponential function: $\phi_{E}(z)=\exp(az),\,z\in[-BM,BM]$, where $a>0$;
- •
Sigmoid function: $\phi_{S}(z)=\frac{1}{1+\exp(-az)},\,z\in[-BM,BM]$, where $a>0$.

Following Lan et al. (), we introduce some settings based on the above definition of $\phi$ in Table [4](#A2.T4).
For detailed derivation for $C_{\mathcal{A}}(\phi)$, please refer to Lan et al. ().

**Table 4: $N(\phi)$ and $C_{\mathcal{A}}(\phi)$ for LTR algorithms $\mathcal{A}$ (e.g., RankCosine (Qin et al., ) or ListNet (Cao et al., )) on different definitions of $\phi$.**
| $\phi$ | $N(\phi)$ | $C_{RankCosine}(\phi)$ | $C_{ListNet}(\phi)$ |
| --- | --- | --- | --- |
| $\phi_{L}(z)=az+b$ | $a$ | $\frac{\sqrt{m}}{2(b-aBM)}$ | $\frac{2m!}{(b-aBM)(\log{m}+\log{\frac{b+aBM}{b-aBM}})}$ |
| $\phi_{E}(z)=\exp(az)$ | $a\exp(aBM)$ | $\frac{\sqrt{m}\exp(aBM)}{2}$ | $\frac{2m!\exp(aBM)}{\log{m}+2aBM}$ |
| $\phi_{S}(z)=\frac{1}{1+\exp(-az)}$ | $\frac{a(1+\exp(aBM))}{(1+\exp(-aBM))^{2}}$ | $\frac{\sqrt{m}(1+\exp(aBM))}{2}$ | $\frac{2m!(1+\exp(aBM))}{\log{m}+aBM}$ |

## Appendix C Details of Different Ranking Losses

In this section, we introduce details of the different ranking losses in this paper, including traditional and recently prevalent losses.
We study different types of ranking losses in this paper, including pointwise (Crammer & Singer, ), pairwise (Köppel et al., ), and listwise losses (Xia et al., ).
Formally, given a list $\mathbf{X}$ of designs and the list $\mathbf{y}$ of their corresponding scores, let $\hat{f}_{\bm{\theta}}(\mathbf{X})=[\hat{f}_{\bm{\theta}}(\mathbf{x}_{1}),\hat{f}_{\bm{\theta}}(\mathbf{x}_{2}),\ldots,\hat{f}_{\bm{\theta}}(\mathbf{x}_{m})]^{\top}$ be the predicted scores.

For the pointwise losses, we consider:

- •
SigmoidCrossEntropy (SCE): a widely used pointwise loss: $l(\mathbf{y},\hat{f}_{\bm{\theta}}(\mathbf{X}))=\sum_{i=1}^{m}\left(-y_{i}\hat{f}_{\bm{\theta}}(\mathbf{x}_{i})+\log(1+\exp(\hat{f}_{\bm{\theta}}(\mathbf{x}_{i})))\right)$.
- •
BinaryCrossEntropy (BCE): a common pointwise loss considering in a binary classification manner, and we consider its variant with a logits input: $l(\mathbf{y},\hat{f}_{\bm{\theta}}(\mathbf{X}))=-\sum_{i=1}^{m}\left[y_{i}\cdot\log(\sigma(\hat{f}_{\bm{\theta}}(\mathbf{x_{i}})))+(1-y_{i})\cdot\log(1-\sigma(\hat{f}_{\bm{\theta}}(\mathbf{x}_{i})))\right]$, where $\sigma(\cdot)$ is the sigmoid function.
- •
Mean Square Error (MSE): a popular pointwise loss aiming to fit the target values: $l(\mathbf{y},\hat{f}_{\bm{\theta}}(\mathbf{X}))=\sum_{i=1}^{m}(y_{i}-\hat{f}_{\bm{\theta}}(\mathbf{x}_{i}))^{2}$.
Note that the difference of RaM combined with MSE from the regression-based model mainly reflects in the different modeling of the training data.

For the pairwise losses, we consider:

- •
RankNet (Burges et al., ): a popular pairwise loss: $l(\mathbf{y},\hat{f}_{\bm{\theta}}(\mathbf{X}))=\sum_{y_{i}>y_{j}}\log\left(1+\exp(\hat{f}_{\bm{\theta}}(\mathbf{x}_{i})-\hat{f}_{\bm{\theta}}(\mathbf{x}_{j}))\right)$.
- •
LambdaRank (Burges et al., ; Wang et al., ): a pairwise loss with $\Delta$NDCG weight: $l(\mathbf{y},\hat{f}_{\bm{\theta}}(\mathbf{X}))=\sum_{y_{i}>y_{j}}\Delta NDCG(i,j)\log_{2}\left(1+\exp(-\alpha(\hat{f}_{\bm{\theta}}(\mathbf{x}_{i})-\hat{f}_{\bm{\theta}}(\mathbf{x}_{j})))\right)$, where $\alpha$ is a smooth parameter and $\Delta$NDCG is the absolute difference between the values of the Normalized Discounted Cumulative Gain (NDCG), a widely used metric in LTR (Järvelin & Kekäläinen, ; ), when the surrogate model swap the predictions of the two designs, $\mathbf{x}_{i}$ and $\mathbf{x}_{j}$, and thus swap their positions in the ranked list.
- •
RankCosine (Qin et al., ): a classical pairwise loss based on cosine similarity: $l(\mathbf{y},\hat{f}_{\bm{\theta}}(\mathbf{X}))=1-\mathbf{y}\cdot\hat{f}_{\bm{\theta}}(\mathbf{X})/(\|\mathbf{y}\|\cdot\|\hat{f}_{\bm{\theta}}(\mathbf{X})\|)$.

For the pairwise losses, we consider:

- •
Softmax (Cao et al., ; Bruch et al., ): a popular listwise loss: $l(\mathbf{y},\hat{f}_{\bm{\theta}}(\mathbf{X}))=-\sum_{i=1}^{m}y_{i}\log\frac{\exp(\hat{f}_{\bm{\theta}}(\mathbf{x}_{i}))}{\sum_{j=1}^{m}\exp(\hat{f}_{\bm{\theta}}(\mathbf{x}_{i}))}$.
- •
ListNet (Cao et al., ): a classical listwise loss minimizing the cross-entropy between the predicted ranking distribution and the true ranking distribution: $l(\mathbf{y},\hat{f}_{\bm{\theta}}(\mathbf{X}))=-\sum_{j=1}^{m}\frac{\exp(y_{j})}{\sum_{i=1}^{m}\exp(y_{i})}\log\frac{\exp(\hat{f}_{\bm{\theta}}(\mathbf{x}_{j}))}{\sum_{i=1}^{m}\exp(\hat{f}_{\bm{\theta}}(\mathbf{x}_{i}))}$.
- •
ListMLE (Xia et al., ): a widely used listwise loss based on the Plackett-Luce model (Marden, ): $l(\mathbf{y},\hat{f}_{\bm{\theta}}(\mathbf{X}))=-\sum_{i=1}^{m}\log\frac{\exp(\hat{f}_{\bm{\theta}}(\mathbf{x}_{\pi(i)}))}{\sum_{j=i}^{m}\exp(\hat{f}_{\bm{\theta}}(\mathbf{x}_{\pi(j)}))}$, where $\pi$ is the permutation derived from the true ranking labels $y$, $\mathbf{x}_{\pi(i)}$ represents the item at the $i$-th position in the true ranking.
- •
ApproxNDCG (Qin et al., ; Bruch et al., ): a listwise that is a differentiable approximation of NDCG: $l(\mathbf{y},\hat{f}_{\bm{\theta}}(\mathbf{X}))=-\frac{1}{DCG(\pi^{*},\mathbf{y})}\sum_{i,r=1}^{m}\frac{2^{y_{i}}-1}{\log_{2}(1+\pi_{\hat{f}_{\bm{\theta}}}(i))}$, where $\pi^{*}$ is the optimal permutation that ranks items by $\mathbf{y}$, $DCG(\pi^{*},\mathbf{y})$ represents the Discounted Cumulative Gain (DCG; Järvelin & Kekäläinen, ; ) of the ideal ranking given $\mathbf{y}$, and $\pi_{\hat{f}_{\bm{\theta}}}(i)=\frac{1}{2}+\sum_{j}\operatorname{Sigmoid}(\frac{\hat{f}_{\bm{\theta}}(\mathbf{x}_{i})-\hat{f}_{\bm{\theta}}(\mathbf{x}_{j})}{T})$ with $T$ a smooth parameter.

We excluded NeuralNDCG (Pobrotyn & Bialobrzeski, ), a recently proposed listwise loss using neural sort techniques to approximate NDCG, due to its high memory requirements.

## Appendix D Detailed Experimental Settings

### D.1 Detailed Experimental Setting of Figure 2

In this experiment, we select five surrogate models: a gradient-ascent baseline and four state-of-the-art approaches, COMs (Trabucco et al., ), IOM (Qi et al., ), ICT (Yuan et al., ), and Tri-Mentoring (Chen et al., ).
These models are chosen due to their common characteristic of employing standard gradient-ascent to obtain the final design.
While BDI (Chen et al., ) and Match-OPT (Hoang et al., ) also utilize gradient-ascent for design generation, we exclude BDI for its intractable model, which is built with JAX (Bradbury et al., ), and Match-OPT for its time-intensive training procedure.

We follow the default setting as in (Chen et al., ; Yuan et al., ) to prepare training data and set the hyper-parameters in Equation [1](#S2.E1) to search inside the model.
For discrete tasks, in order to map the design space to a continuous one, we transform the discrete designs into real-valued logits of a categorical distribution, which is provided in (Trabucco et al., ; ).
We use z-score method to normalize both the designs and scores for a better training.
After the model is trained, we use Adam optimizer (Kingma & Ba, ) to conduct gradient ascent. For discrete tasks, we set $\eta=1e-1$ and $T=100$, and for continuous tasks, we set $\eta=1e-3$ and $T=200$.

Following Chen et al. (), we construct an OOD dataset by selecting the high-scoring designs that are excluded for the training data in Design-Bench (Trabucco et al., ).
In Design-Bench, the training dataset is selected as the bottom performing $x\%$ in the entire collected dataset, (i.e., $x=40,50,60$).
Note that the open-source repository(^4^44[https://github.com/brandontrabucco/design-bench](https://github.com/brandontrabucco/design-bench)) provides an API to access the entire dataset.
We identify the excluded $(100-x)\%$ high-scoring data to comprise the OOD dataset for analysis, except for TF-Bind-10 (Barrera et al., ) task, whose excluded $(100-x)\%$ high-scoring data contains 4161482 samples and is too large for AUPRC evaluation.
Thus, we randomly sample 30000 samples from the $(100-x)\%$ data to construct the OOD dataset for TF-Bind-10 task.

### D.2 Excluded Design-Bench Tasks

Following prior works (Krishnamoorthy et al., ; ; Yun et al., ; Yu et al., ), we exclude three tasks in Design-Bench (Trabucco et al., ) for evaluation, including Hopper (Brockman et al., ), ChEMBL (Gaulton et al., ), and synthetic NAS tasks on CIFAR10 (Hinton et al., ).
As noted in prior works and this [link](https://github.com/brandontrabucco/design-bench/issues/8), this is a bug for the implementation of Hopper in Design-Bench.
For the ChEMBL task, we exclude it because almost all methods produce the same results, as shown in (Krishnamoorthy et al., ; ), which is not suitable for comparison.
We also exclude NAS due to its high computation cost for exact evaluation over multiple seeds, which is beyond our budget.

### D.3 Excluded Offline MBO Algorithms

We exclude NEMO (Fu & Levine, ) since there is no open-source implementation.
We also exclude concurrent works, DEMO (Yuan et al., ) and LEO (Yu et al., ), since they are not yet peer-reviewed and lack an open-source implementation at the time of our initial submission.
For BOSS (Dao et al., ), we exclude it since it is a general trick that can be applied to any regression-based forward method, instead of a single proposed methods.

### D.4 Detailed Experimental Setting of Main Results in Table 2

We set the size $n$ of training dataset to $10000$, and the list length $m=1000$.
To make a fair comparison to regression-based methods, following Trabucco et al. (; ); Chen et al. (); Yuan et al. (), we model the surrogate model $\hat{f}_{\bm{\theta}}$ as a simple multilayer perceptron (MLP) with two hidden layers of size 2048 using PyTorch (Paszke et al., ).
We use ReLU as activation functions. RankCosine (Qin et al., ) and ListNet (Cao et al., ) is used as two main loss functions in our experiments.
Our implementation of different loss functions is either inherited from (Pobrotyn et al., )(^5^55[https://github.com/allegro/allRank](https://github.com/allegro/allRank)) or implemented by ourselves.

We split the dataset into a training set and a validation set of the ratio $8:2$.
The model is trained for $N_{0}=200$ epochs and is optimized using Adam (Kingma & Ba, ) with a learning rate of $3\times 10^{-4}$ and a weight decay coefficient of $1\times 10^{-5}$, and the model with minimal validation loss among $N_{0}$ epochs serves as the final model.

After the model is trained, we fix the model parameters and normalize the output values, then following (Chen et al., ; Yuan et al., ), we set $\eta=1\times 10^{-3}$ and $T=200$ for continuous tasks, and $\eta=1\times 10^{-1}$ and $T=100$ for discrete tasks to search for the final design.

For baselines methods and CbAS (Brookes et al., ), MINs (Kumar & Levine, ) in Table [2](#S4.T2), we use the open-source baselines implementations from the source code of Design-Bench(^6^66[https://github.com/brandontrabucco/design-baselines](https://github.com/brandontrabucco/design-baselines)).
For other offline MBO methods, we use the open-source implementation provided in their papers and use their hyper-parameter settings, except for DDOM (Krishnamoorthy et al., ) and BONET (Krishnamoorthy et al., ), where we modify the evaluation budget $k$ from $256$ to $128$ following the protocol of other works.
A brief review of offline MBO methods is also provided in Appendix [A](#A1).

### D.5 Detailed Experimental Setting of Table 3

In this experiment, for a fair comparison of MSE and ListNet, we do not adopt the data augmentation method, instead, we use the naïve approach introduced in Section [3.3](#S3.SS3), viewing a batch of designs as a list to be ranked.

We choose baselines methods that optimize a trained model, BO-$q$EI (Garnett, ), CMA-ES (Hansen, ), REINFORCE (Williams, ), and Gradient Ascent, two backward approach provided in Trabucco et al. (), CbAS (Brookes et al., ) and MINs (Kumar & Levine, ), and two state-of-the-art forward methods that can replace MSE with ListNet, Tri-Mentoring (Chen et al., ) and PGS (Chemingui et al., ).
Note that the model trained with ranking loss has different prediction scales as regression-based models, as discussed in [3.3](#S3.SS3).
We exclude many forward methods due to the inapplicability of directly replacing MSE with ListNet.
For example, COMs (Trabucco et al., ), RoMA (Yu et al., ), IOM (Qi et al., ) use the prediction values to calculate the loss function, while BDI (Chen et al., ) and ICT (Yuan et al., ) assign weight to each sample, thus MSE in these methods cannot be directly replaced with a ranking loss like ListNet.

In order to adapt the same parameters of the online optimizers (e.g., BO-$q$EI, Gradient Ascent) that optimize the trained model for a fair comparison, we also perform an output adaptation for ranking-based model after it is trained.

All the replacements are conducted fixing their open-source codes by replacing MSE with ListNet when training the forward model.

## Appendix E Additional Experiments

In this section, we provide additional experimental results mentioned in Section [4](#S4).

### E.1 50th Percentile Results on Design-Bench

Following the evaluation protocol in Trabucco et al. (), to validate the robustness of our proposed method, we also provide the detailed results of 50th percentile results in Table [5](#A5.T5).

**Table 5: 50th percentile normalized score in Design-Bench, where the best and runner-up results on each task are Blue and Violet. $\mathcal{D}$(best) denotes the best score in the offline dataset.**
| Method | Ant | D’Kitty | Superconductor | TF-Bind-8 | TF-Bind-10 | Mean Rank |
| --- | --- | --- | --- | --- | --- | --- |
| $\mathcal{D}$(best) | 0.565 | 0.884 | 0.400 | 0.439 | 0.467 | / |
| BO-$q$EI | 0.568 ± 0.000 | 0.883 ± 0.000 | 0.311 ± 0.019 | 0.439 ± 0.000 | 0.467 ± 0.000 | 12.6 / 22 |
| CMA-ES | -0.041 ± 0.004 | 0.684 ± 0.017 | 0.377 ± 0.009 | 0.539 ± 0.017 | 0.482 ± 0.009 | 12.6 / 22 |
| REINFORCE | 0.124 ± 0.042 | 0.460 ± 0.209 | 0.457 ± 0.020 | 0.466 ± 0.023 | 0.464 ± 0.009 | 15.7 / 22 |
| Grad. Ascent | 0.136 ± 0.016 | 0.581 ± 0.128 | 0.471 ± 0.017 | 0.582 ± 0.027 | 0.470 ± 0.004 | 11.2 / 22 |
| Grad. Ascent Mean | 0.185 ± 0.012 | 0.718 ± 0.037 | 0.481 ± 0.023 | 0.630 ± 0.033 | 0.470 ± 0.005 | 9.2 / 22 |
| Grad. Ascent Min | 0.187 ± 0.012 | 0.714 ± 0.040 | 0.480 ± 0.022 | 0.628 ± 0.025 | 0.470 ± 0.004 | 9.6 / 22 |
| CbAS | 0.385 ± 0.027 | 0.740 ± 0.023 | 0.121 ± 0.014 | 0.422 ± 0.022 | 0.457 ± 0.006 | 18.4 / 22 |
| MINs | 0.640 ± 0.029 | 0.886 ± 0.006 | 0.332 ± 0.014 | 0.407 ± 0.014 | 0.465 ± 0.006 | 12.0 / 22 |
| DDOM | 0.598 ± 0.030 | 0.829 ± 0.050 | 0.313 ± 0.017 | 0.416 ± 0.023 | 0.464 ± 0.006 | 14.4 / 22 |
| BONET | 0.795 ± 0.039 | 0.906 ± 0.008 | 0.334 ± 0.032 | 0.476 ± 0.149 | 0.452 ± 0.050 | 10.6 / 22 |
| GTG | 0.593 ± 0.022 | 0.889 ± 0.002 | 0.350 ± 0.023 | 0.542 ± 0.038 | 0.458 ± 0.008 | 9.4 / 22 |
| COMs | 0.532 ± 0.020 | 0.882 ± 0.002 | 0.376 ± 0.067 | 0.513 ± 0.019 | 0.474 ± 0.014 | 9.3 / 22 |
| RoMA | 0.193 ± 0.017 | 0.344 ± 0.097 | 0.368 ± 0.012 | 0.520 ± 0.074 | 0.516 ± 0.004 | 12.0 / 22 |
| IOM | 0.459 ± 0.024 | 0.829 ± 0.022 | 0.291 ± 0.059 | 0.490 ± 0.055 | 0.467 ± 0.000 | 14.9 / 22 |
| BDI | 0.569 ± 0.000 | 0.876 ± 0.000 | 0.389 ± 0.022 | 0.595 ± 0.000 | 0.429 ± 0.000 | 9.4 / 22 |
| ICT | 0.550 ± 0.028 | 0.875 ± 0.006 | 0.333 ± 0.018 | 0.547 ± 0.041 | 0.499 ± 0.012 | 9.2 / 22 |
| Tri-Mentoring | 0.548 ± 0.013 | 0.870 ± 0.002 | 0.363 ± 0.019 | 0.619 ± 0.009 | 0.491 ± 0.001 | 8.0 / 22 |
| PGS | 0.190 ± 0.030 | 0.885 ± 0.001 | 0.233 ± 0.033 | 0.503 ± 0.041 | 0.386 ± 0.177 | 16.0 / 22 |
| FGM | 0.532 ± 0.039 | 0.871 ± 0.017 | 0.353 ± 0.058 | 0.540 ± 0.117 | 0.466 ± 0.004 | 12.1 / 22 |
| Match-OPT | 0.587 ± 0.008 | 0.887 ± 0.001 | 0.381 ± 0.038 | 0.435 ± 0.017 | 0.471 ± 0.013 | 8.0 / 22 |
| RaM-RankCosine (Ours) | 0.566 ± 0.012 | 0.881 ± 0.003 | 0.356 ± 0.013 | 0.544 ± 0.043 | 0.462 ± 0.006 | 11.0 / 22 |
| RaM-ListNet (Ours) | 0.579 ± 0.014 | 0.888 ± 0.003 | 0.359 ± 0.013 | 0.552 ± 0.032 | 0.467 ± 0.009 | 7.4 / 22 |

In Table [5](#A5.T5), we can observe although RaM combined with RankCosine performs not so well on $50^{th}$ percentile results, RaM combined with ListNet, which is the best methods in our main experimental results (Table [2](#S4.T2)), also obtains a best average rank of 7.4 among 22 methods.

### E.2 Results of Different Ranking Losses

We compare a wide range of ranking losses that combined with RaM in the context of offline MBO, including three types of pointwise, pairwise, and listwise losses.
Details of these ranking losses are provided in Appendix [C](#A3), and experimental results of $100^{th}$ percentile normalized score in Design-Bench are provided in Table [6](#A5.T6).

**Table 6: 100th percentile normalized score of RaM combined with different ranking losses in Design-Bench. The best and runner-up results on each task are Blue and Violet. $\mathcal{D}$(best) denotes the best score in the offline dataset.**
| Type | Method | Ant | D’Kitty | Superconductor | TF-Bind-8 | TF-Bind-10 | Mean Rank |
| --- | --- | --- | --- | --- | --- | --- | --- |
| / | $\mathcal{D}$(best) | 0.565 | 0.884 | 0.400 | 0.439 | 0.467 | / |
| Pointwise | RaM-SCE | 0.928 ± 0.012 | 0.953 ± 0.012 | 0.502 ± 0.013 | 0.820 ± 0.065 | 0.662 ± 0.026 | 6.9 / 10 |
| RaM-BCE | 0.925 ± 0.014 | 0.950 ± 0.009 | 0.501 ± 0.012 | 0.825 ± 0.065 | 0.656 ± 0.021 | 8.3 / 10 |  |
| RaM-MSE | 0.933 ± 0.032 | 0.957 ± 0.013 | 0.507 ± 0.028 | 0.962 ± 0.031 | 0.674 ± 0.044 | 3.9 / 10 |  |
| Pairwise | RaM-RankNet | 0.921 ± 0.033 | 0.955 ± 0.008 | 0.510 ± 0.032 | 0.962 ± 0.030 | 0.676 ± 0.037 | 4.4 / 10 |
| RaM-LambdaRank | 0.918 ± 0.020 | 0.949 ± 0.010 | 0.528 ± 0.020 | 0.962 ± 0.020 | 0.650 ± 0.039 | 6.8 / 10 |  |
| RaM-RankCosine | 0.940 ± 0.028 | 0.951 ± 0.017 | 0.514 ± 0.026 | 0.982 ± 0.012 | 0.675 ± 0.049 | 3.2 / 10 |  |
| Listwise | RaM-Softmax | 0.932 ± 0.014 | 0.954 ± 0.011 | 0.509 ± 0.028 | 0.918 ± 0.039 | 0.489 ± 0.115 | 6.2 / 10 |
| RaM-ListNet | 0.949 ± 0.025 | 0.962 ± 0.015 | 0.517 ± 0.029 | 0.981 ± 0.012 | 0.670 ± 0.035 | 2.0 / 10 |  |
| RaM-ListMLE | 0.930 ± 0.032 | 0.953 ± 0.012 | 0.484 ± 0.022 | 0.966 ± 0.020 | 0.656 ± 0.041 | 6.0 / 10 |  |
| RaM-ApproxNDCG | 0.926 ± 0.031 | 0.952 ± 0.004 | 0.507 ± 0.010 | 0.936 ± 0.069 | 0.551 ± 0.058 | 7.3 / 10 |  |

We find that MSE performs the best in all of 3 pointwise losses, RankCosine (Qin et al., ) outperforms other pairwise losses, and ListNet (Cao et al., ) obtains the highest average rank among listwise losses.
Note that prevalent ranking losses such as ApproxNDCG (Bruch et al., ) do not perform well in RaM.
This might due to the simplicity of MLP, which cannot absorb complex information of conveyed by the trending powerful loss functions (Qin et al., ; Pobrotyn et al., ).
However in this work, we parameterize the surrogate model as a simple MLP for a fair comparison to the regression-based methods, and we will consider more complex modeling in our future work.

### E.3 Ablation Studies Results on Main Modules

To better validate the effectiveness of the two moduels, data augmentation and output adaptation, of our method, we perform ablation studies based on the top-performing loss functions shown in Table [6](#A5.T6): MSE for pointwise loss, RankCosine for pairwise loss, and ListNet for listwise loss.
The results in Table [7](#A5.T7) show that for each considered loss, RaM with data augmentation performs better than the naïve approach which treats a batch of the dataset as a list to rank.
The results in Table [8](#A5.T8) show the benefit of using output adaptation.
All of these ablation studies provide strongly positive support to the effectiveness of these two modules.

**Table 7: Ablation studies on data augmentation, considering learning with MSE, RankCosine, and ListNet, which are the best-performing pointwise, pairwise, and listwise loss, respectively, as shown in Table [6](#A5.T6). For each combination of loss and task, the better performance is Bolded. $\mathcal{D}$(best) denotes the best score in the offline dataset.**
| Method | Ant | D’Kitty | Superconductor | TF-Bind-8 | TF-Bind-10 | Mean Rank |
| --- | --- | --- | --- | --- | --- | --- |
| $\mathcal{D}$(best) | 0.565 | 0.884 | 0.400 | 0.439 | 0.467 | / |
| RaM-MSE (w/ Aug.) | 0.933 ± 0.032 | 0.957 ± 0.013 | 0.507 ± 0.028 | 0.962 ± 0.031 | 0.674 ± 0.044 | 3.7 / 6 |
| RaM-MSE (w/o Aug.) | 0.928 ± 0.022 | 0.944 ± 0.017 | 0.502 ± 0.015 | 0.983 ± 0.012 | 0.652 ± 0.045 | 4.7 / 6 |
| RaM-RankCosine (w/ Aug.) | 0.940 ± 0.028 | 0.951 ± 0.017 | 0.514 ± 0.026 | 0.982 ± 0.012 | 0.675 ± 0.049 | 2.2 / 6 |
| RaM-RankCosine (w/o Aug.) | 0.929 ± 0.019 | 0.944 ± 0.005 | 0.504 ± 0.018 | 0.980 ± 0.016 | 0.654 ± 0.038 | 4.7 / 6 |
| RaM-ListNet (w/ Aug.) | 0.949 ± 0.025 | 0.962 ± 0.015 | 0.517 ± 0.029 | 0.981 ± 0.012 | 0.670 ± 0.035 | 2.0 / 6 |
| RaM-ListNet (w/o Aug.) | 0.938 ± 0.025 | 0.964 ± 0.011 | 0.507 ± 0.007 | 0.975 ± 0.010 | 0.640 ± 0.037 | 3.7 / 6 |

**Table 8: Ablation studies on output adaptation, considering learning with MSE, RankCosine, and ListNet, which are the best-performing pointwise, pairwise, and listwise loss, respectively, as shown in Table [6](#A5.T6). For each combination of loss and task, the better performance is Bolded. $\mathcal{D}$(best) denotes the best score in the offline dataset.**
| Method | Ant | D’Kitty | Superconductor | TF-Bind-8 | TF-Bind-10 | Mean Rank |
| --- | --- | --- | --- | --- | --- | --- |
| $\mathcal{D}$(best) | 0.565 | 0.884 | 0.400 | 0.439 | 0.467 | / |
| RaM-MSE (w/ Adapt.) | 0.933 ± 0.032 | 0.957 ± 0.013 | 0.507 ± 0.028 | 0.962 ± 0.031 | 0.674 ± 0.044 | 3.8 / 6 |
| RaM-MSE (w/o Adapt.) | 0.913 ± 0.028 | 0.953 ± 0.012 | 0.506 ± 0.024 | 0.966 ± 0.023 | 0.653 ± 0.030 | 5.2 / 6 |
| RaM-RankCosine (w/ Adapt.) | 0.940 ± 0.028 | 0.951 ± 0.017 | 0.514 ± 0.026 | 0.982 ± 0.012 | 0.675 ± 0.049 | 2.7 / 6 |
| RaM-RankCosine (w/o Adapt.) | 0.908 ± 0.023 | 0.955 ± 0.015 | 0.514 ± 0.025 | 0.970 ± 0.016 | 0.649 ± 0.019 | 4.5 / 6 |
| RaM-ListNet (w/ Adapt.) | 0.949 ± 0.025 | 0.962 ± 0.015 | 0.517 ± 0.029 | 0.981 ± 0.012 | 0.670 ± 0.035 | 1.6 / 6 |
| RaM-ListNet (w/o Adapt.) | 0.932 ± 0.034 | 0.961 ± 0.013 | 0.516 ± 0.029 | 0.968 ± 0.016 | 0.655 ± 0.015 | 3.2 / 6 |

### E.4 Ablation of the List Length m 𝑚 m

Note that the list length $m$ in the training data could have a impact on the generalization ability of the model (Lan et al., ; Tewari & Chaudhuri, ) and its impact on OOD generalization ability of LTR algorithms is undiscovered (Chapelle et al., ).
Besides, popular benchmarks in LTR (Qin et al., ; Qin & Liu, ; Chapelle & Chang, ; Dato et al., ) have different settings of the list length, ranging from 5 to 1000.

Hence, to meet the settings of different LTR benchmarks and to better understand the sensitivity of RaM-ListNet with respect to $m$, we conduct a careful ablation study of the setting of list length $m$, with values varying in $\{10,20,50,100,200,500,1000,1500,2000\}$, as shown in Table [9](#A5.T9).

**Table 9: 100th percentile normalized score in Design-Bench of RaM-ListNet with varying values of $m$, where the best and runner-up results on each task are Blue and Violet. $\mathcal{D}$(best) denotes the best score in the offline dataset.**
| $m$ | Ant | D’Kitty | Superconductor | TF-Bind-8 | TF-Bind-10 | Mean Rank |
| --- | --- | --- | --- | --- | --- | --- |
| $\mathcal{D}$(best) | 0.565 | 0.884 | 0.400 | 0.439 | 0.467 | / |
| 10 | 0.916 ± 0.014 | 0.953 ± 0.016 | 0.508 ± 0.017 | 0.984 ± 0.007 | 0.655 ± 0.063 | 6.6 / 9 |
| 20 | 0.913 ± 0.026 | 0.954 ± 0.011 | 0.518 ± 0.022 | 0.972 ± 0.016 | 0.670 ± 0.043 | 6.2 / 9 |
| 50 | 0.930 ± 0.035 | 0.963 ± 0.014 | 0.524 ± 0.021 | 0.978 ± 0.014 | 0.653 ± 0.046 | 3.5 / 9 |
| 100 | 0.922 ± 0.024 | 0.963 ± 0.015 | 0.525 ± 0.020 | 0.967 ± 0.016 | 0.702 ± 0.129 | 3.4 / 9 |
| 200 | 0.927 ± 0.023 | 0.960 ± 0.017 | 0.526 ± 0.020 | 0.977 ± 0.015 | 0.650 ± 0.015 | 4.6 / 9 |
| 500 | 0.920 ± 0.028 | 0.962 ± 0.007 | 0.518 ± 0.028 | 0.975 ± 0.011 | 0.699 ± 0.128 | 4.0 / 9 |
| 1000 | 0.949 ± 0.025 | 0.962 ± 0.015 | 0.517 ± 0.029 | 0.981 ± 0.012 | 0.670 ± 0.035 | 3.4 / 9 |
| 1500 | 0.918 ± 0.030 | 0.961 ± 0.012 | 0.511 ± 0.025 | 0.971 ± 0.019 | 0.691 ± 0.127 | 5.8 / 9 |
| 2000 | 0.905 ± 0.035 | 0.958 ± 0.016 | 0.515 ± 0.027 | 0.967 ± 0.020 | 0.664 ± 0.043 | 7.5 / 9 |

From the results of Table [9](#A5.T9), we observe that RaM-ListNet obtains the best performance when $m=100$ or $m=1000$, and it will undergo a score drop as $m$ gets relatively large, which demonstrates the need for a careful tuning of the list length.

### E.5 Results on OOD Metrics

In this subsection, we also present the OOD-MSE results in Table [10](#A5.T10) and OOD-AUPRC values in Table [11](#A5.T11).
As the results deliver, RaM combined with RankCosine or ListNet perform poor in OOD-MSE, while they rank the best two in OOD-AUPRC.
Coupled with the fact that RaM obtains the best performance in our main results (Table [2](#S4.T2)), results on OOD-MSE and OOD-AUPRC further demonstrate that: 1) an algorithm with better OOD-AUPRC could result in better performance in offline MBO, no matter what its OOD-MSE is; 2) ranking loss is more suitable than MSE for offline MBO, since RaM obtains a better OOD-AUPRC compared to other regression-based methods.

**Table 10: OOD-MSE of different methods in Design-Bench, where the best and runner-up results on each task are Blue and Violet.**
| Method | Ant | D’Kitty | Superconductor | TF-Bind-8 | TF-Bind-10 | Mean Rank |
| --- | --- | --- | --- | --- | --- | --- |
| Grad. Ascent | 9.134 ± 0.821 | 0.444 ± 0.123 | 1.054 ± 0.229 | 5.543 ± 0.263 | 2.930 ± 0.171 | 3.6 / 7 |
| COMs | 9.084 ± 0.514 | 0.303 ± 0.104 | 0.930 ± 0.043 | 5.941 ± 0.201 | 2.541 ± 0.047 | 2.6 / 7 |
| IOM | 9.520 ± 0.948 | 0.299 ± 0.062 | 0.798 ± 0.041 | 8.779 ± 0.364 | 2.594 ± 0.069 | 3.0 / 7 |
| ICT | 160468.083 ± 354.495 | 71634.341 ± 101.262 | 8444.529 ± 60.098 | 0.163 ± 0.037 | 0.352 ± 0.060 | 4.6 / 7 |
| Tri-Mentoring | 160641.446 ± 111.371 | 71557.860 ± 25.979 | 8190.870 ± 1.785 | 0.115 ± 0.000 | 0.381 ± 0.030 | 4.4 / 7 |
| RaM-RankCosine (Ours) | 17.113 ± 0.174 | 1.503 ± 0.092 | 11.170 ± 0.293 | 18.799 ± 1.345 | 2.632 ± 0.129 | 5.2 / 7 |
| RaM-ListNet (Ours) | 25.011 ± 0.824 | 2.689 ± 0.249 | 8.999 ± 2.449 | 11.546 ± 3.295 | 2.408 ± 0.315 | 4.6 / 7 |

**Table 11: OOD-AUPRC of different methods in Design-Bench, where the best and runner-up results on each task are Blue and Violet.**
| Method | Ant | D’Kitty | Superconductor | TF-Bind-8 | TF-Bind-10 | Mean Rank |
| --- | --- | --- | --- | --- | --- | --- |
| Grad. Ascent | 0.363 ± 0.028 | 0.403 ± 0.002 | 0.731 ± 0.006 | 0.670 ± 0.017 | 0.518 ± 0.009 | 4.7 / 7 |
| COMs | 0.744 ± 0.015 | 0.727 ± 0.005 | 0.391 ± 0.028 | 0.433 ± 0.002 | 0.505 ± 0.001 | 4.4 / 7 |
| IOM | 0.649 ± 0.044 | 0.648 ± 0.085 | 0.436 ± 0.057 | 0.428 ± 0.007 | 0.515 ± 0.056 | 4.6 / 7 |
| ICT | 0.443 ± 0.095 | 0.549 ± 0.089 | 0.596 ± 0.100 | 0.658 ± 0.015 | 0.545 ± 0.023 | 4.6 / 7 |
| Tri-Mentoring | 0.346 ± 0.000 | 0.403 ± 0.000 | 0.740 ± 0.003 | 0.690 ± 0.000 | 0.571 ± 0.000 | 3.7 / 7 |
| RaM-RankCosine (Ours) | 0.492 ± 0.013 | 0.437 ± 0.013 | 0.713 ± 0.003 | 0.714 ± 0.004 | 0.551 ± 0.009 | 3.2 / 7 |
| RaM-ListNet (Ours) | 0.474 ± 0.045 | 0.628 ± 0.017 | 0.723 ± 0.004 | 0.709 ± 0.005 | 0.562 ± 0.008 | 2.8 / 7 |

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: - In MBO problems, the exact value of the objective function is often critical. For example, when optimizing GPU performance, the goal is not only to minimize latency but also to know the precise latency value to assess the suitability of the design. Using LTR losses, however, does not guarantee that the output will match the true objective. Therefore, a potential drawback of the proposed approach is its failure to accurately estimate the objective function, which is highly relevant in real-world applications.
- I am a bit puzzled by the definitions of Precision and Recall in Definition 1. In machine learning, Precision typically measures the ratio of true positives to all instances classified as positive, while Recall measures the ratio of true positives to the total number of ground-truth positives.  If a design is included in the top-k set, it should be considered a true positive. This suggests the numerator should be the same for both Precision and Recall, the dimension of the intersection and the denominator for both Precision and Recall should be k.
- The out-of-distribution issue in MBO has been well studied. Adversarial weight perturbation [1] is one approach. Perturbing and minimizing the worst case performance among the performance is crucial to have better out-of-distribution issue and to improve generalization. Do you claim that the proposed approach do not require such perturbations?
- To achieve generalization, methods like regularization, ensembling weak learners, and training under adversarial inputs are commonly used. It’s unclear how using an LTR loss alone could eliminate the need for such approaches. Even when training with an LTR loss on a static offline dataset, how you can guarantee that the ranking will generalize to out-of-distribution data.

[1]: Yu, Tianhe, et al. "Combo: Conservative offline model-based policy optimization." Advances in neural information processing systems 34 (2021): 28954-28967.
soundness: 3
strengths: **Originality**: This paper utilizes the existing LTR loss functions for offline MBO problems. This is a novel way to approach the offline MBO problem.

**Quality**: I found some concerns which I will raise in the weakness section.

**Clarity**: Overall, the paper has clarity.

**Significance**: This new approach to offline MBO could be of significance, if the following questions are answered.
summary: Offline model-based optimization (MBO) is concerned with the goal of maximizing an unknown objective function using an offline dataset. 
$\min_{x \in X} f(x)$ with unknown $f(.)$. The goal is to learn a proxy function $\hat{f} (.)$. 
A dataset from past observations is available $[  (x_i, f(x_i) ]_{i=1}^n$. The training has to be done completely offline, because online evaluation is not possible in this setup.
The rudimentary approach is to train $\hat{f} (.)$ by minimizing the MSE loss between $\hat{f} (x_i)$ and $f (x_i)$. This vanilla approach has the problem of out-of-distribution issue, because $\hat{f} (.)$ does not approximate well on unseen space within $X$.
This paper proposes to minimize a learning-to-rank (LTR) loss instead of the MSE loss and experimentally compare the proposed RankCosine and ListNet losses with the existing approaches.
weaknesses: Although the approach is novel, there are concerns regarding how it can generalize performance and alleviate the out-of-distribution issue. It is not convincing that just plugging an LTR loss would address would solve the problem.
(See Questions)

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 4
questions: Please see above.
soundness: 3
strengths: Paper was clearly written and original. To the best of my knowledge, the LTR framework was not previously used for offline optimization. Empirical results show positive improvement over previous baselines.
summary: This paper proposes a new approach for offline optimization which focuses on training a surrogate to rank samples rather than to accurately predict their values. This surrogate is trained using the LTR framework with ListNet loss.
weaknesses: I think the idea that MSE is not always optimal for offline opt was recently pointed out in the MATCH-OPT paper (Hoang et al., 2024) (in which they use the term "value-matching surrogates" to describe model trained with MSE loss). Personally, I think that MSE is not worse than any other metric for optimization, but it could be harder to be estimated accurately for OOD samples. This seems to have been acknowledged in the MATCH-OPT paper. Section 3.1 and 3.2 of this paper seems to focus on justifying this insight using empirical evidence. 3.3 describes the LTR algorithm (which is not original), and 3.4 quotes some generalization result of LTR.

Regarding 3.1 and 3.2, it makes no sense to compare the two metrics MSE and AUPRC. MSE evaluation does not use info about the top-k design (which has very strong correlation with the rank of the 100th percentile candidate), whereas AUPRC explicitly uses this info. It seems obvious to me that the latter would have higher Pearson correlation. Additionally, one cannot use the top-k info during training, so this would have little relevance to the actual method (which uses the ListNet loss). Most offline opt frameworks don't simply rely on the MSE loss though. As pointed out in the intro, there are some that use generative modeling or regularization to condition the surrogate model. How do those techniques relate to the idea of ranking samples?
 
The theory is, unfortunately, quite shallow and needs some extension. Theorem 1 (which is more of a proposition), states an equivalence, so the reverse argument could also be put forward -- why not MSE instead of ranking? As mentioned, I think the key challenge of offline opt is that we cannot control the surrogate's behavior OOD. Early error can lead the gradient search astray, so one should focus on quantifying this compounding effect.

Theorem 2 is a direct quote from previous work, but it seems to add no value to the current setting. Does LTR generalize better than MSE on OOD data? I doubt that we can conclude anything general like that without a particular set of assumptions. So, the right question is, in which scenario will LTR generalize better than MSE on OOD data? I think this is important to answer this, especially when this paper is trying to advocate for a paradigm shift in offline opt. 

Empirical results seem reasonable. However, given the landscape of offline opt research, in which every recent paper seems to report a different set of results on the Design Bench (typically suggesting that it is best), I find it hard to conclude that any method is more significant and robust than the others without proper theoretical backing. The paper should also compare with the recent DDOM method (Krisnamoorthy et al., 2023) which is representative of the generative modeling line of work.

---

confidence: 5
contribution: 3
initial_rating: 8
presentation: 3
questions: 1. As mentioned by the authors in the introduction, one of the key challenges in offline optimization is Out-of-Distribution (OOD) data. While this ranking approach offers a better indication than the traditional OOD-MSE and aligns well with the offline optimization context, how do the authors address the OOD issue in the ranking setting? Could you provide a detailed analysis of the OOD problem in this context?

2. In Definition 1, regarding Recall@k, why does the Recall@k value depend solely on k and N, with no relation to the model's predictions or true ranking? Additionally, this definition does not seem to reflect the model’s recall performance. Should the authors consider revising the definition, or reconsider using the term "recall" to refer to this variable?

3. In Table 2, the results for backward methods like BONET[1] and GTG[2] appear to be much lower than those in the original paper (Table 2 in [1] and Table 1 in [2]). Could this be due to differences in experimental settings? If so, why don't the authors use the same configuration?

[1] Generative pretraining for black-box optimization. ICML 2023. 

[2] Guided trajectory generation with diffusion models for offline model-based optimization. NeurIPS 2024.
soundness: 3
strengths: - Utilizing ranking scores in place of MSE to train the offline optimization surrogate model is an innovative approach. It aligns more intuitively with the objectives of offline optimization, making it a fitting metric for this context.

- The algorithm is very clear and easy to understand. At the same time, this paper conducts a theoretical analysis of generalization error bound.

- The authors conducte a wide range of experiments, providing comprehensive comparisons with the latest methods. The experimental results show the feasibility and effectiveness of the proposed approach.

- This article is well-structured, with clear logic and easy-to-follow explanations that enhance readability and comprehension. 

- The supplementary material is sufficient and the algorithm can be fully reproduced without any problems. The appendices contain ample additional experiments and ablation studies. The appendices also contain a very comprehensive review of related work.
summary: This paper challenges the use of MSE for training surrogate models in offline MBO, demonstrating its weak correlation with final performance. Instead, the authors propose a ranking-based model leveraging AUPRC, which achieves superior results compared to existing methods and suggests promising new directions for offline MBO.
weaknesses: - Some concepts need to be clarified, such as OOD in the ranking model and Recall@k in Definition 1. These are raised in the "questions" below.

- Typos. For example, "e.g" in line 48 should be "e.g.", "caculated" in line 241 should be "calculated". A thorough proofreading is recommended.

Meta-review: The paper considers the problem of offline model based optimization. The key idea is to use learning to rank loss instead of mean squared error loss for training surrogate models. Although this point has already been discussed earlier in MatchOpt, the paper does more thorough and comprehensive analysis. I think the community will benefit from this paper. Therefore, I recommend acceptance.

Decision: Accept (Poster)