AReUReDi: Annealed Rectified Updates for Refining Discrete Flows with Multi-Objective Guidance

## Abstract

Abstract Designing sequences that satisfy multiple, often conflicting, objectives is a central challenge in therapeutic and biomolecular engineering. Existing generative frameworks largely operate in continuous spaces with single-objective guidance, while discrete approaches lack guarantees for multi-objective Pareto optimality. We introduce AReUReDi ( A nnealed Re ctified U pdates for Re fining Di screte Flows), a discrete optimization algorithm with theoretical guarantees of convergence to the Pareto front. Building on Rectified Discrete Flows (ReDi), AReUReDi combines Tchebycheff scalarization, locally balanced proposals, and annealed Metropolis-Hastings updates to bias sampling toward Pareto-optimal states while preserving distributional invariance. Applied to peptide and SMILES sequence design, AReUReDi simultaneously optimizes up to five therapeutic properties (including affinity, solubility, hemolysis, half-life, and non-fouling) and outperforms both evolutionary and diffusion-based baselines. These results establish AReUReDi as a powerful, sequence-based framework for multi-property biomolecule generation.

## 1 Introduction

The design of biological sequences must account for multiple, often conflicting, objectives (Naseri and Koffas,, 2020). Therapeutic peptides, for example, must combine high binding affinity with low toxicity and favorable pharmacokinetics (Tominaga et al.,, 2024; Tang et al., 2025b,); CRISPR guide RNAs require both high on-target activity and minimal off-target effects (Mohr et al.,, 2016; Schmidt et al.,, 2025); and synthetic promoters must deliver strong expression while remaining tissue-specific (Artemyev et al.,, 2024). These examples illustrate that biomolecular engineering is inherently a multi-objective optimization problem.

Yet, most computational frameworks continue to optimize single objectives in isolation (Zhou et al.,, 2019; Nehdi et al.,, 2020; Nisonoff et al.,, 2025). While such approaches can reduce toxicity (Kreiser et al.,, 2020; Sharma et al.,, 2022) or improve thermostability (Komp et al.,, 2025), they often create adverse trade-offs: high-affinity peptides may be insoluble or hemolytic, and stabilized proteins may lose specificity (Bigi et al.,, 2023; Rinauro et al.,, 2024). Black-box multi-objective optimization (MOO) methods such as evolutionary search and Bayesian optimization have long been applied to molecular design (Zitzler and Thiele,, 1998; Deb,, 2011; Ueno et al.,, 2016; Frisby and Langmead,, 2021), but these approaches scale poorly in high-dimensional sequence spaces.

To overcome this, recent generative approaches have incorporated controllable multi-objective sampling (Li et al.,, 2018; Sousa et al.,, 2021; Yao et al.,, 2024). For instance, ParetoFlow (Yuan et al.,, 2024) leverages continuous-space flow matching to generate Pareto-optimal samples. However, extending such guarantees to biological sequences is challenging, since discrete data typically require embedding into continuous manifolds, which distorts token-level structure and complicates property-based guidance (Beliakov and Lim,, 2007; Michael et al.,, 2024).

A more direct path lies in discrete flow models (Campbell et al.,, 2024; Gat et al.,, 2024; Dunn and Koes,, 2024). These models define probability paths over categorical state spaces, either through simplex-based interpolations (Stark et al.,, 2024; Davis et al.,, 2024; Tang et al., 2025a,) or jump-process flows that learn token-level transition rates (Campbell et al.,, 2024; Gat et al.,, 2024). Recent advances have shown their promise for controllable single-objective generation (Nisonoff et al.,, 2025; Tang et al., 2025a,), but no framework yet achieves Pareto guidance across multiple objectives.

Here, the notion of rectification provides a crucial building block. In the continuous setting, Rectified Flows (Liu et al.,, 2023) learn to straighten ODE paths between distributions, thereby reducing convex transport costs and enabling efficient few-step or even one-step sampling. Recently, ReDi (Rectified Discrete Flows) (Yoo et al.,, 2025) extended this principle to discrete domains. By iteratively refining the coupling between source and target distributions, ReDi provably reduces factorization error (quantified as conditional total correlation) while maintaining distributional fidelity. This makes ReDi highly effective for efficient discrete sequence generation. However, ReDi does not address the multi-objective setting, as it lacks a mechanism to steer sampling toward the Pareto front, where improvements in one objective cannot be made without degrading another. This is a critical limitation for biomolecular design, where trade-offs define practical success.

To address this, we introduce AReUReDi (Annealed Rectified Updates for Refining Discrete Flows), a new framework that extends rectified discrete flows with multi-objective guidance. AReUReDi integrates three innovations: (i) annealed Tchebycheff scalarization, which gradually sharpens the focus on balanced solutions across objectives (Lin et al., 2024a,); (ii) locally balanced proposals, which combine the generative prior of ReDi with multi-objective guidance while ensuring reversibility; and (iii) Metropolis-Hastings updates, which preserve exact distributional invariance and guarantee convergence to Pareto-optimal states. Together, these mechanisms refine rectified discrete flows into a principled Pareto sampler.

Our key contributions are:

- 1.
We propose AReUReDi, the first multi-objective extension of rectified discrete flows, integrating annealed scalarization, locally balanced proposals, and MCMC updates.
- 2.
We provide theoretical guarantees that AReUReDi preserves distributional invariance and converges to the Pareto front with full coverage.
- 3.
We demonstrate that AReUReDi can optimize up to five competing biological properties simultaneously, including affinity, solubility, hemolysis, half-life, and non-fouling, for therapeutic peptide design.
- 4.
We benchmark AReUReDi against classical MOO algorithms and state-of-the-art discrete diffusion approaches, showing superior trade-off navigation and biologically plausible peptide sequence designs.

## 2 Preliminaries

### 2.1 Discrete Flow Matching

Let $\mathcal{S}=V^{L}$ denote the discrete state space, where $V$ is a vocabulary of size $K$ and each $x=(x_{1},\dots,x_{L})\in\mathcal{S}$ is a sequence of tokens.
A discrete flow matching (DFM) model (Campbell et al.,, 2024; Gat et al.,, 2024; Dunn and Koes,, 2024) defines a probability path $\{p_{t}\}_{t\in[0,1]}$ interpolating between a simple source distribution $p_{0}$ and a target distribution $p_{1}$ by means of a coupling $\pi(x_{0},x_{1})$ and conditional bridge distributions $p_{t}(x_{t}\mid x_{0},x_{1})$.
The model is trained to approximate conditional transitions $p_{s|t}(x_{s}\mid x_{t})$ for $0\leq t<s\leq 1$.

Since the joint distribution over $L$ coordinates is intractable, DFMs employ a factorization

$$ $p_{s|t}(x_{s}\mid x_{t})\;\approx\;\prod_{i=1}^{L}p_{s|t}\!\left(x_{s}^{i}\mid x_{t}\right),$ $$

which introduces a discrepancy measured by the conditional total correlation

$$ $\mathrm{TC}_{s|t}\;=\;\mathrm{KL}\!\left(p_{s|t}(x_{s}\mid x_{t})\;\middle\|\;\prod_{i=1}^{L}p_{s|t}(x_{s}^{i}\mid x_{t})\right).$ $$

This quantity captures the inter-dimensional dependencies neglected under factorization, and grows with larger step sizes (Stark et al.,, 2024; Davis et al.,, 2024; Tang et al., 2025a,). As a result, DFMs are accurate in the many-step regime but degrade under few-step or one-step generation.

### 2.2 Rectified Discrete Flow

To mitigate factorization error, Rectified Discrete Flow (ReDi) (Yoo et al.,, 2025) introduces an iterative rectification of the coupling $\pi$.
Starting from an initial coupling $\pi^{(0)}(x_{0},x_{1})$, a DFM is trained under $\pi^{(k)}$ to produce new source–target pairs, defining an empirical joint distribution $\hat{\pi}^{(k)}$.
The coupling is then updated via

$$ $\pi^{(k+1)}(x_{0},x_{1})\;\propto\;\pi^{(k)}(x_{0},x_{1})\,\frac{p_{\theta^{(k)}}(x_{1}\mid x_{0})}{p_{\theta^{(k)}}(x_{1})},$ $$

where $p_{\theta^{(k)}}(x_{1}\mid x_{0})$ is the conditional distribution learned at iteration $k$.
This yields a sequence of couplings $\{\pi^{(k)}\}_{k\geq 0}$ with provably decreasing conditional TC,

$$ $\mathrm{TC}_{s|t}(\pi^{(k+1)})\;\leq\;\mathrm{TC}_{s|t}(\pi^{(k)}).$ $$

By progressively reducing factorization error, ReDi produces a well-calibrated base distribution $p_{1}$ with low inter-dimensional correlation. This base distribution provides reliable marginal transition probabilities $p_{t}^{i}(\cdot\mid x_{t})$ for each coordinate $i$ at time $t$, which serve as the generative prior in the AReUReDi framework.
Rectification follows the same principle as Rectified Flow in continuous domains (Liu et al.,, 2023), where iterative refinement straightens ODE paths and decreases transport costs.

### 2.3 Multi-Objective Setup

In biomolecular design and related applications, the generation task is inherently multi-objective (Zitzler and Thiele,, 1998; Deb,, 2011; Frisby and Langmead,, 2021).
Let $s_{1},\dots,s_{N}:\mathcal{S}\to\mathbb{R}$ denote $N$ scalar objectives, and let $\tilde{s}_{n}(x)$ be their normalized counterparts mapping into $[0,1]$ to ensure comparability.
Given weights $\omega\in\Delta^{N-1}$, the Tchebycheff scalarization is defined as

$$ $S_{\omega}(x)\;=\;\min_{1\leq n\leq N}\;\omega_{n}\,\tilde{s}_{n}(x),$ $$

which balances objectives by rewarding solutions that are simultaneously strong across all metrics rather than excelling in just one (Miettinen,, 1999).
This scalarization will serve as the core of the guidance mechanism in AReUReDi.

Figure: Figure 1: AReUReDi. Discrete flow matching is first rectified to reduce conditional total correlation. At each timestep, candidate single-position mutations with ReDi-predicted probabilities (visualized by arrows of varying thickness) are evaluated by multiple objective functions. A locally balanced proposal is then constructed using Tchebycheff scalarization with annealed guidance strength, and the next state is selected via a Metropolis-Hastings update. This iterative process drives the generated sequences toward the Pareto front.
Refer to caption: x1.png

## 3 AReUReDi: Annealed Rectified Updates for Refining Discrete Flows

With an efficient discrete flow-based generation framework in hand, we develop AReUReDi that extends ReDi (Yoo et al.,, 2025) to the multi-objective optimization setting, where the goal is to generate discrete samples that approximate the Pareto front of multiple competing objectives. Starting from a pre-trained ReDi model, AReUReDi incorporates annealed guidance, locally balanced proposals, and Metropolis-Hastings updates to progressively bias the sampling process toward Pareto-optimal states while preserving the probabilistic guarantees of the underlying flow (Figure [1](https://arxiv.org/html/2510.00352v2#S2.F1), Algorithm [1](https://arxiv.org/html/2510.00352v2#alg1)).

### 3.1 Problem Setup

Let the discrete search space be $\mathcal{S}=\mathcal{V}^{L}$, where $\mathcal{V}$ is a finite vocabulary of size $K$ and each state $x=(x_{1},\dots,x_{L})\in\mathcal{S}$ is a sequence of tokens. We assume access to a pre-trained ReDi model that provides marginal transition probabilities $p_{t}^{i}(\cdot\mid x_{t})$ for each position $i$ and time $t$. In addition, we are given $N$ pre-trained scalar objective functions $s_{n}:\mathcal{S}\to\mathbb{R}$, where $n=1,\dots,N$, and $\tilde{s}_{n}(x)$ are their normalized counterparts with outputs mapped to $[0,1]$ to support balanced updates for each objective. The sampling task is to construct a Markov chain whose stationary distribution concentrates on states that approximate the Pareto front of the normalized objectives $\tilde{s}_{1},\dots,\tilde{s}_{N}$.

### 3.2 Annealed Multi-Objective Guidance

To direct sampling toward the Pareto front, AReUReDi introduces a scalarized reward

$$ $S_{\omega}(x)=\min_{1\leq n\leq N}\omega_{n}\,\tilde{s}_{n}(x),$ $$

where the weight vector $\omega=[\omega_{1},\dots,\omega_{N}]$ lies in the probability simplex $\Delta^{N-1}$ and balances the different objectives. This Tchebycheff scalarization promotes solutions that are simultaneously strong across all objectives rather than excelling in only a subset (Miettinen,, 1999). The scalarized reward is converted into a guidance weight

$$ $W_{\eta_{t},\omega}(x)=\exp\big(\eta_{t}S_{\omega}(x)\big),$ $$

where the parameter $\eta_{t}>0$ controls the strength of the guidance at each iteration $t$. AReUReDi incorporates an annealing schedule for $\eta_{t}$:

$$ $\eta_{t}\;=\;\eta_{\min}+\bigl(\eta_{\max}-\eta_{\min}\bigr)\frac{t}{\,T-1\,},$ $$

so that the chain begins with a small value of $\eta_{t}$ to encourage wide exploration of the state space and gradually increases $\eta_{t}$ to focus sampling on high-quality Pareto candidates. This annealing strategy mirrors simulated annealing but operates directly on the scalarized objectives within the discrete flow framework.

### 3.3 Locally Balanced Proposals

Given the current state $x_{t}$, AReUReDi updates one coordinate $i\in\{1,\dots,L\}$ at a time using a locally balanced proposal that blends the generative prior of ReDi with the multi-objective guidance. First, a candidate set of replacement tokens is drawn from the ReDi marginal $p_{t}^{i}(\cdot\mid x_{t})$, optionally pruned using top-p to retain only the most promising alternatives for computational efficiency. For each candidate token $y$, the algorithm computes the ratio

$$ $r_{i}(y;x_{t})=\frac{W_{\eta_{t},\omega}\big(x_{t}^{(i\leftarrow y)}\big)}{W_{\eta_{t},\omega}(x_{t})},$ $$

which measures the change in scalarized reward if $x_{t}^{i}$ were replaced by $y$. The ratio $r_{i}(y;x_{t})$ is then transformed by a balancing function $g:\mathbb{R}_{+}\to\mathbb{R}_{+}$ that satisfies the symmetry condition $g(u)=u\,g(1/u)$. Typical choices include Barker’s function $g(u)=\frac{u}{1+u}$ and the square-root function $g(u)=\sqrt{u}$. This symmetry ensures that the resulting Markov chain admits the desired stationary distribution. Using the balanced function, the unnormalized proposal for a candidate token $y$ takes the form

$$ $\tilde{q}_{i}(y\mid x_{t})=p_{t}^{i}(y\mid x_{t})\,g\big(r_{i}(y;x_{t})\big),$ $$

which is then normalized over the candidate set to yield the final proposal distribution $q_{i}(y\mid x_{t})$. This construction allows the proposal to favor states with higher scalarized reward while remaining reversible with respect to the target distribution.

### 3.4 Metropolis-Hastings Update

A candidate token $y^{\star}$ is drawn from the final proposal distribution $q_{i}(\cdot\mid x_{t})$ and forms the proposed state $x_{\mathrm{prop}}=x_{t}^{(i\leftarrow y^{\star})}$. The proposal is accepted with the standard Metropolis-Hastings probability (Hastings,, 1970)

$$ $\alpha_{i}(x_{t},x_{\mathrm{prop}})=\min\left\{1,\,\frac{\pi_{\eta_{t},\omega}(x_{\mathrm{prop}})\,q_{i}(x_{t}^{i}\mid x_{\mathrm{prop}})}{\pi_{\eta_{t},\omega}(x_{t})\,q_{i}(y^{\star}\mid x_{t})}\right\},$ $$

where we define $\pi_{\eta_{t},\omega}(x)\;\propto\;p_{1}(x)\,W_{\eta_{t},\omega}(x)\;=\;p_{1}(x)\,\exp\!\big(\eta_{t}S_{\omega}(x)\big)$. With Barker’s balancing function, the acceptance probability simplifies to one, ensuring automatic acceptance of proposals and faster mixing. Other choices, such as the square-root function, trade higher acceptance rates for more conservative moves.

The annealed, locally balanced updates are repeated for $T$ iterations and end with the final sample $x_{1}$ whose objective scores are jointly optimized. Building on the ReDi model’s well-calibrated base distribution with low inter-dimensional correlation, AReUReDi safely biases this base toward Pareto-optimal regions while preserving full coverage of the state space, thereby guaranteeing convergence to Pareto-optimal solutions with complete coverage of the Pareto front (Proof [S1](https://arxiv.org/html/2510.00352v2#A1)).

## 4 Experiments

To the best of our knowledge, no public datasets exist for benchmarking multi-objective optimization algorithms on biological sequences. We therefore developed two benchmarks to evaluate AReUReDi, focusing on the generation of wild-type peptide sequences and chemically-modified peptide SMILES. These tasks are supported by two core components: the generative models described in Section [4.1](https://arxiv.org/html/2510.00352v2#S4.SS1) and the objective-scoring models validated in Section [S4](https://arxiv.org/html/2510.00352v2#A4). Leveraging these models, we demonstrate AReUReDi’s efficacy on a wide range of tasks and examples.

Although AReUReDi provides theoretical guarantees of Pareto optimality and full coverage, in practice, these guarantees hold only in the limit of an infinitely long Markov chain. Reaching the Pareto front with high probability can therefore require a vast number of sampling steps. To improve sampling efficiency in all reported experiments, we introduce a monotonicity constraint that accepts only token updates that increase the weighted sum of the current objective scores. Empirical results prove the accelerated convergence toward high-quality Pareto solutions without altering the underlying optimization objectives (Table [S2](https://arxiv.org/html/2510.00352v2#A5.T2)). Therefore, this monotonicity constraint was involved in all the following experiments.

### 4.1 PepReDi and SMILESReDi Generate Diverse and Biologically Plausible Sequences

To enable the efficient generation of peptide binders, we developed an unconditional peptide generator, PepReDi, based on the ReDi framework. The model backbone of PepReDi is a Diffusion Transformer (DiT) architecture (Peebles and Xie,, 2022). We trained PepReDi on a custom dataset comprising approximately 15,000 peptides from the PepNN and BioLip2 datasets, as well as sequences from the PPIRef dataset, with lengths ranging from 6 to 49 amino acids (Abdin et al.,, 2022; Zhang et al.,, 2024; Bushuiev et al.,, 2023). Using this trained model, we generated new data couplings containing 10,000 sequences for each peptide length and used them to fine-tune PepReDi in an iterative rectification procedure. This rectification was performed three times and yielded substantial improvements in training loss, validation negative log-likelihood (NLL), perplexity (PPL), and conditional TC (Table [1](https://arxiv.org/html/2510.00352v2#S4.T1)). Notably, the conditional TC rises after the first rectification, likely due to the distributional shift from the large, model-generated coupling, whose absolute TC can be higher even though ReDi guarantees a monotonic decrease within each coupling. The low validation NLL and PPL metrics showcase PepReDi’s reliability to generate biologically plausible wild-type peptide sequences.

**Table 1: Training and validation performance of PepReDi over successive rectification rounds. Each row reports the training loss, validation negative log-likelihood (NLL), validation perplexity (PPL), and conditional total correlation (TC). PepReDi without superscript denotes the base model, while PepReDi^1, PepReDi^3, PepReDi^3 indicate the first, second, and third rounds of rectification, respectively.**
|  | Train Loss | Val NLL | Val PPL | Conditional TC |
| --- | --- | --- | --- | --- |
| PepReDi | 1.6567 | 1.6458 | 5.19 | 10.6027 |
| PepReDi^1 | 1.6170 | 1.6101 | 5.00 | 12.6250 |
| PepReDi^2 | 1.5347 | 1.5238 | 4.59 | 11.7279 |
| PepReDi^3 | 1.3538 | 1.3548 | 3.88 | 11.2339 |

**Table 2: Evaluation metrics for the generative quality of peptide SMILES sequences of max token length set to 200. SMILESReDi without superscription denotes the base model, while SMILESReDi^1 refers to the model that has undergone one round of rectification.**
| Model | Validity (↑) | Uniqueness (↑) | Diversity (↑) | SNN (↓) |
| --- | --- | --- | --- | --- |
| Data | 1.000 | 1.000 | 0.885 | 1.000 |
| PepMDLM | 0.450 | 1.000 | 0.705 | 0.513 |
| SMILESReDi | 0.763 | 1.000 | 0.719 | 0.593 |
| SMILESReDi^1 | 0.986 | 1.000 | 0.665 | 0.579 |
| PepTune | 1.000 | 1.000 | 0.677 | 0.486 |
| AReUReDi | 1.000 | 1.000 | 0.789 | 0.392 |

SMILESReDi adopts the same backbone structure as PepReDi, enhanced with Rotary Positional Embeddings (RoPE), which effectively captures the relative inter-token interactions in peptide SMILES (Su et al.,, 2024). SMILESReDi also incorporates a time-dependent noising schedule to improve its capability to generate valid peptide SMILES sequences (Section [S2.2](https://arxiv.org/html/2510.00352v2#A2.SS2)). We applied the same training data as PepMDLM, a state-of-the-art diffusion model that generates valid peptide SMILES sequences (Tang et al., 2025b,). After only two training epochs, SMILESReDi converged to a validation NLL of 0.722 and achieved a sampling validity of 76.3% using just 16 generation steps. One hundred SMILES sequences were then generated by the trained SMILESReDi for each length from 4 to 1035, forming a large and diverse new data coupling. Following a single round of rectification, the validation NLL further decreased to 0.608, and the sampling validity rose dramatically to 98.6% with 16 steps and 100% with 32 steps (Table [2](https://arxiv.org/html/2510.00352v2#S4.T2)). While its similarity-to-nearest-neighbor (SNN) score and diversity are comparable to those of PepMDLM (details on metrics are provided in Section [S2.2](https://arxiv.org/html/2510.00352v2#A2.SS2)), SMILESReDi substantially outperforms PepMDLM in validity, highlighting its superior capability of generating diverse chemically-modified peptide SMILES sequences.

### 4.2 AReUReDi effectively balances each objective trade-off

With pre-trained PepReDi in hand, we first focus on validating AReUReDi’s capability of balancing multiple conflicting objectives. We performed two sets of experiments for wild-type peptide binder generation with three property guidance, and in ablation experiment settings, we removed one or more objectives. In the binder design task for target 7LUL (hemolysis, solubility, affinity guidance; Table [S3](https://arxiv.org/html/2510.00352v2#A5.T3)), omitting any single guidance causes a collapse in that property, while the remaining guided metrics may modestly improve. Likewise, in the binder design task for target CLK1 (affinity, non‑fouling, half‑life guidance; Table [S4](https://arxiv.org/html/2510.00352v2#A5.T4)), disabling non‑fouling guidance allows half‑life to exceed 96 hours but drives non‑fouling near zero, and disabling half‑life guidance preserves non‑fouling yet reduces half‑life below 2 hours. In contrast, enabling all guidance signals produces the most balanced profiles across all objectives. These results confirm that AReUReDi precisely targets chosen objectives while preserving the flexibility to navigate conflicting objectives and push samples toward the Pareto front.

Figure: Figure 2: (A), (B) Complex structures of PDB 1B8Q with an AReUReDi-designed binder and its pre-existing binder. (C), (D) Complex structures of OX1R and EWS::FLI1 with an AReUReDi-designed binder. Five property scores are shown for each binder, along with the ipTM score from AlphaFold3 and docking score from AutoDock VINA. Interacting residues on the target are visualized. (E) Plots showing the mean scores for each property across the number of iterations during AReUReDi’s design of binders of length 12-aa for EWS::FLI1. (F) A density plot illustrating the distribution of predicted property scores for AReUReDi-designed EWS::FLI1 binders of length 12-aa, compared to the peptides generated unconditionally by PepReDi^3.
Refer to caption: Figures/four_structures.png

**Table 3: AReUReDi generates wild-type peptide binders for 8 diverse protein targets, optimizing five therapeutic properties: hemolysis, non-fouling, solubility, half-life (in hours), and binding affinity. Each value represents the average of 100 AReUReDi-designed binders.**
| Name | Binder Length | Hemolysis | Non-Fouling | Solubility | Half-Life (h) | Affinity |
| --- | --- | --- | --- | --- | --- | --- |
| AMHR2 | 8 | 0.9156 | 0.8613 | 0.8564 | 45.73 | 7.0608 |
| AMHR2 | 12 | 0.9384 | 0.8872 | 0.8810 | 52.52 | 7.2284 |
| AMHR2 | 16 | 0.9420 | 0.8914 | 0.8755 | 63.34 | 7.2533 |
| EWS::FLI1 | 8 | 0.9186 | 0.8630 | 0.8619 | 44.77 | 5.8424 |
| EWS::FLI1 | 12 | 0.9345 | 0.8819 | 0.8796 | 59.11 | 6.2007 |
| EWS::FLI1 | 16 | 0.9416 | 0.8875 | 0.8807 | 64.32 | 6.4195 |
| MYC | 8 | 0.9180 | 0.8627 | 0.8627 | 44.13 | 6.4082 |
| OX1R | 10 | 0.9302 | 0.8687 | 0.8563 | 50.14 | 7.1882 |
| DUSP12 | 9 | 0.9240 | 0.8669 | 0.8633 | 48.14 | 6.1276 |
| 1B8Q | 8 | 0.9214 | 0.8680 | 0.8654 | 42.63 | 5.7130 |
| 5AZ8 | 11 | 0.9293 | 0.8732 | 0.8605 | 58.33 | 6.2792 |
| 7JVS | 11 | 0.9313 | 0.8840 | 0.8743 | 56.49 | 6.8449 |

### 4.3 AReUReDi generates wild-type peptide binders under five property guidance

We next benchmark AReUReDi on a wild-type peptide binder generation task guided by five different properties that are critical for therapeutic discovery: hemolysis, non-fouling, solubility, half-life, and binding affinity. To evaluate AReUReDi in a controlled setting, we designed 100 peptide binders per target for 8 diverse proteins, structured targets with known binders (3IDJ, 5AZ8, 7JVS), structured targets without known binders (AMHR2, OX1R, DUSP12), and intrinsically disordered targets (EWS::FLI1, MYC) (Table [3](https://arxiv.org/html/2510.00352v2#S4.T3)). Across all targets and across multiple binder lengths, the generated peptides achieve superior hemolysis rates (0.91-0.94), high non‑fouling ($>$0.86) and solubility ($>$0.85), extended half‑life (42-64 h), and strong affinity scores (5.7-7.3), demonstrating both balanced optimization and robustness to sequence length.

For the target proteins with pre-existing binders, we compared the property values between their known binders with AReUReDi-designed ones (Figure [2](https://arxiv.org/html/2510.00352v2#S4.F2)A,B, [S1](https://arxiv.org/html/2510.00352v2#A5.F1)). The designed binders significantly outperform the pre-existing binders across all properties without compromising the binding potential, which is further confirmed by the ipTM scores computed by AlphaFold3 (Abramson et al.,, 2024) and docking scores calculated by AutoDock VINA (Trott and Olson,, 2010). Although the AReUReDi-designed binders bind to similar target positions as the pre-existing ones, they differ significantly in sequence and structure, demonstrating AReUReDi’s capacity to explore the vast sequence space for optimal designs. For target proteins without known binders, complex structures were visualized using one of the AReUReDi-designed binders (Figure [S2](https://arxiv.org/html/2510.00352v2#A5.F2)). The corresponding property scores, as well as ipTM and docking scores, are also displayed. Some of the designed binders showed longer half-life, while others excelled in non-fouling and solubility, underscoring the comprehensive exploration of the sequence space by AReUReDi.

To evaluate our guided generation strategy, we tracked the mean and standard deviation of five property scores across 100 generated binders (length 12) targeting EWS::FLI1 at each iteration (Figure [2](https://arxiv.org/html/2510.00352v2#S4.F2)E). All five properties steadily improved, with average scores for solubility and non-fouling properties increasing markedly from 0̃.4 to 0.9. The large standard deviation observed in the final half-life and binding affinity values reflects this property’s high sensitivity to guidance, as AReUReDi balances the trade-offs between multiple conflicting objectives. We further visualized AReUReDi’s impact by comparing the property distribution of the 100 guided peptides to that of 100 peptides unconditionally sampled from PepReDi^3 (Figure [2](https://arxiv.org/html/2510.00352v2#S4.F2)F). The results show that AReUReDi effectively shifted the distribution towards peptides with higher binding affinity. Collectively, these findings demonstrate AReUReDi’s capability to steer generation toward simultaneous multi-property optimization.

**Table 4: AReUReDi outperforms traditional multi-objective optimization algorithms in designing wild-type peptide binders guided by five objectives. Each value represents the average of 100 designed binders. The table also records the average runtime for each algorithm to design a single binder. The best result for each metric is highlighted in bold.**
| Target | Method | Time (s) | Hemolysis | Non-Fouling | Solubility | Half-Life (h) | Affinity |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 1B8Q | MOPSO | 8.54 | 0.8934 | 0.4763 | 0.4684 | 4.45 | 6.0594 |
| NSGA-III | 33.13 | 0.9138 | 0.5715 | 0.5825 | 7.32 | 7.2178 |  |
| SMS-EMOA | 8.21 | 0.8804 | 0.3450 | 0.3511 | 3.02 | 5.955 |  |
| SPEA2 | 17.48 | 0.9181 | 0.4973 | 0.5057 | 4.13 | 7.3240 |  |
| PepTune + DPLM | 2.46 | 0.8547 | 0.3085 | 0.3213 | 1.17 | 5.2398 |  |
| AReUReDi | 55 | 0.9214 | 0.8680 | 0.8654 | 22.93 | 5.7130 |  |
| PPP5 | MOPSO | 11.34 | 0.9117 | 0.4711 | 0.4255 | 1.77 | 6.6958 |
| NSGA-III | 37.30 | 0.9521 | 0.7138 | 0.7066 | 2.90 | 7.3789 |  |
| SMS-EMOA | 8.43 | 0.8758 | 0.4269 | 0.4334 | 1.03 | 6.2854 |  |
| SPEA2 | 19.02 | 0.9445 | 0.6221 | 0.6098 | 2.61 | 7.6253 |  |
| PepTune + DPLM | 4.80 | 0.8816 | 0.2752 | 0.2636 | 1.27 | 5.8454 |  |
| AReUReDi | 195 | 0.9412 | 0.896 | 0.8832 | 38.28 | 6.7186 |  |

We benchmarked AReUReDi against four established multi-objective optimization (MOO) baselines (NSGA-III (Deb and Jain,, 2013), SMS-EMOA (Beume et al.,, 2007), SPEA2 (Zitzler et al.,, 2001), and MOPSO (Coello and Lechuga,, 2002)) on two protein targets: 1B8Q, a small protein with known peptide binders (Zhang et al.,, 1999), and PPP5, a larger protein without characterized binders (Yang et al.,, 2004) (Table [4](https://arxiv.org/html/2510.00352v2#S4.T4)). Each method generated 100 candidate binders optimized for five properties: hemolysis, non-fouling, solubility, half-life, and binding affinity. While AReUReDi required longer runtimes than evolutionary baselines, it consistently produced the best trade-offs. For both targets, it designed targets with top hemolysis scores, increased non-fouling and solubility by 30-50%, maintained competitive binding affinity, and even extended the half-life by a factor of 3-13 relative to the next-best method. These results underscore AReUReDi’s effectiveness in navigating high-dimensional property landscapes to yield peptide binders with balanced, optimized profiles.

We also compared against PepTune (Tang et al., 2025b,), a recent masked discrete diffusion model for peptide design that couples generation with Monte Carlo Tree Search for MOO. PepTune’s backbone was adapted to the existing DPLM model (Wang et al.,, 2024) for wild-type peptide sequence generation. Despite longer runtimes, AReUReDi substantially outperformed PepTune across all objectives, yielding nearly threefold improvements in non-fouling and solubility and a 22-fold increase in half-life. Together, these comparisons demonstrate that AReUReDi surpasses not only traditional MOO algorithms but also the current state-of-the-art diffusion-based approach for multi-objective-guided wild-type peptide binder design.

Figure: Figure 3: (A) Example 2D SMILES structure of AReUReDi-designed peptide binders with four property scores. (B) Plots showing the mean scores for each property across the number of iterations during AReUReDi’s design of binders of length 200 for NCAM1.
Refer to caption: x3.png

### 4.4 AReUReDi generates therapeutic peptide SMILES under four property guidance

To demonstrate the broad applicability of AReUReDi for multi-objective guided generation of biological sequences, we employed the rectified SMILESReDi model to design chemically-modified peptide binder SMILES sequences for five diverse therapeutic targets. These included the metabolic hormone receptor Glucagon-like peptide-1 receptor (GLP1), the iron transport protein Transferrin receptor (TfR), the Neural Cell Adhesion Molecule 1 (NCAM1), the neurotransmitter transporter GLAST, and the developmental Anti-Müllerian Hormone Receptor Type 2 (AMHR2). For each target, sequence generation was jointly conditioned on a predicted binding-affinity score to the target protein, as long as hemolysis, solubility, and non-fouling, to ensure both potency and desirable physicochemical profiles. Although PepTune is also able to perform multi-property guided design of peptide-binder SMILES sequences, it does not report average property scores for its generated binders, making a direct quantitative comparison with AReUReDi infeasible (Tang et al., 2025b,).

We selected and visualized representative binders with the highest predicted binding affinities for each target (Figure [3](https://arxiv.org/html/2510.00352v2#S4.F3)A, [S3](https://arxiv.org/html/2510.00352v2#A5.F3)A,C, [S4](https://arxiv.org/html/2510.00352v2#A5.F4)A,C). All selected binders achieved high scores across hemolysis, solubility, non-fouling, and binding affinity. During generation, we recorded the mean and standard deviation of all four property scores over 100 binders at each iteration to assess the effectiveness of the multi-objective guidance (Figure [3](https://arxiv.org/html/2510.00352v2#S4.F3)B, [S3](https://arxiv.org/html/2510.00352v2#A5.F3)B,D, [S4](https://arxiv.org/html/2510.00352v2#A5.F4)B,D). Across all targets, binding affinity scores and non-fouling scores showed steady upward trends throughout the generation process, while hemolysis and solubility scores fluctuated, indicating AReUReDi’s effort to balance the four conflicting objectives. Moreover, AReUReDi produces valid sequences with substantially higher diversity and lower SNN than PepTune, indicating both superior novelty and structural variability (Table [2](https://arxiv.org/html/2510.00352v2#S4.T2)). These findings highlight the versatility and reliability of AReUReDi for the de novo design of chemically modified peptide binders across a wide range of therapeutic targets.

### 4.5 Ablation Studies for Rectification and Annealed Guidance Strength

To determine if rectification offers an advantage over standard discrete flow matching, we compared the performance of AReUReDi using three generative models: the base PepReDi model (no rectification), PepReDi (three rounds of rectification), and PepDFM, a standard discrete flow model that follows (Gat et al.,, 2024) and was trained on the same data (Section [S2.3](https://arxiv.org/html/2510.00352v2#A2.SS3)). Under the three settings, wild-type binders were designed for two distinct protein targets: 5AZ8 and AMHR2 (Table [S5](https://arxiv.org/html/2510.00352v2#A5.T5)). For the AMHR2 target, the rectified model achieved the highest scores across all five properties, with its predicted half-life surpassing the next-best method by nearly 13 hours. For the 5AZ8 target, the rectified model yielded a significantly higher half-life while maintaining comparable performance on other metrics. These results indicate that by lowering conditional TC and improving the quality of the probability path, rectification enables AReUReDi to achieve stronger Pareto trade-offs on the more demanding objectives.

We further demonstrated the advantage of using an annealed guidance strength (Table [S6](https://arxiv.org/html/2510.00352v2#A5.T6)). AReUReDi was applied to design wild-type peptide binders for two distinct proteins: a structured protein with known binders (PDB 1DDV) and an intrinsically disordered protein without known binders (P53). Across both targets, any fixed guidance strength, whether set to $\eta_{\min}$, $\eta_{\max}$, or their midpoint, failed to match the performance achieved with an annealed schedule. For 1DDV, annealing produced binders with markedly higher half-life and the best solubility, while maintaining hemolysis, non-fouling, and affinity scores that meet or exceed those of all fixed-$\eta$ settings. A similar trend holds for P53, where the annealing schedule consistently delivers the strongest results across all objectives. These findings confirm that gradually increasing the guidance strength enables AReUReDi to attain more favorable Pareto trade-offs, enhancing challenging properties such as half-life without sacrificing other therapeutic metrics.

## 5 Related Works

Online Multi-Objective Optimization.
Recent work in multi-objective guided generation has focused on online or sequential decision-making, where solutions are refined with new data (Gruver et al.,, 2023; Jain et al.,, 2023; Stanton et al.,, 2022; Ahmadianshalchi et al.,, 2024). A common approach is Bayesian optimization (BO), which builds a surrogate model and proposes evaluations via acquisition functions (Yu et al.,, 2020; Shahriari et al.,, 2015). Multi-objective BO often uses advanced criteria such as EHVI (Emmerich and Klinkenberg,, 2008), information gain (Belakaria et al.,, 2021), or scalarization (Knowles,, 2006; Zhang and Li,, 2007; Paria et al.,, 2020). While AReUReDi also employs Tchebycheff scalarization, it operates in an offline setting, where each sequence requires costly evaluation. This contrasts with the sequential, feedback-driven nature of online methods, making direct comparison inappropriate.

Tchebycheff Scalarization.
Tchebycheff scalarization can identify any Pareto-optimal point and is widely used in multi-objective optimization (Miettinen,, 1999). Recent variants include smooth scalarization for gradient-based algorithms (Lin et al., 2024b,) and OMD-TCH for online learning (Liu et al.,, 2024). AReUReDi is, to our knowledge, the first to apply Tchebycheff scalarization for offline generative design of discrete therapeutic sequences. Future work may extend to many-objective problems or alternative utility functions (Lin et al., 2024a,; Tu et al.,, 2023).

Diffusion and Flow Matching.
Generative approaches such as ParetoFlow and PGD-MOO adapt flow matching or diffusion models for multi-objective optimization (Yuan et al.,, 2024; Annadani et al.,, 2025). These operate in continuous or latent spaces, whereas AReUReDi is designed for discrete token spaces inherent to biological sequences. This domain mismatch precludes direct benchmarking.

Biomolecule Generation.
Offline multi-objective frameworks such as EGD and MUDM have optimized molecules with multiple properties (Sun et al.,, 2025; Han et al.,, 2023), but these emphasize 3D structural representations. By contrast, AReUReDi is sequence-only, operating directly over amino acids or SMILES, which makes structural methods unsuitable as direct comparators.

## 6 Discussion

In this work, we have presented AReUReDi, a multi-objective optimization framework that extends rectified discrete flows to generate biomolecular sequences satisfying multiple, often conflicting, properties. By integrating annealed Tchebycheff scalarization, locally balanced proposals, and Metropolis-Hastings updates, AReUReDi provides theoretical guarantees of convergence to the Pareto front while maintaining full coverage of the solution space. Built on high-quality base generators such as PepReDi and SMILESReDi, the method demonstrates broad applicability across amino acid sequences and chemically modified peptide SMILES. Superior in silico results establish AReUReDi as a general, theoretically-grounded tool for multi-property-guided biomolecular sequence design.

While AReUReDi excels in domains like wild-type and chemically-modified peptide designs, future work will extend to other biological modalities, including DNA, RNA, antibodies, and combinatorial genotype libraries, where multi-objective trade-offs are central. From a theoretical perspective, improving AReUReDi’s efficiency while maintaining the Pareto convergence guarantees and incorporating uncertainty-aware or feedback-driven guidance remain key directions to explore. Ultimately, AReUReDi provides a foundation for designing the next generation of therapeutic molecules that are not only potent but also explicitly optimized for the diverse properties required for clinical success.

## Declarations

### Acknowledgments

We thank Mark III Systems for providing database and hardware support that has contributed to the research reported within this manuscript. We further thank Sophia Tang and Sophia Vincoff for assistance with PepTune benchmarking, and Qiang Liu for discussions related to Rectified Flow Matching.

### Author Contributions

T.C. designed and evaluated PepReDi, SMILESReDi, PepDFM, and AReUReDi, and performed model benchmarking and visualizations. Y.Z. curated and processed the PPIRef dataset for training, and performed molecular docking. P.C. conceived, designed, directed, and supervised the study.

### Data and Materials Availability

All sequences and data needed to evaluate the conclusions are presented in the paper and tables. All code to train PepReDi, SMILESReDi and generate sequences with AReUReDi are accessible by the academic community at [https://huggingface.co/ChatterjeeLab/AReUReDi](https://huggingface.co/ChatterjeeLab/AReUReDi).

### Competing Interests

P.C. is a co-founder of Gameto, Inc., UbiquiTx, Inc., and Atom Bioworks, Inc., and advises companies involved in peptide development. P.C.’s interests are reviewed and managed by the University of Pennsylvania in accordance with their conflict-of-interest policies.

## References

- Abdin et al., (2022)
Abdin, O., Nim, S., Wen, H., and Kim, P. M. (2022).
Pepnn: a deep attention model for the identification of peptide binding sites.
Communications biology, 5(1):503.
- Abramson et al., (2024)
Abramson, J., Adler, J., Dunger, J., Evans, R., Green, T., Pritzel, A., Ronneberger, O., Willmore, L., Ballard, A. J., Bambrick, J., et al. (2024).
Accurate structure prediction of biomolecular interactions with alphafold 3.
Nature, pages 1–3.
- Ahmadianshalchi et al., (2024)
Ahmadianshalchi, A., Belakaria, S., and Doppa, J. R. (2024).
Pareto front-diverse batch multi-objective bayesian optimization.
In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 10784–10794.
- Akiba et al., (2019)
Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. (2019).
Optuna: A next-generation hyperparameter optimization framework.
In International Conference on Knowledge Discovery and Data Mining, pages 2623–2631.
- Annadani et al., (2025)
Annadani, Y., Belakaria, S., Ermon, S., Bauer, S., and Engelhardt, B. E. (2025).
Preference-guided diffusion for multi-objective offline optimization.
arXiv preprint arXiv:2503.17299.
- Artemyev et al., (2024)
Artemyev, V., Gubaeva, A., Paremskaia, A. I., Dzhioeva, A. A., Deviatkin, A., Feoktistova, S. G., Mityaeva, O., and Volchkov, P. Y. (2024).
Synthetic promoters in gene therapy: Design approaches, features and applications.
Cells, 13(23):1963.
- Belakaria et al., (2021)
Belakaria, S., Deshwal, A., and Doppa, J. R. (2021).
Output space entropy search framework for multi-objective bayesian optimization.
Journal of artificial intelligence research, 72:667–715.
- Beliakov and Lim, (2007)
Beliakov, G. and Lim, K. F. (2007).
Challenges of continuous global optimization in molecular structure prediction.
European journal of operational research, 181(3):1198–1213.
- Beume et al., (2007)
Beume, N., Naujoks, B., and Emmerich, M. (2007).
Sms-emoa: Multiobjective selection based on dominated hypervolume.
European journal of operational research, 181(3):1653–1669.
- Bigi et al., (2023)
Bigi, A., Lombardo, E., Cascella, R., and Cecchi, C. (2023).
The toxicity of protein aggregates: new insights into the mechanisms.
- Bostrom et al., (2008)
Bostrom, J., Lee, C. V., Haber, L., and Fuh, G. (2008).
Improving antibody binding affinity and specificity for therapeutic development.
In Therapeutic Antibodies: Methods and Protocols, pages 353–376. Springer.
- Bushuiev et al., (2023)
Bushuiev, A., Bushuiev, R., Kouba, P., Filkin, A., Gabrielova, M., Gabriel, M., Sedlar, J., Pluskal, T., Damborsky, J., Mazurenko, S., et al. (2023).
Learning to design protein-protein interactions with enhanced generalization.
arXiv preprint arXiv:2310.18515.
- Campbell et al., (2024)
Campbell, A., Yim, J., Barzilay, R., Rainforth, T., and Jaakkola, T. (2024).
Generative flows on discrete state-spaces: Enabling multimodal flows with applications to protein co-design.
In Forty-first International Conference on Machine Learning.
- Chen et al., (2009)
Chen, S., Cao, Z., and Jiang, S. (2009).
Ultra-low fouling peptide surfaces derived from natural amino acids.
Biomaterials, 30(29):5892–5896.
- Coello and Lechuga, (2002)
Coello, C. C. and Lechuga, M. S. (2002).
Mopso: A proposal for multiple objective particle swarm optimization.
In Proceedings of the 2002 Congress on Evolutionary Computation. CEC’02 (Cat. No. 02TH8600), volume 2, pages 1051–1056. IEEE.
- Davis et al., (2024)
Davis, O., Kessler, S., Petrache, M., Ceylan, I., Bronstein, M., and Bose, J. (2024).
Fisher flow matching for generative modeling over discrete data.
Advances in Neural Information Processing Systems, 37:139054–139084.
- Deb, (2011)
Deb, K. (2011).
Multi-objective optimisation using evolutionary algorithms: an introduction.
In Multi-objective evolutionary optimisation for product design and manufacturing, pages 3–34. Springer.
- Deb and Jain, (2013)
Deb, K. and Jain, H. (2013).
An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part i: solving problems with box constraints.
IEEE transactions on evolutionary computation, 18(4):577–601.
- Dunn and Koes, (2024)
Dunn, I. and Koes, D. R. (2024).
Exploring discrete flow matching for 3d de novo molecule generation.
ArXiv, pages arXiv–2411.
- D’Aloisio et al., (2021)
D’Aloisio, V., Dognini, P., Hutcheon, G. A., and Coxon, C. R. (2021).
Peptherdia: database and structural composition analysis of approved peptide therapeutics and diagnostics.
Drug Discovery Today, 26(6):1409–1419.
- Emmerich and Klinkenberg, (2008)
Emmerich, M. and Klinkenberg, J.-w. (2008).
The computation of the expected improvement in dominated hypervolume of pareto front approximations.
Rapport technique, Leiden University, 34:7–3.
- Fosgerau and Hoffmann, (2015)
Fosgerau, K. and Hoffmann, T. (2015).
Peptide therapeutics: current status and future directions.
Drug discovery today, 20(1):122–128.
- Frisby and Langmead, (2021)
Frisby, T. S. and Langmead, C. J. (2021).
Bayesian optimization with evolutionary and structure-based regularization for directed protein evolution.
Algorithms for Molecular Biology, 16(1):13.
- Gat et al., (2024)
Gat, I., Remez, T., Shaul, N., Kreuk, F., Chen, R. T., Synnaeve, G., Adi, Y., and Lipman, Y. (2024).
Discrete flow matching.
Advances in Neural Information Processing Systems, 37:133345–133385.
- Gruver et al., (2023)
Gruver, N., Stanton, S., Frey, N., Rudner, T. G., Hotzel, I., Lafrance-Vanasse, J., Rajpal, A., Cho, K., and Wilson, A. G. (2023).
Protein design with guided discrete diffusion.
Advances in neural information processing systems, 36:12489–12517.
- Guntuboina et al., (2023)
Guntuboina, C., Das, A., Mollaei, P., Kim, S., and Barati Farimani, A. (2023).
Peptidebert: A language model based on transformers for peptide property prediction.
The Journal of Physical Chemistry Letters, 14(46):10427–10434.
- Han et al., (2023)
Han, X., Shan, C., Shen, Y., Xu, C., Yang, H., Li, X., and Li, D. (2023).
Training-free multi-objective diffusion model for 3d molecule generation.
In The Twelfth International Conference on Learning Representations.
- Hastings, (1970)
Hastings, W. K. (1970).
Monte carlo sampling methods using markov chains and their applications.
- Jain et al., (2023)
Jain, M., Raparthy, S. C., Hernández-Garcıa, A., Rector-Brooks, J., Bengio, Y., Miret, S., and Bengio, E. (2023).
Multi-objective gflownets.
In International conference on machine learning, pages 14631–14653. PMLR.
- Jain et al., (2024)
Jain, S., Gupta, S., Patiyal, S., and Raghava, G. P. (2024).
Thpdb2: compilation of fda approved therapeutic peptides and proteins.
Drug Discovery Today, page 104047.
- Knowles, (2006)
Knowles, J. (2006).
Parego: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems.
IEEE transactions on evolutionary computation, 10(1):50–66.
- Komp et al., (2025)
Komp, E., Phillips, C., Lee, L. M., Fallin, S. M., Alanzi, H. N., Zorman, M., McCully, M. E., and Beck, D. A. (2025).
Neural network conditioned to produce thermophilic protein sequences can increase thermal stability.
Scientific Reports, 15(1):14124.
- Kreiser et al., (2020)
Kreiser, R. P., Wright, A. K., Block, N. R., Hollows, J. E., Nguyen, L. T., LeForte, K., Mannini, B., Vendruscolo, M., and Limbocker, R. (2020).
Therapeutic strategies to reduce the toxicity of misfolded protein oligomers.
International journal of molecular sciences, 21(22):8651.
- Li et al., (2018)
Li, Y., Zhang, L., and Liu, Z. (2018).
Multi-objective de novo drug design with conditional graph generative model.
Journal of cheminformatics, 10:1–24.
- (35)
Lin, X., Liu, Y., Zhang, X., Liu, F., Wang, Z., and Zhang, Q. (2024a).
Few for many: Tchebycheff set scalarization for many-objective optimization.
arXiv preprint arXiv:2405.19650.
- (36)
Lin, X., Zhang, X., Yang, Z., Liu, F., Wang, Z., and Zhang, Q. (2024b).
Smooth tchebycheff scalarization for multi-objective optimization.
arXiv preprint arXiv:2402.19078.
- Lin et al., (2023)
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2023).
Evolutionary-scale prediction of atomic-level protein structure with a language model.
Science, 379(6637):1123–1130.
- Liu et al., (2024)
Liu, M., Zhang, X., Xie, C., Donahue, K., and Zhao, H. (2024).
Online mirror descent for tchebycheff scalarization in multi-objective optimization.
arXiv preprint arXiv:2410.21764.
- Liu et al., (2023)
Liu, X., Gong, C., and qiang liu (2023).
Flow straight and fast: Learning to generate and transfer data with rectified flow.
In The Eleventh International Conference on Learning Representations.
- Mathur et al., (2016)
Mathur, D., Prakash, S., Anand, P., Kaur, H., Agrawal, P., Mehta, A., Kumar, R., Singh, S., and Raghava, G. P. (2016).
Peplife: a repository of the half-life of peptides.
Scientific reports, 6(1):36617.
- Michael et al., (2024)
Michael, R., Bartels, S., González-Duque, M., Zainchkovskyy, Y., Frellsen, J., Hauberg, S., and Boomsma, W. (2024).
A continuous relaxation for discrete bayesian optimization.
arXiv preprint arXiv:2404.17452.
- Miettinen, (1999)
Miettinen, K. (1999).
Nonlinear multiobjective optimization.
Kluwer, Boston, USA.
- Mohr et al., (2016)
Mohr, S. E., Hu, Y., Ewen-Campen, B., Housden, B. E., Viswanatha, R., and Perrimon, N. (2016).
Crispr guide rna design for research applications.
The FEBS journal, 283(17):3232–3238.
- Naseri and Koffas, (2020)
Naseri, G. and Koffas, M. A. (2020).
Application of combinatorial optimization strategies in synthetic biology.
Nature communications, 11(1):2446.
- Nehdi et al., (2020)
Nehdi, A., Samman, N., Aguilar-Sánchez, V., Farah, A., Yurdusev, E., Boudjelal, M., and Perreault, J. (2020).
Novel strategies to optimize the amplification of single-stranded dna.
Frontiers in Bioengineering and Biotechnology, 8:401.
- Nisonoff et al., (2025)
Nisonoff, H., Xiong, J., Allenspach, S., and Listgarten, J. (2025).
Unlocking guidance for discrete state-space diffusion and flow models.
Proceedings of the 13th International Conference on Learning Representations (ICLR).
- Paria et al., (2020)
Paria, B., Kandasamy, K., and Póczos, B. (2020).
A flexible framework for multi-objective bayesian optimization using random scalarizations.
In Uncertainty in Artificial Intelligence, pages 766–776. PMLR.
- Pedregosa et al., (2011)
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011).
Scikit-learn: Machine learning in python.
J. Mach. Learn. Res., 12(null):2825–2830.
- Peebles and Xie, (2022)
Peebles, W. and Xie, S. (2022).
Scalable diffusion models with transformers.
arXiv preprint arXiv:2212.09748.
- Pirtskhalava et al., (2013)
Pirtskhalava, M., Vishnepolsky, B., and Grigolava, M. (2013).
Transmembrane and antimicrobial peptides. hydrophobicity, amphiphilicity and propensity to aggregation.
arXiv preprint arXiv:1307.6160.
- Rinauro et al., (2024)
Rinauro, D. J., Chiti, F., Vendruscolo, M., and Limbocker, R. (2024).
Misfolded protein oligomers: Mechanisms of formation, cytotoxic effects, and pharmacological approaches against protein misfolding diseases.
Molecular Neurodegeneration, 19(1):20.
- Ronneberger et al., (2015)
Ronneberger, O., Fischer, P., and Brox, T. (2015).
U-net: Convolutional networks for biomedical image segmentation.
In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234–241. Springer.
- Sahoo et al., (2024)
Sahoo, S. S., Arriola, M., Schiff, Y., Gokaslan, A., Marroquin, E., Chiu, J. T., Rush, A., and Kuleshov, V. (2024).
Simple and effective masked diffusion language models.
Advances in Neural Information Processing Systems.
- Schmidt et al., (2025)
Schmidt, H., Zhang, M., Chakarov, D., Bansal, V., Mourelatos, H., Sánchez-Rivera, F. J., Lowe, S. W., Ventura, A., Leslie, C. S., and Pritykin, Y. (2025).
Genome-wide crispr guide rna design and specificity analysis with guidescan2.
Genome biology, 26(1):1–25.
- Shahriari et al., (2015)
Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., and De Freitas, N. (2015).
Taking the human out of the loop: A review of bayesian optimization.
Proceedings of the IEEE, 104(1):148–175.
- Sharma et al., (2022)
Sharma, N., Naorem, L. D., Jain, S., and Raghava, G. P. (2022).
Toxinpred2: an improved method for predicting toxicity of proteins.
Briefings in bioinformatics, 23(5):bbac174.
- Sousa et al., (2021)
Sousa, T., Correia, J., Pereira, V., and Rocha, M. (2021).
Combining multi-objective evolutionary algorithms with deep generative models towards focused molecular design.
In Applications of Evolutionary Computation: 24th International Conference, EvoApplications 2021, Held as Part of EvoStar 2021, Virtual Event, April 7–9, 2021, Proceedings 24, pages 81–96. Springer.
- Stanton et al., (2022)
Stanton, S., Maddox, W., Gruver, N., Maffettone, P., Delaney, E., Greenside, P., and Wilson, A. G. (2022).
Accelerating bayesian optimization for biological sequence design with denoising autoencoders.
In International conference on machine learning, pages 20459–20478. PMLR.
- Stark et al., (2024)
Stark, H., Jing, B., Wang, C., Corso, G., Berger, B., Barzilay, R., and Jaakkola, T. (2024).
Dirichlet flow matching with applications to dna sequence design.
Proceedings of the 41st International Conference on Machine Learning (ICML).
- Su et al., (2024)
Su, J., Ahmed, M., Lu, Y., Pan, S., Bo, W., and Liu, Y. (2024).
Roformer: Enhanced transformer with rotary position embedding.
Neurocomputing, 568:127063.
- Sun et al., (2025)
Sun, R., Feng, D., Yang, S., Wang, Y., and Wang, H. (2025).
Evolutionary training-free guidance in diffusion model for 3d multi-objective molecular generation.
arXiv preprint arXiv:2505.11037.
- Swanson, (2014)
Swanson, R. (2014).
Long live peptides—evolution of peptide half-life extension technologies and emerging hybrid approaches.
Drug Discovery World, 15:57–61.
- (63)
Tang, S., Zhang, Y., and Chatterjee, P. (2025a).
Gumbel-softmax flow matching with straight-through guidance for controllable biological sequence generation.
arXiv preprint arXiv:2503.17361.
- (64)
Tang, S., Zhang, Y., and Chatterjee, P. (2025b).
Peptune: De novo generation of therapeutic peptides with multi-objective-guided discrete diffusion.
Proceedings of the 41st International Conference on Machine Learning (ICML).
- Tominaga et al., (2024)
Tominaga, M., Shima, Y., Nozaki, K., Ito, Y., Someda, M., Shoya, Y., Hashii, N., Obata, C., Matsumoto-Kitano, M., Suematsu, K., et al. (2024).
Designing strong inducible synthetic promoters in yeasts.
Nature Communications, 15(1):10653.
- Trott and Olson, (2010)
Trott, O. and Olson, A. J. (2010).
Autodock vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading.
Journal of computational chemistry, 31(2):455–461.
- Tsuboyama et al., (2023)
Tsuboyama, K., Dauparas, J., Chen, J., Laine, E., Mohseni Behbahani, Y., Weinstein, J. J., Mangan, N. M., Ovchinnikov, S., and Rocklin, G. J. (2023).
Mega-scale experimental analysis of protein folding stability in biology and design.
Nature, 620(7973):434–444.
- Tu et al., (2023)
Tu, B., Kantas, N., Lee, R. M., and Shafei, B. (2023).
Multi-objective optimisation via the r2 utilities.
arXiv preprint arXiv:2305.11774.
- Ueno et al., (2016)
Ueno, T., Rhone, T. D., Hou, Z., Mizoguchi, T., and Tsuda, K. (2016).
Combo: An efficient bayesian optimization library for materials science.
Materials discovery, 4:18–21.
- Wang et al., (2024)
Wang, X., Zheng, Z., Ye, F., Xue, D., Huang, S., and Gu, Q. (2024).
Diffusion language models are versatile protein learners.
arXiv preprint arXiv:2402.18567.
- Yang et al., (2004)
Yang, J., Roe, S. M., Cliff, M. J., Williams, M. A., Ladbury, J. E., Cohen, P. T. W., and Barford, D. (2004).
Molecular basis for tpr domain-mediated regulation of protein phosphatase 5.
The EMBO Journal, 24(1):1–10.
- Yao et al., (2024)
Yao, Y., Pan, Y., Li, J., Tsang, I., and Yao, X. (2024).
Proud: Pareto-guided diffusion model for multi-objective generation.
Machine Learning, 113(9):6511–6538.
- Yoo et al., (2025)
Yoo, J., Kim, W., and Hong, S. (2025).
Redi: Rectified discrete flow.
arXiv preprint arXiv:2507.15897.
- Yu et al., (2020)
Yu, T., Kumar, S., Gupta, A., Levine, S., Hausman, K., and Finn, C. (2020).
Gradient surgery for multi-task learning.
Advances in neural information processing systems, 33:5824–5836.
- Yuan et al., (2024)
Yuan, Y., Chen, C., Pal, C., and Liu, X. (2024).
Paretoflow: Guided flows in multi-objective optimization.
arXiv preprint arXiv:2412.03718.
- Zhang et al., (2024)
Zhang, C., Zhang, X., Freddolino, P. L., and Zhang, Y. (2024).
Biolip2: an updated structure database for biologically relevant ligand–protein interactions.
Nucleic Acids Research, 52(D1):D404–D412.
- Zhang et al., (1999)
Zhang, M., Tochio, H., Zhang, Q., Mandal, P., and Li, M. (1999).
Solution structure of the extended neuronal nitric oxide synthase pdz domain complexed with an associated peptide.
Nature Structural Biology, 6(5):417–421.
- Zhang and Li, (2007)
Zhang, Q. and Li, H. (2007).
Moea/d: A multiobjective evolutionary algorithm based on decomposition.
IEEE Transactions on evolutionary computation, 11(6):712–731.
- Zhang et al., (2023)
Zhang, R., Wu, H., Xiu, Y., Li, K., Chen, N., Wang, Y., Wang, Y., Gao, X., and Zhou, F. (2023).
Pepland: a large-scale pre-trained peptide representation model for a comprehensive landscape of both canonical and non-canonical amino acids.
arXiv preprint arXiv:2311.04419.
- Zhou et al., (2019)
Zhou, R., Jiang, Z., Yang, C., Yu, J., Feng, J., Adil, M. A., Deng, D., Zou, W., Zhang, J., Lu, K., et al. (2019).
All-small-molecule organic solar cells with over 14% efficiency by optimizing hierarchical morphologies.
Nature communications, 10(1):5393.
- Zitzler et al., (2001)
Zitzler, E., Laumanns, M., and Thiele, L. (2001).
Spea2: Improving the strength pareto evolutionary algorithm.
TIK report, 103.
- Zitzler and Thiele, (1998)
Zitzler, E. and Thiele, L. (1998).
Multiobjective optimization using evolutionary algorithms—a comparative case study.
In International conference on parallel problem solving from nature, pages 292–301. Springer.

## Supplementary Information

## Appendix S1 Theoretical Guarantees

In this section, we establish that AReUReDi converges to Pareto-optimal
solutions while preserving coverage of the entire Pareto front. We assume throughout that the state space $\mathcal{S}$ is finite, all objective functions $s_{n}$ are bounded, and their normalized versions $\tilde{s}_{n}$ map to $[0,1]$.

### S1.1 Preliminary Definitions

Definition (Pareto Optimality).
A state $x^{*}\in\mathcal{S}$ is *Pareto optimal* if there exists no $y\in\mathcal{S}$ such that $\tilde{s}_{n}(y)\geq\tilde{s}_{n}(x^{*})$ for all $n\in\{1,\ldots,N\}$ with strict inequality for at least one $n$.

Definition (Pareto Front).
The Pareto front is $\mathcal{P}=\{x\in\mathcal{S}:x\text{ is Pareto optimal}\}$.

Definition (Interior Weight Vector).
A weight vector $\omega\in\Delta^{N-1}$ is *interior* if $\omega_{n}>0$ for all $n$.

### S1.2 Main Theoretical Results

Theorem (Invariance).
The Markov kernel defined by the Locally Balanced Proposal (LBP) and Metropolis–Hastings update leaves the distribution

$$ $\pi_{\eta,\omega}(x)\propto p_{1}(x)\exp\!\big(\eta S_{\omega}(x)\big)$ $$

invariant for every guidance strength $\eta>0$ and weight vector $\omega\in\Delta^{N-1}$.

###### Proof.

We prove this in two steps: first showing that single-coordinate updates preserve detailed balance, then that random-scan mixtures preserve invariance.

Step 1: Single-coordinate detailed balance.
Let $x$ and $x^{\prime}$ differ only at coordinate $i$, where $x^{\prime}_{i}=y$ for some token $y$. The proposal probability is

$$ $q_{i}(y\mid x)=\frac{p_{t}^{i}(y\mid x_{t})g(r_{i}(y;x_{t}))}{\sum_{z\in\text{candidates}}p_{t}^{i}(z\mid x_{t})g(r_{i}(z;x_{t}))},$ $$

where $r_{i}(y;x_{t})=\frac{W_{\eta_{t},\omega}(x_{t}^{(i\leftarrow y)})}{W_{\eta_{t},\omega}(x_{t})}$ and $g$ satisfies $g(u)=u\cdot g(1/u)$.

The acceptance probability is

$$ $\alpha_{i}(x,x^{\prime})=\min\left\{1,\frac{\pi_{\eta,\omega}(x^{\prime})q_{i}(x_{i}\mid x^{\prime})}{\pi_{\eta,\omega}(x)q_{i}(y\mid x)}\right\}.$ $$

By the symmetry property of $g$ and the construction of the proposal, we have

$$ $\frac{q_{i}(y\mid x)}{q_{i}(x_{i}\mid x^{\prime})}=\frac{W_{\eta,\omega}(x^{\prime})}{W_{\eta,\omega}(x)}.$ $$

Since $\pi_{\eta,\omega}(x)=Z^{-1}p_{1}(x)W_{\eta,\omega}(x)$, it follows that

$$ $\frac{\pi_{\eta,\omega}(x^{\prime})q_{i}(x_{i}\mid x^{\prime})}{\pi_{\eta,\omega}(x)q_{i}(y\mid x)}=1.$ $$

Therefore, $\alpha_{i}(x,x^{\prime})=1$ and detailed balance is satisfied.

Step 2: Random-scan mixture.
The overall kernel is $K(x,x^{\prime})=\frac{1}{L}\sum_{i=1}^{L}K_{i}(x,x^{\prime})$, where $K_{i}$ is the kernel for updating coordinate $i$. Since each $K_{i}$ satisfies detailed balance with respect to $\pi_{\eta,\omega}$, their convex combination also satisfies detailed balance and hence preserves invariance.
∎

Theorem (Convergence to Pareto Front).
Fix any $\omega\in\mathrm{int}\,\Delta^{N-1}$ with strictly positive entries and let $S_{\omega}(x)=\min_{n}\omega_{n}\tilde{s}_{n}(x)$. If $\eta\to\infty$, samples drawn from $\pi_{\eta,\omega}(x)\propto p_{1}(x)\exp(\eta S_{\omega}(x))$ concentrate on the set

$$ $\mathcal{F}_{\omega}=\arg\max_{x}S_{\omega}(x),$ $$

and every element of $\mathcal{F}_{\omega}$ is Pareto optimal.

###### Proof.

Step 1: Maximizers of $S_{\omega}$ are Pareto optimal.
Suppose $x^{*}\in\mathcal{F}_{\omega}$ but $x^{*}$ is not Pareto optimal. Then there exists $y\in\mathcal{S}$ with

$$ $\tilde{s}_{n}(y)\geq\tilde{s}_{n}(x^{*})\ \forall n,\quad\text{and}\quad\tilde{s}_{m}(y)>\tilde{s}_{m}(x^{*})\ \text{for some }m.$ $$

Since $\omega_{n}>0$ for all $n$, multiplying preserves inequalities. If $m$ is the bottleneck coordinate of $x^{*}$, then $S_{\omega}(y)>S_{\omega}(x^{*})$, contradiction. Otherwise, equality requires special weight alignments (measure zero). Thus maximizers are Pareto optimal almost surely.

Step 2: Concentration as $\eta\to\infty$.
Let $S_{\omega}^{*}=\max_{x}S_{\omega}(x)$ and $\Delta_{\omega}=S_{\omega}^{*}-\max_{x\notin\mathcal{F}_{\omega}}S_{\omega}(x)>0$. Then for $x\notin\mathcal{F}_{\omega}$,

$$ $\pi_{\eta,\omega}(x)\leq e^{-\eta\Delta_{\omega}}\cdot\frac{p_{1}(x)}{\sum_{z\in\mathcal{F}_{\omega}}p_{1}(z)}.$ $$

Summing gives $\pi_{\eta,\omega}(\mathcal{S}\setminus\mathcal{F}_{\omega})\to 0$ as $\eta\to\infty$. Hence the mass concentrates on $\mathcal{F}_{\omega}$.
∎

Theorem (Pareto Point Representability).
For every Pareto-optimal state $x^{\dagger}\in\mathcal{P}$ there exists $\omega\in\Delta^{N-1}$ such that $x^{\dagger}\in\arg\max_{x}S_{\omega}(x)$. Moreover, if $\tilde{s}_{n}(x^{\dagger})>0$ for all $n$, then $x^{\dagger}$ can be made the unique maximizer.

###### Proof.

If $\tilde{s}_{n}(x^{\dagger})>0$, define

$$ $\omega_{n}=\frac{1/\tilde{s}_{n}(x^{\dagger})}{\sum_{k=1}^{N}1/\tilde{s}_{k}(x^{\dagger})}.$ $$

Then $S_{\omega}(x^{\dagger})=\frac{1}{\sum_{k}1/\tilde{s}_{k}(x^{\dagger})}$, and for any $y\neq x^{\dagger}$, some $m$ satisfies $\tilde{s}_{m}(y)<\tilde{s}_{m}(x^{\dagger})$, implying $S_{\omega}(y)<S_{\omega}(x^{\dagger})$.
If some $\tilde{s}_{n}(x^{\dagger})=0$, perturb objectives by $\varepsilon>0$ and take the limit.
∎

Theorem (Coverage Guarantee).
Let $\mu$ be any probability distribution with full support on $\mathrm{int}\,\Delta^{N-1}$. If $\omega\sim\mu$ and $\eta\to\infty$, then the induced sampler visits every Pareto-optimal state with positive probability.

###### Proof.

By representability, each Pareto point $x^{\dagger}$ maximizes $S_{\omega}$ for some interior $\omega^{\dagger}$. By continuity, there exists a neighborhood $U_{x^{\dagger}}$ where $x^{\dagger}$ remains optimal. Since $\mu(U_{x^{\dagger}})>0$, randomizing $\omega$ ensures $x^{\dagger}$ is visited with positive probability in the high-$\eta$ limit.
∎

Remark.
The guarantees hold for any finite $\mathcal{S}$ and bounded objectives. In practice, convergence depends on the chain mixing rate, the annealing schedule for $\eta$, and the choice of balancing function $g$.

###### Proof.

###### Proof.

###### Proof.

###### Proof.

## Appendix S2 Base Model Details

### S2.1 PepReDi

Model Architecture. The backbone of PepReDi is built on a Diffusion Transformer (DiT) framework implemented within a Masked Diffusion Language Model (MDLM) paradigm (Peebles and Xie,, 2022; Sahoo et al.,, 2024). Input amino acid sequences are transformed to discrete tokens using the ESM-2-650M tokenizer (Lin et al.,, 2023). Tokenized amino acid sequences and time-steps are converted to continuous embedding vectors using two separate layers, which are then fused and processed by stacked DiT transformer blocks equipped with multi-head self-attention to capture long-range dependencies in the amino-acid sequence. Residual connections and layer normalization stabilize the training dynamics, and a final projection layer outputs token logits for each position.

Dataset Curation. The dataset for PepReDi training was curated from the PepNN, BioLip2, and PPIRef dataset (Abdin et al.,, 2022; Zhang et al.,, 2024; Bushuiev et al.,, 2023). All peptides from PepNN and BioLip2 were included, along with sequences from PPIRef ranging from 6 to 49 amino acids in length. The dataset was divided into training, validation, and test sets at an 80/10/10 ratio.

Training Strategy. Training was conducted on a single node equipped with one NVIDIA GPU and 128 GB of GPU memory using the SLURM workload manager. The model was trained for 100 epochs using the Adam optimizer and a learning rate of 1e-4 with weight decay of 1e-5. A learning rate scheduler with 10 warm-up epochs and cosine decay was used, with initial and minimum learning rates both 1e-5. The network architecture included a model dimension of 512, 6 transformer layers, and 8 attention heads, with a vocabulary size of 24 and a maximum sequence length of 100 tokens. Conditional total correlation estimation was performed using 20 batches and 50 samples per batch to monitor rectification quality during training. The model checkpoint with the lowest total correlation was saved. For training rectified models, the same hyperparameter setting was applied, except for the loaded pre-trained model checkpoint and the weight decay being increased to 2e-5.

Dynamic Batching. To enhance computational efficiency and manage variable-length token sequences, we implemented dynamic batching. Drawing inspiration from ESM-2’s approach (Lin et al.,, 2023), input peptide sequences were sorted by length to optimize GPU memory utilization, with a maximum token size of 100 per GPU.

Rectification. The trained model applied 16 sampling steps to generate 10k sequences for each peptide length, ranging from 6 to 49, with a temperature hyperparameter set to 1. After generation, dynamic batching was used to optimize GPU memory utilization for future rectified training.

### S2.2 SMILESReDi

Model Architecture. SMILESReDi follows the ReDi paradigm and uses a Diffusion Transformer (DiT) backbone embedded in a Masked Diffusion Language Model (MDLM) design to generate molecular SMILES sequences (Peebles and Xie,, 2022; Sahoo et al.,, 2024). Input SMILES sequences are transformed to discrete tokens using the PeptideCLM -23M tokenizer. Tokenized amino acid sequences and time-steps are converted to continuous embedding vectors using two separate layers. Both embeddings are then fused and processed by stacked DiT transformer blocks that incorporate Rotary Positional Embeddings (RoPE) and multi-head attention modules to capture long-range structural dependencies while preserving positional information (Su et al.,, 2024). A final layer normalization and linear projection outputs token logits for each position.

Time-dependent bond-aware noising schedule. Peptide SMILES share a conserved backbone of alternating carbonyl and amide groups connected by chemically constrained peptide bonds, while their side chains remain highly diverse. Standard discrete flow matching can corrupt these critical bond tokens too early, hindering the flow from recovering the backbone along the probability path. Inspired by previous work in bond-dependent masking, we devised a time-dependent bond-aware noising schedule that preserves backbone tokens longer than side-chain tokens, allowing the model to reconstruct the invariant scaffold before generating variable side chains. Specifically, for each position $j$ with a bond indicator $b_{j}\in\{0,1\}$, the time-$t$ marginal of the probability path is

$$ $p_{t}(x_{t}^{(j)}\mid x_{0}^{(j)},x_{1}^{(j)})=\big[\,b_{j}t^{\gamma}+(1-b_{j})t\,\big]\,\delta_{x_{1}^{(j)}}+\big[\,1-b_{j}t^{\gamma}-(1-b_{j})t\,\big]\,\delta_{x_{0}^{(j)}},\quad t\in[0,1],\ \gamma>1,$ $$

so each token is equal to $x_{1}^{(j)}$ with the indicated mixture coefficient and to $x_{0}^{(j)}$ otherwise, ensuring that backbone tokens ($b_{j}=1$) transition more slowly than non-bond tokens along the DFM probability path.

Training Strategy. The training is conducted on a 4*A6000 NVIDIA RTX 6000 Ada GPU system with 48 GB of VRAM for 5 epochs. The model checkpoint with the lowest evaluation loss was saved. The Adam optimizer was employed with a learning rate of 1e-4. A learning rate scheduler with 10% total training steps and cosine decay was used, with initial and minimum learning rates both 1e-5. The network architecture included a model dimension of 768, 8 transformer layers, and 8 attention heads. Gradient clip value was set to 1.0 and $\gamma$ to 2.0 in the time-dependent bond-aware noising schedule. For training rectified models, the same hyperparameter setting was applied, except for the loaded pre-trained model checkpoint and the total training epochs set to 10.

Rectification. The trained model applied 100 sampling steps to generate 100 sequences for each peptide length, ranging from 4 to 1035, with a temperature hyperparameter set to 1. After generation, dynamic batching was used to optimize GPU memory utilization for future rectified training.

Evaluation Metrics.

- •
Validity is defined as the fraction of peptide SMILES that pass the SMILES2PEPTIDE filter (Tang et al., 2025b,), indicating that it translates to a synthesizable peptide.
- •
Uniqueness is defined as the fraction of mutually distinct peptide SMILES.
- •
Diversity is defined as one minus the average Tanimoto similarity between the Morgan fingerprints of every pair of generated sequences, which measures the similarity in structure across generated peptides.
$\text{Diversity}=1-\frac{1}{\binom{N_{\text{generated}}}{2}}\sum_{i,j}\frac{\mathbf{f}(\mathbf{x}_{i})\cdot\mathbf{f}(\mathbf{x}_{j})}{|\mathbf{f}(\mathbf{x}_{i})|+|\mathbf{f}(\mathbf{x}_{j})|-\mathbf{f}(\mathbf{x}_{i})\cdot\mathbf{f}(\mathbf{x}_{j})}$
where $\mathbf{f}(\mathbf{x}_{i})$ and $\mathbf{f}(\mathbf{x}_{j})$ are the 2048-dimensional Morgan fingerprint with radius 3 for a pair of generated sequences $\mathbf{x}_{i}$ and $\mathbf{x}_{j}$.
- •
Similarity to Nearest Neighbor (SNN) is defined as the maximum Tanimoto similarity between a generated sequence $\mathbf{x}_{i}$ with a sequence in the dataset $\tilde{\mathbf{x}}_{j}$.
$\text{SNN}=\max_{j\in|\mathcal{D}|}\left(\frac{\mathbf{f}(\mathbf{x}_{i})\cdot\mathbf{f}(\tilde{\mathbf{x}}_{j})}{|\mathbf{f}(\mathbf{x}_{i})|+|\mathbf{f}(\tilde{\mathbf{x}}_{j})|-\mathbf{f}(\mathbf{x}_{i})\cdot\mathbf{f}(\tilde{\mathbf{x}}_{j})}\right)$

### S2.3 PepDFM

Model Architecture. The base model is a time-dependent architecture based on U-Net (Ronneberger et al.,, 2015). It uses two separate embedding layers for sequence and time, followed by five convolutional blocks with varying dilation rates to capture temporal dependencies, while incorporating time-conditioning through dense layers. The final output layer generates logits for each token. We used a polynomial convex schedule with a polynomial exponent of 2.0 for the mixture discrete probability path in the discrete flow matching.

Dataset Curation. The dataset for PepDFM training was curated from the PepNN, BioLip2, and PPIRef dataset (Abdin et al.,, 2022; Zhang et al.,, 2024; Bushuiev et al.,, 2023). All peptides from PepNN and BioLip2 were included, along with sequences from PPIRef ranging from 6 to 49 amino acids in length. The dataset was divided into training, validation, and test sets at an 80/10/10 ratio.

Training Strategy. The training is conducted on a 2xH100 NVIDIA NVL GPU system with 94 GB of VRAM for 200 epochs with batch size 512. The model checkpoint with the lowest evaluation loss was saved. The Adam optimizer was employed with a learning rate 1e-4. A learning rate scheduler with 20 warm-up epochs and cosine decay was used, with initial and minimum learning rates both 1e-5. The embedding dimension and hidden dimension were set to be 512 and 256 respectively for the base model.

Performance. PepDFM achieved a validation loss of 3.1051. Its low generalized KL loss during evaluation demonstrates PepDFM’s strong capability to generate sequences with high biological plausibility (Gat et al.,, 2024).

## Appendix S3 Objective Description

In this work, five key property objectives are considered in the peptide binder tasks: hemolysis, non-fouling, solubility, half-life, and binding affinity. Each of these properties plays a crucial role in optimizing the therapeutic potential of peptides. Hemolysis refers to the peptide’s ability to minimize red blood cell lysis, ensuring safe systemic circulation (Pirtskhalava et al.,, 2013). Non-fouling properties describe the peptide’s resistance to unwanted interactions with biomolecules, thus enhancing its stability and bioavailability in vivo (Chen et al.,, 2009). Solubility is critical for ensuring adequate peptide dissolution in biological fluids, directly influencing its absorption and therapeutic efficacy (Fosgerau and Hoffmann,, 2015). Half-life indicates the duration for which the peptide remains active in circulation, which is vital for reducing dosing frequency (Swanson,, 2014). Finally, binding affinity measures the strength of the peptide’s interaction with its target, directly correlating to its biological activity and potency in therapeutic applications (Bostrom et al.,, 2008).

## Appendix S4 Score Model Details

We applied the score models from (Tang et al., 2025b,) to guide the generation of chemically-modified peptide binders. We now introduce the score model developed for the wild-type peptide binder generation task. We collected hemolysis (9,316), non-fouling (17,185), solubility (18,453), and binding affinity (1,781) data for classifier training from the PepLand and PeptideBERT datasets (Zhang et al.,, 2023; Guntuboina et al.,, 2023). All sequences taken are wild-type L-amino acids and are tokenized and represented by the ESM-2 protein language model (Lin et al.,, 2023).

### S4.1 Boosted Trees for Classification

For hemolysis, non-fouling, and solubility classification, we trained XGBoost boosted tree models for logistic regression. We split the data into 0.8/0.2 train/validation using stratified splits from scikit-learn (Pedregosa et al.,, 2011) and generated mean-pooled ESM-2-650M (Lin et al.,, 2023) embeddings as input features to the model. We ran 50 trials of OPTUNA (Akiba et al.,, 2019) search to determine the optimal XGBoost hyperparameters (Table [S1](https://arxiv.org/html/2510.00352v2#A4.T1)), tracking the best binary classification F1 scores. The best models for each property reached F1 scores of 0.58, 0.71, and 0.68 on the validation sets respectively.

**Table S1: XGBoost Hyperparameters for Classification**
| Hyperparameter | Value/Range |
| --- | --- |
| Objective | binary:logistic |
| Lambda | $[1\text{e}{-8},10.0]$ |
| Alpha | $[1\text{e}{-8},10.0]$ |
| Colsample by Tree | $[0.1,1.0]$ |
| Subsample | $[0.1,1.0]$ |
| Learning Rate | $[0.01,0.3]$ |
| Max Depth | $[2,30]$ |
| Min Child Weight | $[1,20]$ |
| Tree Method | hist |

### S4.2 Binding Affinity Score Model

We developed an unpooled reciprocal attention transformer model to predict protein-peptide binding affinity, leveraging latent representations from the ESM-2 650M protein language model (Lin et al.,, 2023). Instead of relying on pooled representations, the model retains unpooled token-level embeddings from ESM-2, which are passed through convolutional layers followed by cross-attention layers. The binding affinity data were split into a 0.8/0.2 ratio, maintaining similar affinity score distributions across splits. We used OPTUNA (Akiba et al.,, 2019) for hyperparameter optimization, tracing validation correlation scores. The final model was trained for 50 epochs with a learning rate of 3.84e-5, a dropout rate of 0.15, 3 initial CNN kernel layers (dimension 384), 4 cross-attention layers (dimension 2048), and a shared prediction head (dimension 1024) in the end. The classifier reached 0.64 Spearman’s correlation score on validation data.

### S4.3 Half-Life Score Model

Dataset Curation. The half-life dataset is curated from three publicly available datasets: PEPLife, PepTherDia, and THPdb2 (Mathur et al.,, 2016; D’Aloisio et al.,, 2021; Jain et al.,, 2024). Data related to human subjects were selected, and entries with missing half-life values were excluded. After removing duplicates, the final dataset consists of 105 entries.

Pre-training on stability data. Given the small size of the half-life dataset, which is insufficient for training a model to capture the underlying data distribution, we first pre-trained a score model on a larger stability dataset to predict peptide stability (Tsuboyama et al.,, 2023). The model consists of three linear layers with ReLU activation functions, and a dropout rate of 0.3 was applied. The model was trained on a 2xH100 NVIDIA NVL GPU system with 94 GB of VRAM for 50 epochs. The Adam optimizer was employed with a learning rate of 1e-2. A learning rate scheduler with 5 warm-up epochs and cosine decay was used, with initial and minimum learning rates both 1e-3. After training, the model achieved a validation Spearman’s correlation of 0.7915 and an $R^{2}$ value of 0.6864, demonstrating the reliability of the stability score model.

Fine-tuning on half-life data. The pre-trained stability score model was subsequently fine-tuned on the half-life dataset. Since half-life values span a wide range, the model was adapted to predict the base-10 logarithm of the half-life (h) values to stabilize the learning process. After fine-tuning, the model achieved a validation Spearman’s correlation of 0.8581 and an $R^{2}$ value of 0.5977.

## Appendix S5 Sampling Details

Score Model Settings. We cap the predicted log-scale half-life at 2 (i.e., 100 h) to prevent it from dominating the optimization and ensure balanced trade-offs across all properties. For the remaining objectives, hemolysis, non-fouling, solubility, and binding affinity, we directly employ their model outputs during sampling.

Wild-Type Peptide Binder Generation Task Settings. The total sampling steps are set to 20 multiplied by the binder length. All possible candidate token transitions are evaluated during each sampling step. We applied the same weight for each objective in all wild-type peptide binder generation tasks.

Chemically-Modified Peptide Binder Generation Task Settings. The total sampling steps are set to 128. With a vocabulary size of 586, evaluating all the possible candidate tokens is too computationally intensive. We therefore only evaluated the top 200 candidate tokens during each sampling step. We applied weight 0.7 for binding affinity, and 0.1 for hemolysis, non-fouling, and solubility, respectively. Instead of random initialization, the initial sequences $x_{0}$ are sampled from the pre-trained SMILESReDi^1 with 16 generation steps. During generation, AReUReDi rejects any transitions that will make the SMILES sequence an invalid peptide.

**Table S2: Adding a sampling constraint greatly improves AReUReDi’s performance. Wild-type binders for two protein targets (PDB 8CN1 and 4EBP2) were generated with or without a sampling constraint using the same number of generation steps. The table reports the average score for each objective, calculated from 100 generated binders per setting. The best score for each objective is highlighted in bold.**
| Target | Method | Hemolysis | Non-Fouling | Solubility | Half-Life | Affinity |
| --- | --- | --- | --- | --- | --- | --- |
| 8CN1 | w/o constraints | 0.8650 | 0.4782 | 0.4627 | 2.54 | 5.2412 |
| w/ constraints | 0.9213 | 0.8676 | 0.8697 | 44.70 | 5.5143 |  |
| 4EBP2 | w/o constraints | 0.8879 | 0.4288 | 0.4257 | 1.8781 | 5.7132 |
| w/ constraints | 0.9356 | 0.8767 | 0.8692 | 53.95 | 6.4571 |  |

**Table S3: Ablation results for wild-type peptide binder design targeting PDB 7LUL with different guidance settings. For each setting, 100 binders of length 7 were designed.**
| Guidance Settings<br>Hemolysis Solubility Affinity | Hemolysis | Solubility | Affinity |
| --- | --- | --- | --- |
| $\checkmark$     $\checkmark$     $\checkmark$ | 0.9389 | 0.9398 | 6.2559 |
| $\times$     $\checkmark$     $\checkmark$ | 0.8964 | 0.9465 | 6.3272 |
| $\checkmark$     $\times$     $\checkmark$ | 0.9502 | 0.4013 | 6.9798 |
| $\checkmark$     $\checkmark$     $\times$ | 0.9535 | 0.9642 | 5.2611 |
| $\times$     $\times$     $\checkmark$ | 0.8812 | 0.2877 | 7.5057 |
| $\times$     $\checkmark$     $\times$ | 0.9036 | 0.9725 | 5.2449 |
| $\checkmark$     $\times$     $\times$ | 0.9802 | 0.6135 | 5.0985 |
| $\times$     $\times$     $\times$ | 0.8431 | 0.5810 | 4.8919 |

**Table S4: Ablation results for wild-type peptide binder design targeting PDB CLK1 with different guidance settings. For each setting, 100 binders of length 12 were designed.**
| Guidance Settings<br>Non-Fouling Half-Life (h)  Affinity | Non‑Fouling | Half‑Life (h) | Affinity |
| --- | --- | --- | --- |
| $\checkmark$<br>$\checkmark$<br>$\checkmark$ | 0.8285 | 74.04 | 6.8099 |
| $\times$<br>$\checkmark$<br>$\checkmark$ | 0.2902 | 96.59 | 7.3906 |
| $\checkmark$<br>$\times$<br>$\checkmark$ | 0.9365 | 1.33 | 7.2029 |
| $\checkmark$<br>$\checkmark$<br>$\times$ | 0.9479 | 75.68 | 6.3437 |
| $\times$<br>$\times$<br>$\checkmark$ | 0.9625 | 1.23 | 6.2319 |
| $\times$<br>$\checkmark$<br>$\times$ | 0.3540 | 100.00 | 6.4116 |
| $\checkmark$<br>$\times$<br>$\times$ | 0.2531 | 2.96 | 8.6580 |
| $\times$<br>$\times$<br>$\times$ | 0.4988 | 1.82 | 5.4739 |

**Table S5: Rectification of the base generation model improves AReUReDi’s performance. Wild-type binders for two protein targets (PDB 5AZ8 and AMHR2) were generated using AReUReDi with three different base models: PepDFM, PepReDi (without rectification), and PepReDi^3 (with three rounds of rectification). The table reports the average score for each objective, calculated from 100 generated binders per setting. The best score for each objective is highlighted in bold.**
| Target | Base Model | Hemolysis | Non-Fouling | Solubility | Half-Life | Affinity |
| --- | --- | --- | --- | --- | --- | --- |
| 5AZ8 | PepDFM | 0.9296 | 0.8867 | 0.8743 | 37.30 | 6.2291 |
| PepReDi | 0.9326 | 0.8759 | 0.8572 | 50.16 | 6.4391 |  |
| PepReDi^3 | 0.9293 | 0.8732 | 0.8605 | 58.33 | 6.2792 |  |
| AMHR2 | PepDFM | 0.9412 | 0.8774 | 0.8612 | 47.84 | 7.2373 |
| PepReDi | 0.9127 | 0.8602 | 0.8460 | 50.92 | 7.0101 |  |
| PepReDi^3 | 0.9420 | 0.8914 | 0.8755 | 63.34 | 7.2533 |  |

**Table S6: Annealed guidance strength improves AReUReDi’s performance. Wild-type binders for two protein targets (PDB 1DDV and P53) were generated under four guidance schedules: (1) fixed at the minimum strength $\eta_{min}=1.0$, (2) fixed at the maximum strength $\eta_{max}=20.0$, (3) fixed at the midpoint $\frac{1}{2}(\eta_{min}+\eta_{max})=10.5$, and (4) an annealed schedule where $\eta_{t}$ increases from $\eta_{min}$ to $\eta_{max}$ over optimization steps. The table reports the average score for each objective, calculated from 100 generated binders per setting. The best score for each objective is highlighted in bold.**
| Target | Method | Hemolysis | Non-Fouling | Solubility | Half-Life (h) | Affinity |
| --- | --- | --- | --- | --- | --- | --- |
| 1DDV | $\eta=\eta_{min}$ | 0.9130 | 0.8575 | 0.8429 | 38.70 | 5.3554 |
| $\eta=\eta_{max}$ | 0.9156 | 0.8512 | 0.8479 | 40.27 | 5.4359 |  |
| $\eta=\frac{1}{2}(\eta_{min}+\eta_{max})$ | 0.9108 | 0.8641 | 0.8544 | 40.43 | 5.5396 |  |
| $\eta_{t}\;=\;\eta_{\min}+\bigl(\eta_{\max}-\eta_{\min}\bigr)\frac{t}{\,T-1\,}$ | 0.9128 | 0.8545 | 0.8565 | 44.73 | 5.4482 |  |
| P53 | $\eta=\eta_{min}$ | 0.9335 | 0.8800 | 0.8706 | 49.97 | 6.2538 |
| $\eta=\eta_{max}$ | 0.9293 | 0.8693 | 0.8657 | 61.76 | 6.3043 |  |
| $\eta=\frac{1}{2}(\eta_{min}+\eta_{max})$ | 0.9294 | 0.8713 | 0.8653 | 59.43 | 6.3060 |  |
| $\eta_{t}\;=\;\eta_{\min}+\bigl(\eta_{\max}-\eta_{\min}\bigr)\frac{t}{\,T-1\,}$ | 0.9353 | 0.8818 | 0.8785 | 62.83 | 6.3508 |  |

Figure: Figure S1: Complex structures of target proteins with pre-existing binders. (A)-(B) 5AZ8 (C)-(D) 7JVS. Each panel shows the complex structure of the target with either an AReUReDi-designed binder or its pre-existing binder. For each binder, five property scores are provided, as well as the ipTM score from AlphaFold3 and the docking score from AutoDock VINA. Interacting residues on the target are visualized.
Refer to caption: Figures/peptide_known_binders.png

Figure: Figure S2: Complex structures of target proteins without pre-existing binders. (A)-(C) AMHR2, (D)-(E) EWS::FLI1, (F) MYC, (G) DUSP12. Each panel shows the complex structure of the target with an AReUReDi-designed binder. For each binder, five property scores are provided, as well as the ipTM score from AlphaFold3 and the docking score from AutoDock VINA. Interacting residues on the target are visualized.
Refer to caption: Figures/peptide_ABCD.png

Figure: Figure S3: (A), (C) Example 2D SMILES structure of AReUReDi-designed peptide binders with four property scores for GLP1 and GLAST, respectively. (B), (D) Plots showing the mean scores for each property across the number of iterations during AReUReDi’s design of binders of length 200 for GLP1 and GLAST, respectively.
Refer to caption: x4.png

Figure: Figure S4: (A), (C) Example 2D SMILES structure of AReUReDi-designed peptide binders with four property scores for TfR and AMHR2, respectively. (B), (D) Plots showing the mean scores for each property across the number of iterations during AReUReDi’s design of binders of length 200 for TfR and AMHR2, respectively.
Refer to caption: x6.png

Figure: Algorithm 1 AReUReDi: Annealed Rectified Updates for Refining Discrete Flows

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: Please refer to the **Weaknesses** section.
soundness: 2
strengths: - The problem statement and the method are clearly written.
- The authors provide theoretical guarantees that AReUReDi preserves distributional invariance and converges to the Pareto front.
summary: The authors propose AReUReDi, an optimization algorithm to bias sampling towards Pareto optimal states while preserving distributional invariance. Experiments on peptide and SMILES sequence generation demonstrate the effectiveness of AReUReDi.
weaknesses: - The method seems to be a direct adaptation of the Metropolis-Hastings algorithm to the ReDi model. Could the authors clarify the novelty of AReUReDi?
- Ablation studies on the computational cost of AReUReDi are missing. It's helpful to show how the performance of AReUReDi scales as the computation increases.
- In Table 3, AReUReDi takes significantly longer than PepTune+DPLM. Therefore, a naive improvement of PepTune+DPLM under the same computational cost would be generating multiple samples from PepTune+DPLM (to keep the total time the same as AReUReDi) and utilizing best-of-N on top of these samples. This should offer a fair comparison between PepTune+DPLM and AReUReDi.
- The experimental results lack comparison to baseline methods. All the tables and figures, except for table 3, only present the performance of AReUReDi or its comparison with the pretrained model without any guidance. Even in table 3, only PepTune and other very traditional MOO methods are compared with. The comparison with other reward-guided sampling baselines or RL fine-tuning baselines is missing. Although these guidance-based methods are not designed for Pareto optimization, they can be naively adapted to achieve multi-objective optimization by enforcing certain weights to balance these objectives into a single aggregated objective. Also, how does the model perform compared with [1]?
- Why does the author select ReDi as the base generative model, instead of the more commonly used discrete diffusion or flow matching methods, like MDLM and discrete flow matching? A comparison of these base models' performance would be helpful.

[1] Uncertainty-Aware Multi-Objective Reinforcement Learning-Guided Diffusion Models for 3D De Novo Molecular Design. NeurIPS 2025.

---

confidence: 3
contribution: 1
initial_rating: 2
presentation: 3
questions: 1.  Could you clarify the significant discrepancy between the theoretical MH-based sampler and the greedy algorithm (using the "monotonicity constraint") applied in all experiments? Given this constraint, how should we interpret the paper's theoretical claims of convergence?

2.  The half-life score model (App E.3) was fine-tuned on only 105 samples. Given this extremely small dataset, could you provide more evidence or justification for this model's reliability? How can we be sure the strong half-life results are not an artifact of overfitting to this low-data proxy model?

3.  What was the rationale for framing the SMILES task as a refinement (starting from $p_1$ samples) rather than a full generation task (starting from $p_0$)? This is a valid task, but it is a different and less difficult problem. How does the method perform in the full generation ($p_0$ start) setting for SMILES?

4.  Could you provide results showing how the generated sequences change when the Tchebycheff weights ($\omega$) are varied? This is essential to demonstrate that the method can indeed explore the Pareto front.
soundness: 3
strengths: The paper addresses a critical and high-impact problem: the de novo design of discrete biological sequences (like peptides) that must satisfy multiple, conflicting therapeutic properties.

The core idea of integrating on-the-fly MOO guidance directly into the sampling process of a discrete flow model (ReDi) is an elegant and well-motivated approach.

The paper presents strong experimental results, particularly in Table 2, where AReUReDi shows significant improvements over the PepTune baseline on several metrics.

The paper is well-written, logically structured, and the theoretical motivation is clearly explained.
summary: This paper introduces AReUReDi (Annealed Rectified Updates for Refining Discrete Flows), a new framework designed for multi-objective optimization (MOO) in discrete sequence generation, with a focus on biomolecular design. The core idea is to extend Rectified Discrete Flows (ReDi), a generative model for discrete data, with multi-objective guidance.

The method combines three key components:
1.  Tchebycheff Scalarization: To combine multiple, conflicting objectives ($S_{\omega} (x) = \min_{n} \omega_n \tilde{s}_n(x)$) into a single scalar reward.
2.  Annealed Guidance: A simulated annealing-like schedule for the guidance strength ($\eta_t$) to balance exploration and exploitation.
3.  Locally Balanced Proposals & Metropolis-Hastings (MH) Updates: A theoretically-grounded MCMC sampling procedure designed to make the ReDi model's sampling process converge to the target multi-objective distribution ($\pi_{\eta_t, \omega} (x) \propto p_1(x) \exp(\eta_t S_{\omega} (x))$).

The authors provide theoretical guarantees that AReUReDi converges to the Pareto front with full coverage. Empirically, they demonstrate that AReUReDi can successfully generate peptide and SMILES sequences that optimize for up to five properties (e.g., affinity, solubility, half-life), outperforming evolutionary and diffusion-based baselines.
weaknesses: 1. The proposed theoretical framework is a combination of several well-established, existing components: Tchebycheff scalarization (a standard MOO technique), annealed guidance (from simulated annealing), and Metropolis-Hastings sampling (a standard MCMC method). While applying this stack to Rectified Discrete Flows is new, the conceptual contribution of this specific combination is limited.

2. There is a fundamental contradiction between the paper's theory and its experimental execution. The theory (Sec 3, App A) proposes an MCMC sampler (MH) that must accept worse-scoring states to escape local optima. However, all experiments (Sec 4) use a "monotonicity constraint," which is a greedy search that only accepts improvements.

3. This greedy search algorithm invalidates all theoretical guarantees of Pareto convergence proven in Appendix A. The paper's own ablation (Table 6) confirms this, showing the theoretical MH method ("w/o constraints") performs poorly, while the greedy version ("w/ constraints") is the one that achieves the strong results. This strongly suggests the core theoretical contribution is not practically viable as proposed.

---

confidence: 3
contribution: 2
initial_rating: 6
presentation: 3
questions: - How are the weight vectors for the Tchebycheff scalarization chosen in practice? Does the proposed method heavily rely on careful selection of these weights?
- Could the authors comment more on existing steering methods [1-4] that work for discrete diffusion/flow model? Is Tchebycheff scalarization directly applicable to these guiding methods as well?
- It is stated that "a monotonicity constraint that accepts only token updates that increase the weighted sum of the current objective score" is employed during Metropolis-Hasting. I view this as a trick to accelerate the slow convergence of Metropolis Hasting, but does this trick breaks down the theoretical guarantee of convergence at the same time?

[1] Li et al. "Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decoding", arXiv: 2408.08252.

[2] Nisonoff et al. "Unlocking guidance for discrete state-space diffusion and flow models", ICLR 2025.

[3] Chu et al. "Split Gibbs Discrete Diffusion Posterior Sampling", NeurIPS 2025.

[4] Venkatraman et al. "Amortizing intractable inference in diffusion models for vision, language, and control", NeurIPS 2024.
soundness: 3
strengths: - This paper addresses an important, underexplored problem: cleanly steering discrete flow models with multiple objectives, with clear potential for real-world impact.
- The introduced AReUReDi uses Tchebycheff scalarization as a guidance potential for sampling in discrete state spaces, automatically balancing tradeoffs and pushing toward Pareto-optimal solutions.
- Extensive experiments with promising results, demonstrating effective multi-objective control and practical applicability.
summary: This paper presents a multi-objective guidance method (AReUReDi) for discrete flow models, enabling simultaneous optimization of multiple desired properties in biomolecule sequence design. AReUReDi performs reward-tilted sampling, using a pretrained rectified discrete flow model as prior, and a Tchebycheff scalarized reward function that pushes the Pareto front of multiple objectives. The guided sampling is done using a Metropolis-Hasting algorithm, with carefully designed locally balanced proposals built upon the pretrained flow model. The capability of AReUReDi is validated on the generation of wild-type peptide sequences and chemically-modified peptide SMILES, both guided by multiple therapeutic objectives. Experimental results show the effectiveness of the proposed method in steering generation toward multiple properties, compared to established multi-objective optimization baselines.
weaknesses: - While the Metropolis-Hasting algorithm with a token flipping proposal guarantees an optimal stationary solution, the convergence speed could be slow and the sampling cost may become substantial as the dimensionality scales. A theoretical guarantee or an empirical justification regarding convergence or mixing properties of the sampler would provide more insight.
- The method is evaluated only on in-silico benchmarks, leaving open whether learned policies exploit metric proxies rather than true utility. 
- While AReUReDi is framed as sampling from the reward-tilted distribution $p_1(x)\exp(\eta S_w(x))$, the paper does not analyze whether samples remain consistent with the prior $p_1(x)$. An ablation study on using a completely uninformed prior versus using the discrete flow model $p_1(x)$ may help illustrate the necessity of a prior as reference.

---

confidence: 3
contribution: 2
initial_rating: 4
presentation: 3
questions: Please see the comments in the weakness section.
soundness: 3
strengths: The presentation is clear. The components used to derive the proposed method is clearly described. 
I am not an expert on the biological tasks considered in the experiment section of this paper, but the empirical validations in Table 2 seem to suggest the proposed approach has advantages over the included baselines.
summary: This paper considers the multi-objective reward guidance for the discrete flow models. The proposed approach has three main components: (1) Rectified Discrete Flow (ReDi) from (Yoo et al., 25) which is the unconditional transition kernel; (2) the Tchebycheff scalarization that transforms a multi-objective problem into a single-objective task; (3) the balancing function technique from [Informed proposals for local MCMC in discrete spaces, (Zanella, 17)] to formulate a locally balanced proposal to have a higher acceptance rate in the MH step. Various experiments are conducted on biological tasks.
weaknesses: The strategy proposed seems to be direct combination of existing methods. It is hard to say what is the insight one can get from this submission. 

Since there are several existing approaches for reward guidance for discrete state, e.g. DSearch [Dynamic Search for Inference-Time Alignment in Diffusion Models (Li et. al, 25)], comparing the proposed approach with these methods (the reward should be replaced by the output of the Tchebycheff scalarization) would enhance the paper.

The content in section 2.3 and the beginning of section 3.2 overlap quite a lot. No need to define the scalarized reward twice.


Decision: Reject