Property-Driven Protein Inverse Folding with Multi-Objective Preference Alignment
Authors: Junqi Liu
Xiaoyang Hou
Chence Shi
Xin Liu
Zhi Yang
Jian Tang
Venue: ICLR 2026 Poster (2026)
Keywords: protein design
preference alignment
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Protein sequence design must balance designability, defined as the ability to recover a target backbone, with multiple, often competing, developability properties such as solubility, thermostability, and expression.
Existing approaches address these properties through post hoc mutation, inference-time biasing, or retraining on property-specific subsets, yet they are target dependent and demand substantial domain expertise or careful hyperparameter tuning.
In this paper, we introduce ProtAlign, a multi-objective preference alignment framework that fine-tunes pretrained inverse folding models to satisfy diverse developability objectives while preserving structural fidelity.
ProtAlign employs a semi-online Direct Preference Optimization strategy with a flexible preference margin to mitigate conflicts among competing objectives and constructs preference pairs using in silico property predictors.
Applied to the widely used ProteinMPNN backbone, the resulting model MoMPNN enhances developability without compromising designability across tasks including sequence design for CATH 4.3 crystal structures, de novo generated backbones, and real-world binder design scenarios, making it an appealing framework for practical protein sequence design.

Published as a conference paper at ICLR 2026
PROPERTY-DRIVEN PROTEIN INVERSE FOLDING WITH PREFERENCE ALIGNMENT
Junqi Liu 1.2., Xiaoyang Hou Chence Shi Xin Liu", Zhi Yang" Jian Tang3.5.62..
School of Computer Science, Peking University 2 BioGeomeery 3 Mila Québec Al Institute Université de Montréal HEC Montréal 6 CIFAR Al Research Chair
(1 (liujunqi, yangzhi] 8pku. cn
houxiaoyangi com
jian. tang@hec.caa
ABSTRACT
Protein sequence design must balance designability, defined as the ability to re- cover target backbone, with multiple, often competing, developability properties such as solubility, thermostability. and expression. Existing approaches address these properties through post hoc mutation, inference-time biasing, or retraining property property-specific subsets, yet they are target dependent and demand substantial domain expertise or careful hyperparameter tuning. In this paper, we introduce Pro- tAlign, multi oojeti-obeective preference alignment framework that fine-tunes pretrained inverse folding models to satisfy diverse developability objectives while preserving structural fidelity. ProtAlign employs semi-online Direct Preference Optimization strategy with a flexible preference margin to mitigate conflicts among competing objectives and constructs preference pairs using in silico property predictors. Ap- plied to the widely used ProteinMPPNN backbone, the resulting model MoMPNN enhances developability without compromising designability across tasks including sequence design for CATH 4.3 crystal structures, de novo generated backbones, and wolld binder design scenarios, making it an appealing framework for practical protein sequence design.
INTRODUCTION
Inverse folding is fundamental task in protein design, spanning applications from refining sequences of natural proteins to generating sequences for de novo designed backbones (Yue & Dill 1992]
Notin al. 2024 Khakzad et al. (002).. Substantial progress has been made with models trained to accurately recover sequences compatible with a target backbone, demonstrating strong capacity to capture sequeee relationships (Gao al. 2023 Qiu et al. 2024b]] Xue et al. 22025), and post-training approaches have been explored to further improve sequence quality (Widatalla et al. 2024) Xue et al. [0255] Xu et al., 2025 Wang al., 2025). However, real-world design pipelines demand more than high sequence recovery: they typically require proteins that are both designable and developable, exhibiting properties such as solubility, thermostabillity and expression level, with additional traits depending on specific design goals (Peterson et al. 2007: Salihu & Alam 2015).
Several strategies have been explored to incorporate developability preferences into the generation process. (1) Post-hoe mutation: generate sequences with existing tools and then introduce mu- tations to improve propertics. While simple, beneficial mutations are often sparse and difficult to identify (Broom et al. 2017). (2) Inference-time biasing: adjust amino acid sampling probabili- ties (Goverde et al.] 2024h or use reward signals to guide sequence generation (Xiong et al., (025). These techniques can introduce instability and require careful hyperparameter tuning to balance property optimization with sequence quality. (3) Retraining on curated subsets: construct datasets filtered for desired properties and retrain the model to implicitly learn the bias (Goverde al. 2024
Corresponding authors.
'Tese authors contributed equally and share first authorship; each author may adjusted author order when appropriate. Work conducted during their internships BioGeooeety.

---

Published as a conference paper at ICLR 2026
Rollout
Eve ppl.
Sol
PTM
Samples from (gla))
F22
F3
D
MLFVLVTGEL
ERK L TIGE
16.43
0.433
0.203
D2
taund)
Backbone
M G V V V
15,48
6479
0.290
1a22.A
LLTVLVE VOGEE
D
faind
Multl-Objective Training
Optimia Current Polity T
D
7'2
Polyu (z), Prar(yulz)
D2
Sample
Win 0.712
MLTVVVOGEE
folsnn (x), firr(tnn
Esso
Lose 0.478 18.46 0.260
3R MGRVLVTGFE
Estimated Probabiltyy Referenoa Policy
D3
Adapiive
Margn
Semi-Online Evolve
Pollcy
Rollout
D
D2
D3
Training
Initiatine Cumeent Palicy
Figure 1: The ProtAlign framework. ProtAlign optimizes the policy model in a semi-online regime composed of alternating rollout and training stages. In the rollout stage, protein backbones are sampled from training set, and the current policy model generates rollouts at a higher temperature. These rollouts evaluated with property predictors, and pairwise preference datasets constructed for each property. During training. pairwise entries are drawn evenly across the datasets, and an adaptive preference margin is introduced to resolve conflicts among multiple objectives.
Ertelt et al. 2024). Although such methods have achieved wet-lab validated success, they rely on carefully curated datasets and are difficult to generalize across diverse design objectives.
To address these challenges, we introduce ProtAlign, an optimization framework that aligns pretrained inverse folding models with both designability and diverse developability objectives. ProtAlign em- ploys a novel semi-online Direct Preference Optimization algorithm with a flexible preference margin to balance competing goals (Figure D This strategy enables robust optimization for developability without sacrificing sequence-structie fidelity, even though developability metrics do not directly capture sequence-strucce
consistency. Training data are generated by annotating sequences with suite of property predictors and forming pairwise preference sets for each property. During training, pairs are sampled evenly across properties, and the flexible margin in the DPO loss helps reconcile conflicting optimization directions. The overall training proceeds in a semi semi-online manner through iterative rollout, annotation, and updating, which avoids running property predictor models during training. thereby significantly reducing computational cost.
We instantiate our framework on ProteinMPNNN (Dauparas et 2022 one the most widely used inverse folding models, training it on the commonly adopted CATH dataset (Sillitoe et al. 2021 ) to obtain MoMPNN. We evaluate two key developability properties, i.e., solubility and thermostability. and show that MoMPNN outperforms subset-traing baselines such SolubleMPNN (Goverde et al., 2024) and HyperMPNN (Ertelt et al., 2024). which are specifically designed for these properties. We comprehensively assess MoMPNN across redesigning sequences for crystal structures in the CATH 4.3 test set, designing sequences for de novo generated backbones, and applications to realistic binder design scenarios. MoMPNN shows superior performance across these diverse evaluation tasks, demonstrating the broad applicability and effectiveness of our alignment framework.
Our main contributions
2

---

Published as a conference paper at ICLR 2026
We propose a multi-objective alignment framework, ProtAlign, for optimizing protein inverse folding models towards arbitrary desired developability properties without compromising designability with semi-online multiple-objective preference optimization.
Applying ProtAlign to ProteinMPNN, our resulting model MoMPNN achieves significant improvement on developability properties, outperforming existing baselines across crystal, de novo and real-scenario benchmarks.
By adding de novo benchmarks and incorporating developability metrics into inverse folding evaluation, we offer a systematic framework for assessing model performance beyond recovery, thereby opening new avenues for future research.
2 RELATED WORK
Protein Inverse Folding, Protein inverse folding is to generate a protein's amino acid sequences given its structure. Early work like GraphTrans (Carscadden et al.
2021), StructGNN (Chou et al.
2024) and Geometric Vector Perceptrons (GVPs) (Jing et al. (0200) utilize the graph neural network to design protein sequences. And, ProteinMPNN (Dauparas et
2022) extends GraphTrans by
introducing more geometry features and random decoding. ESM-IF
(Hsu et al. [2022) trains a large-
scale inverse folding framework based. PiFold (Gao et al.) 2022) accelerates sequence generation
with one-shot predicting strategy. FMIF (Nisonoff et al. 20024) explores applying flow matching to
inverse folding. Additionally, LM-Design (Zheng et al. 2023), CarbonDesign (Ren et al. 2024) and InstructPLM (Qiu et al. 2024b) utilize protein language models in sequence design. There are also
emerging approaches that move beyond traditional inverse folding (Song et al. 2024 Tang et al.; Wu et al.) design functional sequences by considering not only the backbone strueture but also the broader biochemical context of the protein. While ProteinMPNN still the most widely used and wet-lab-verified model, we chose it as our backbone model during evaluation.
Preference Optimization. Recently, many methods to align LLMs with human feedback have emerged. To better align these models with human preferences, these methods can be categorized into two classes; online and offline methods. Online methods (Shao et al.) 2024), such as PPO (Schulman al. 2017), are typically employed optimize policy models through direct reward optimization.
While effective, online RL methods are computationally intensive and potentially unstable (Gupta al. (222).. Recently, offline methods DPO (Rafailov et al.) (023) and its variants (Azar et al.) [002]] Meng et al. 2024) have been introduced. They can align models to pairwise preferences rather than an explicit reward model directly. To address the possible overfitting and collapse problems of offline
DPO (Guo et al. 2024a), several online or semi-online variants (Guo et al. [024a Calandriello et al.) 2024 Lanchantin et al. 2025) have been developed. Inspired by these works, we design a semi-online approach combining the benefits of self-evolving and computational efficiency by separating the rollout phase from training.
Preference optimization has been used inverse folding models to improve their performance. ProteinDPO (Widatalla et al. 2024) leverages an experimentally derived stability preference dataset to enhance the stability of designed sequences. ResiDPO (Xue al. 2025) uses residue-level labeled dataset to improve sequence designability. InstructPLM-DDPP (Xu et al. 2025) fine-tuned on TTM-Score-constructe dataset (Zhang & Skolnick 2005) better align sequence outputs with target structures, ProteinZero (Wang et al.) (2025) directly defines a composite reward function combining TM-Score and energy, and optimizes through online GRPO. These works mainly focus on improving designability and cannot extend to developability properties that may conflict with designability.
Multi-Objective Preference Alignment The multi- multi-diimensoonal nature of human preferences 2018 Bai et al. (202)) motivates various methods for handling multiple objectives (Wang et al. 2024). Early work explored parameter-merging approaches like rewarded soups (Jang
(Vamplew al.
ct al. 2023 Lin et al., 2023 Rame et al. 2023 Wortsman et al. 2022), preference-condittonee
prompting was introduced (Zhu et al. 2023 Basaklar al. 2022 Guo et al. 2024b] Yang et al.)
2024), enabling direct control over preference weightings (Zhou et al. 2023b). Beyond inference-ting
techniques, retraining- based strategies have emerged as a promising direction: MORLHF (Wu et al.
2023 Bai et al. 2022) incorporates multiple rewards through scalarization, and MODPO (Zhou et al.
2023a) integrates multi-objective optimization directly into reward learning, theoretically matching
3

---

Published as a conference paper at ICLR 2026
MORLHF yet practically offering greater stability and efficiency. We leverage similar techniques when optimizing inverse folding models towards multiple objectives.
PRELIMINARIES
ProteinMPNN. ProteinMPNN (Dauparas et al. 2022) aims to model the conditional distribution P(seq x) and generate sequences that are compatible with the target structure given a backbone structure It is an order-agnostic autoregressive model that generates sequences conditioned on given backbone structure. For a backbone x with I. residues, the probability of sequence
y "" ,LL is factorized as:
T x.
II (o((o(t) a (a(<i))
(I)
where a is random permutation of residue indices that enforces order invariance.
The model is trained structure-scquence pairs from the Protein Data Bank (PDB) using cross- entropy loss with teacher forcing. At each step, a random permutation a is sampled, and the loss is computed
(C) (a E
log no (Vo()) x No(<s)
Direct Preference Optimization. Direct Preference Optimization (DPO) (Rafailov al. 2023 is recent framework for aligning generative models with human or task-specific preferences without requiring explicit reward modeling. The key idea is to learn from pairwise preference data: given context æ (e.g., a protein backbone) and two candidate outputs (preferred) and (less preferred), the preference model assumes
P( > y 12) ((((w))
where r(-) an implicit reward function and f is typically modeled as a logistic function over reward differences. Instead of explicitly estimating from pairwise data, DPO derives tractable training loss by enforcing that the conditional likelihood ratio between the fine-tuned policy no and the reference policy Nre matches the observed preference:
(0) = E.(......))) log o
( (3
Mart (o)) 3 log
Te le)
ma (x) Mar(an (2)
where a is the sigmoid function and 8 controls the strength of preference alignment. This objective directly optimizes the model to prefer over while regularizing towards the reference model.
METHOD
In this section, present our multi-objective preference alignment framework ProtAlign. Firstly, we describe our method of multi-objective optimization protein sequence design tasks in Section 4.2 Then, outline semi-online multi-objective training strategy for efficient exploration in Section Finally, Section 4.4 introduces the construction of preference pairs.
4.1 NOTATIONS
Let æ denote the input backbone structure, y a protein sequence, and 0 the model parameters. We write for the sequence conditional distribution induced by 0, and Tief for the reference model obtained prior to gost-training. A superscript t indicates variables in the t-th stage of semi-online training, such n² or y². Let K be the number of properties, and let {Mx (æ, y) - R} denote the corresponding in silico predictors. Finally, let = [D). ... Dxx represent the pairwise datasets for each property, generated by the respective M₁-
4

---

Published as a conference paper at ICLR 2026
4.2 MULTI-OBIECTIVE OPTIMIZATION WITH FLEXIBLE PREFERENCE MARGIN
Our optimization process aligns a pretrained inverse folding model based on series of pairwise datasets {Dx} for each target property k, annotated by in silico predictors Mr (Section 4.4). The ulti- mate goal is to simultancously improve the model's 's performance on all properties while maintaining limited divergence from the original model. Formally, we maximize the following objective:
Σ ww wark (z. w)
arg max ((TO) BEEDD 8
where D is the distribution of possible protein backbones, rk is the implicit reward function from Dj and wy is an adjustable weight.
As in the original Direct Preference Optimization (DPO) derivation, assume the preference relations follow the Bradley-Terry model:
exp(r(x,)1))
pu > y2 x) exp(r(x,81)) + exp(p(xx,)))) oxp(rer = (r((,, vn) r(x,, 32))
(3)
We integrate the multi-property policy objective in Eq. with the pairwise preference model Eq. (details in the Appendix [B.T) (Zhou et al., 2023a), we derive flexible-margin Direct Preference Optimization (DPO) loss, denoted (0: ; D₂). explicitly accounts for both multi-property rewards and adaptive preference margins. The intuition of adaptive preference margins is that if yo performs worse than Y/ on some auxiliary property, the required margin for this pair should be reduced, preventing conflicting optimization from overemphasizing single property at the cost of others.
xa((aal)) LMo = =E (r.Ne.)))DD log o (3log no (xwl=) - 3 Blog mo(wi (x) m& (Bw: (ol)ni)))))))
my(yw: 91)
Σ kepk
Wy (rw (2,, (Ir vw) Tw (x,
(4)
During training, we sample entries from the pairwise datasets Da evenly. The adaptive margin m(yw: 31 is precomputed before training with our property predictors.
Next, we describe the definition of ProteinMPNN's probability term To in the loss function LMO- Unlike LLMs, ProteinMPNN is not left-to-right model but order-agnostic autoregressive model. While Ty can be easily calculated for left-to-right causal models, its exact estimation for order- agnostic models inherently requires extensive sampling across different decoding orders. We adopt an efficient approach for estimating the log-ratio in our loss function inspired by recent works on discrete diffusion-barn based LLMs (Zhu al. (2025). The probabilities To and Tee of an order-agnostic inverse folding model by sampling multiple random residue orders. Crucially, both models are evaluated under the same sampled orders:
poly x))
Σ Toly IK) Prer(y xx)
K Teely
x, ok),
(5)
where { are the same sampled permutations. This shared-order evaluation significantly reduces the variance of the estimated log-ratio, leading to more stable optimization.
4.3 SEMI-ONLINE TRAINING FOR EFFICIENT EXPLORATION
It is widely acknowledged that online exploration plays important role in RL alignment (Tang et al., 2024). However, such training regime requires significant resource and infrastructure engineering for rollout and evaluation during training. Fortunately, semi-online training has been shown to
5

---

Published as a conference paper at ICLR 2026
be as effective pure online training (Lanchantin et al. 2025). Thus, build a semi-online DPO framework to decouple rollout and evaluation from training which allows for efficient batch computation and is easy deploy.
As detailed in Algorithm the semi-online DPO framework proceeds in an iterative manner. At each iteration the current policy zá first generates rollout sequences under rollout temperature which deliberately set higher than the evaluation temperature in order to promote diversity. These rollouts subsequently evaluated by K property predictors, from which pairwise preference datasets {D&} are constructed. The model is then optimized on the newly generated preference data for several steps. yielding the updated policy me Overall, this paradigm integrates the advantages of both online and offline learning: the model alternates between online data generation and update across iterations, while the optimization within each iteration performed in an offline mode. In addition, our approach requires no modification to the property predictors, ensuring strong compatibility with existing methods; it allows batch inference maximize resource utilization; and each predictor can fully exploit the available computational capacity without introducing additional overhead.
Algorithm Iterative Training Algorithm for
DPO
1: Input: Base model To. Preference predictors {M1 M2,,M2,.... ..b.....MM''. Preference weights [Wi, W2....: Wk] J. Backbone dataset X, Number of iterations T' Number of designed sequences per backbone n, Sampling temperature т, Number of sampling backbones at each iteration N.
2: Initialize model T Tg.
3: for t = 1 to 7 do
4: for = to do
5: Sample backbone x <- Sample(X').
6: Generate n sequences per backbone = { 1 ~ To(. x.
7: Use reward models (M1, M2, Mx] } to calculate rewards for each sample.
8: for 1: = 1 to do
9:
Construct preference pairs for each reward Dj (Vos 9.)).
10: end for
end for
12: Update model parameters 6 g -p-11-as ao (EK] wA Cmo (x:)))
13:
end for
14:
Output: Final optimized model Ty
4.4 CONSTRUCTION OF PREFERENCE DATASETS
We leverage existing protein property predictors as proxy annotators to provide pairwise preferences, building separate datasets for each property k. Given a backbone with N candidate sequences, each sequence is scored with (y) and ranked accordingly. Following Xu et al. 2025). the i-th ranked sequence is paired with the (N/2 + 2)-th ranked sequence (i < N/2), denoted Vh and yt- A pair (yw. y1) is included in the dataset Dx only if the score gap satisfies M4 (y)) - M& > 8g, where dg property-specio threshold. This procedure filters out ambiguous comparisons and yields consistent annotations from which DPO can learn implicit reward signals.
To capture diverse aspects of protein design, we categorize properties into two classes. Designability properties measure structural consistency between designed sequences and the input backbone, such as TM-score between a predicted structure and the target backbone or confidence metries reported by structure prediction models. These metrics reflect the fundamental ability of an inverse folding model. Developability properties, in contrast, do not directly compare the designed sequence to the backbone and are primarily concerned with whether the protein sequence can achieve the intended purpose. We consider two main types: (1) General quality metrics, assessed for example by pseudo-likelihen scores from protein language models such as ESM (Lin et al.) 2022), which correlate with evolutionary plausibility and often predict downstream outcomes such as solubility or expression (Adaptyv Bio 2024); and (2) Targeted quality metrics, which capture properties directly related to whether the designed sequence can fulfill desired purposes, such as solubility and thermostability. These are important for practical use and typically approximated by in silico predictors given the expense of wet-lab assays. Since developability properties does not consider the consistency between sequence and input structure, jointly optimize developability properties

---

Published as a conference paper at ICLR 2026
together with designability properties in our multi-objectiin alignment framework ensure structural consistency while optimizing for desired developability.
5 EXPERIMENTS
We evaluate our framework by fine-tuning the widely adopted ProteinMPNN model toward two critical functional properties, solubility and thermostability, thereby deriving the MOMPNN models. Section presents the details of model training and our evaluation setup. In Section 5.2 we assess MoMPNN's ability to redesign sequences for crystal structures in the CATH4.3 test set. Section 5.3 extends the evaluation to more practical setting on de novo generated protein backbones. Finally, Section focuses on a more specific application, evaluating MoMPNN's performance in designing sequences for de novo binders against set of challenging protein targets.
5.1 EXPERIMENTAL SETUP
Training and Testing. Our model is trained on the CATH 4.3 training set (Orengo et al. 1997) based on the train-test-valideen velt-valio split referenced in (Hsu et al., 2022). During training, we generate eight sequences at a temperature of 1.0 for each sampled structure encourage diversity. We use a temperature of 0.1 1 during evaluation for ProteinMPNN-rellan
models, while other baselines are evaluated at their recommended temperature. More details on the preparation of the training and testing datasets are provided in the Appendix C
Baseline Methods. We compare our model against representative methods from three categories: (1) sfate-of-the-art inverse folding models, including ProteinMPNN (Dauparas et al. [022), ESM-IF (ssu et al.] 2022); ) and InstructPLM (Qiu et al., 2024a): (2) RL-based DPO method PPO Widatalla et al. (2024); (3) task-specific models trained protein subsets, SolubleMPNN (Goverde) ct al. 2024) for solubility and HyperMPNN (Ertelt et al. 2024) for thermostabbility; and (4) guidance-
conditional models to guide the original ProteinMPNN model, following the approach of Nisonoff et al. (2024), resulting in
based methods, where
SolubleMPNN and HyperMPNN
Guidance[Sol] and Guidance[Thermo)] Additional details are provided in the Appendix B.3
Property Predictors. We employ several computational predictors as in silico proxies for protein properties. For designability, we use the TM-score (TM), computed between ESMMooddpreddcteed (Lin et al. 2022) structures and reference structures, or alternatively the pTM score from AlphaFold2 (Jumper et al. (2021) using the Initial Guess (IG) (Bennett al.) 2023) approach. For developability, we adopt Protein-Sol Sol (Hebditch et al. 2017) as a widely used proxy for solubility (Sol), TemBERTure (Rodella et al. 2024), a model trained on large-scale datasets, as a proxy for thermostability (Thermo), and the pseudo- likelloooo score from the ESM-2 model (Lin et al., (022) (Evolutionary Perplexity. EP) as an indicator of sequence quality. In our experiments, we systematically compare the benefits of training with different combinations of these properties. Additional details are provided in the Appendix B.4.
5.2 SEQUENCE REDESIGN FOR CATH4.3 CRYSTAL STRUCTURES
We first evaluate our model on the CATH4.3 test set, a benchmark dataset commonly used for protein inverse folding models. CATH4.3 is classification dataset of observed crystal structures, and performing inverse folding on these protein backbones reflects a model's ability to redesign sequences based on experimentally determined crystal structures. It is also worth noting that some ground-truth -tuth sequences in CATH are not soluble, and most of them are not thermally stable, so the solubility and thermostability of ground-truth sequences are low.
As shown in Table I our MoMPNN preserves the designability level of ProteinMPNN while sig- nificantly enhancing developability, achieving the best solubility and thermostability by explicitly optimizing for multiple desired properties. ProteinMPNN and other inverse folding baselines show high designability and moderate developability. SolubleMPNN achieves strong solubility and main- tains reasonable structural quality. HyperMPNN could reach high thermostability but suffers from degraded designability, likely due its smaller training data. Guidance-based methods partially address the trade-off by striking a balance between the property gains of subset-trained models and the
InsttrucPP.. M trained the 4.2 version of the CATH dataset.
7

---

Published as a conference paper at ICLR 2026
designability of ProteinMPNN. We also find that higher amino acid recovery does not correlate with higher designability and developability, and a practical perspective, we focus more on designability and developability rather than the amino acid recovery.
We further analyze the results to understand how different objectives shape model behavior. TM leads higher TM--scores and thus slightly stronger structural consistency, while IG consistently yields lower evolutionary perplexity, since evaluates not only whether a sequence can refold but also how confident AlphaFold is in that prediction. In addition, directly incorporating EP does substantially improve evolutionary plausibility, but consistently enhances non-targeted metrics, serving as useful regularizer that complements both TM and IG. To assess whether MoMPNN effectively captures the underlying patterns of protein solubility and thermal stability, we conducted an in-depth statistical analysis of generation sequences in Appendix A.T We also provide the early
results comparing multi-ohjective optimizing strategies in Appendix A.2
5.3 SEQUENCE DESIGN FOR de novo GENERATED BACKBONES
We next extend our evaluation to setting that more closely reflects practical protein design workflows, where we generate sequences for de novo backbones produced by RFDiffusion. We designed unconditional backbones for each length in the range of [50, 500] with RFDiffusion the input of the inverse folding models. This represents a more common application scenario for inverse folding models, serving a tool identify suitable sequences for newly designed protein backbones. Details of this de novo benchmark set are provided in Appendix
As shown in Table MoMPNN demonstrates the strongest overall performance in this setting, even surpassing ProteinMP in structural consistency. IF and InstructPLM exhibit a substantial performance drop under de novo conditions, consistent with previous reports (Ren et al. 2024), whereas ProteinMPNN retains performance levels similar to those observed on crystal structures. SolubleMPNN achieves markedly better structural consistency than ProteinMPNN, but our models consistently outperform this bascline. In terms of training objectives, observe that IG-based opti- mization yields higher structural consistency than TM in the de novo setting, while other phenomena remain consistent with the observations from CATH4.3.
Table 1:
Comparison of protein sequence design methods on the CATH 4.3 test across various metrics. Results for our RL-based MoMPNN trained with different annotator combinations (TM, IG. EP, Sol, and Thermo) are shown. The best and second- best values are highlighted bold and underlined, respectively.
Designability Metries
Developability Metries
Method
AAR
RMSD
TM score
PLDDT 1
EP
Solf
Thermo 1
Test Dataset
3.97
0.761
80.8
5.80
0.620
0.246
1.000
ESM-IF
0.737
78.4
6.11
0.733
0.719
0.464
InstruciPLM LM default)
0.653
0.396
0.574
InstructPLM (T=0.1)
6.96
0.632
74.4
7.31
0.657
0.455
0.584
ProteinMPNN
4,30
0.740
79.1
6.70
0.719
0.769
0.389
ProtcinDPO
5.49
0.667
72.0
10.50
0.629
0.357
0.388
SolubleMPNN Guidance [Sol]
4.48
0.733
78.8
6.54
0.794
0.815
0.382
4.33
0.740
79.4
6.40
0.762
0.805
0.393
McMPNN [Sol+TM]
4.37
0.738
79.3
6,27
0.884
0.747
0.384
MoMPNN [SoI+TM-EP)
4.38
0.739
79.5
6.18
0.852
0.790
0.387
MoMPNN [So++G]]
0.727
0.883
0.751
0.382
MoMPNN [Sol+IG+EB]]
4.61
0.731
79.3
$.99
0.856
0.789
0.384
HyperMPNN Guidance [Thermo]
4.90
0.706
7.81
0.719
0.929
0.359
4.30
0.737
77.6
6.88
0.735
0.901
0.386
MoMPNN [Thermo+TM
4,30
0.739
78.4
6.24
0.704
0.947
0.386
MoMPNN [Thermo+TM+BBB TM+EP)
4.30
0.742
78.6
6.12
0.731
0.946
0.387
MoMPNN [ThrrmoIG]
4,38
0.734
78.2
5.85
0.694
0.963
0.382
MoMPNN [Thermo+IG-EB
4.37
0.737
78.5
5.97
0.723
0.947
0.385
8

---

Published as a conference paper at ICLR 2026
Table 2:
Comparison of protein sequence design methods across different evaluation metrics on de novo backbone structures from RFDiffusion. The best results and the second-best results are marked bold and underlined. Note that AAR is not evaluated this setting, as alignment reference sequences is not applicable in de novo design.
Designability Metries
Developability Metries
Method
RMSD +
TM score
PLDDT
EP +
Sol
Thermo
ESM-IF
13.51
0.461
57.6
7.27
0.616
0.806
InstruciPLMM LM (default)
22.44
0.134
32.6
3.33
0.539
0.278
InstructPLM (T=0.1)
22.58
0.132
33.5
0.538
0,367
ProteinMPNN
6.86
0.718
70.0
8.32
0.731
0,978
ProteinDPO
16.77
0.296
43.5
14.70
0.596
0.145
SolubleMPNN Guidance [Sol]
0.733
70.5
0.799
0,992
6.20
0.748
71.8
8.14
0.774
0.989
MoMPNN [So[+TM]
70,9
0,987
MoMPNN [SoI-TM-EP)
6.40
0.742
71.3
7.47
0.843
0.993
MoMPNN [Sol+IG]
6.37
0.742
71.5
7.21
0.867
0.983
MoMPNN [So++IG+EP)
6.17
0.751
72.0
7.34
0.843
0.993
HyperMPNN Guidance [TThrrmo]
7.51
0.693
68.0
8.25
0.727
0.992
6.34
0.743
71.5
7.88
0.757
0.993
MoMPNN [ThermoTTM]
6.29
0.744
70.8
7.75
0.704
0.997
MoMPNN [Thermo+TMM44
6.48
0.737
70.5
7.64
0.736
0.999
MoMPNN I'Termo+
71.1
0.684
0.998
MoMPNN [Thermo-IG-EP)
6.20
0.748
71.2
7.44
0.723
0.998
5.4 SEQUENCE DESIGN FOR de novo BINDERS
In this section, we extend our evaluation to specific design tasks, in which inverse folding models are used design sequences for de novo binders generated by RFDiffusion. These binders target set of challenging proteins, providing practical assessment of the model's potential to support real-world applications. The inverse folding models gets 100 de novo backbones are designed for each binder as inputs and are required to generate 8 sequences for each backbone. A sequence is considered success binder sequence pLDDT > 80, inter-chain PAE < 10, and overall Ca RMSD <
Backbone
Succos Rate
3:20
Ad7
SCAR0
Perplexity
Solubilitty
Se Shrsaaiisk
10
ESA
Figure 2: The result for ProteinMPNN, SolubleMPNN and MoMPNN on the binder design bench- mark.

---

Published as a conference paper at ICLR 2026
2 A. A backbone is considered success if the model generates least one success sequence for it. Details of the evaluation pipeline are provided in Appendix C.3
our soluble variant MoMPNN [SoI+IG+EP] exhibits slightly higher success rates in both sequence and backbone than ProteinMPNN. It also achieves substantial performance gains over ProteinMPNN across two developability properties: evolutionary plausibility and solubility. Besides, MoMPNN performs on par with SolubleMPNN on designability, with better evolutionary plausibility and solubility. These results collectively demonstrate that MoMPNN preserves the essential capacity for binder design despite being post-trained only on monomeric inputs, and that the improvements developability translate into complex settings without sacrificing designability.
As illustrated in Figure 2
CONCLUSION
We presented ProtAlign, multi-objective alignment framework that extends inverse folding models beyond sequence recovery to jointly optimize for designability and diverse developability properties. By introducing semi-online Direct Preference Optimization algorithm with flexible preference margin, our approach achieves robust improvements in solubility and thermostability without compro- mising sequence-structtur structure fidelity. Applied to ProteinMPNN, the resulting MoMPNN consistently outperforms subset-trained baselines across crystal, de novo, and real-world design tasks, highlighting effectiveness and generality of our framework for practical protein engineering.
The limitations of MoMPNN primarily include the following two aspects. First, while our experiments have validated the model's effectiveness through various metrics, wet-lab experimental verification is still lacking. Second, this study primarily focuses protein monomer properties; although testing was conducted on binders, no exploration was performed on complex-speciff properties, which we will further investigate in subsequent work.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENT
We thank Tian Zhu for helpful discussions and suggestions, and Martin Pacesa for providing the BindCraft target files.
REPRODUCIBILITY STATEMENT
We provide a detailed description of our algorithm in the Method section, ensuring that all steps of the approach are clearly explained. The hyperparameters used in our experiments are listed in Appendix [B.2] We will release the source code and checkpoints in httpp://githun /ht..h...t.gggggggg com/ Qivon McMPNN
REFERENCES
Adaptyv Bio. Protein optimization 102: Lessons from the protein design competition. https October 2024.
Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. A general theoretical paradigm to understand learning from human preferences. International Conference on Artificial Intelligence and Statistics, pp. 4447-4455. PMLR, 2024.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.0500 256622 2022.
Toygun Basaklar, Suat Gumussoy, and Umit Y Ogras. Pd-morl: Preference-driven multi-objective reinforcement learning algorithm. arXiv preprint arXiv:2208.074 001.22008.07944 2022.
Nathaniel R Bennett, Brian Coventry, Inna Goreshnik, Buwei Huang, Aza Allen, Dionne Vafeados, Ying Po Peng, Justas Dauparas, Minkyung Baek, Lance Stewart, et al. Improving de novo protein binder design with deep learning. Nature Communications, 14(1):2625, 2023.
Aron Broom, Zachary Jacobi, Kyle Trainor, and Elizabeth M Meiering. Computational tools help improve protein stability but with solubility tradeoff. Journal of Biological Chemistry, 292(35): 11349-14336 2017.
Daniele Calandriello, Daniel Guo, Remi Munos, Mark Rowland, Yunhao Tang, Bernardo Avila Pires, Pierre Harvey Richemond, Charline Le Lan, Michal Valko, Tianqi Liu, et al. Human alignment of large language models through online preference optimisation. arXiv preprint 24Xv::2403.88635, 2024.
Henry L Carscadden, Lucas Machi, Chris J Kuhlman, Dustin Machi, and SS Ravi. Graphtrans: a software system for network conversions for simulation, structural analysis, and graph operations. In 2021 Winter simulation conference (WSC), pp. 1-12. IEEE, 2021.
Yuan-Tung Chou, Wei-Tze Chang. Jimmy G Jean, Kai-Hung Chang, Yin-Nan Huang, and Chuin- Shan Chen. Structgan: an efficient graph neural network framework for static structural analysis. Computers & Structures, 299:107385, 2024.
Justas Dauparas, Ivan Anishchenko, Nathaniel Bennett, Hua Bai, Robert Ragotte, Lukas F Milles, Basile IM Wicky, Alexis Courbet, Rob de Haas, Neville Bethel, et al. Robust deep learning-bass protein sequence design using proteinmpnn. Science, (7866115):9-56, 2022.
Moritz Ertelt, Phillip Schlegel, Max Beining, Leonard Kaysser, Jens Meiler, and Clara T Schoeder. Hypermpnn-a general strategy to design thermostable proteins learned from hyperthermophiles. bioRxiv. 2024.
Zhangyang Gao, Cheng Tan, Pablo Chacón, and Stan Li. Pifold: Toward effective and efficient protein inverse folding, arXiv preprint a209 12643, 2022.

---

Published as a conference paper at ICLR 2026
Zhangyang Gao, Cheng Tan, Yijie Zhang, Xingran Chen, Lirong Wu, and Stan Li. Proteininvbench: Benchmarking protein inverse folding on diverse tasks, models, and metrics. Advances in Neural Information Processing Systems, 36:68207-68220, 2023.
Casper Goverde, Martin Pacesa, Nicolas Goldbach, Lars Dornfeld, Petra EM Balbi, Sandrine Georgeon, Stéphane Rosset, Srajan Kapoor, Jagrity Choudhury, Justas Dauparas, et al. Computa- tional design of soluble and functional membrane protein analogues. Nature, 63 (81(800))4449-455 2024.
Shangmin Guo, Biao Zhang, Tianlin Liu, Tianqi Liu, Misha Khalman, Felipe Llinares, Alexandre Rame, Thomas Mesnard, Yao Zhao, Bilal Piot, al. Direct language model alignment from online ai feedback. arXiv preprint arXiv:2402.04792 2024a.
Yiju Guo, Ganqu Cui, Lifan Yuan, Ning Ding, Zexu Sun, Bowen Sun, Huimin Chen, Ruobing Xie, Jie Zhou, Yankai Lin, et al. Controllable preference optimization: Toward controllable multi-objective alignment. arXiv preprint arXiv:2402.1 (9000.//9080 /9085, 2024b.
Raghav Gupta, Ryan Sullivan, Yunxuan Li, Samrat Phatale, and Abhinav Rastogi. Robust multi- objective preference alignment with online dpo. In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pp. 27321 2025.
Max Hebditch, M Alejandro Carballo-Amatee Spyros Charonis, Robin Curtis, and Jim Warwicker. Protein-sol: a web tool for predicting protein solubility from sequence. Bioinformatics, 33(19): 3098-3100, 2017.
Chloe Hsu, Robert Verkuil, Jason Liu, Zeming Lin, Brian Hie, Tom Sercu, Adam Lerer, and Alexander Rives. Learning inverse folding from millions of predicted structures. In International conference on machine learning, pp. 8946-8970. PMLR, 2022.
Joel Jang, Seungone Kim, Bill Yuchen Lin, Yizhong Wang, Jack Hessel, Luke Zettlemoyer, Hannaneh Hajishirzi, Yejin Choi, and Prithviraj Ammanabrolu. Personalized soups: Personalized large language model alignment via post-hoc parameter merging, arXiv preprint arXiv:23100 11.3311011614 11564, 2023.
Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael JL Townshend, and Ron Dror. Learning from protein structure with geometric vector perceptrons. arXiv preprint w:Xiv:2009.01411 2020.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvuakkool, Russ Bates, Augustin Zidek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. nature, 596(7873):583-588) 2021.
Pranav Kantroo, Günner Wagner, and Benjamin B Machta. Pseudo-perplexity in one fell swoop for protein fitness estimation. PRX Life, 3(3)0033014. 2025.
Hamed Khakzad, Ilia Igashov, Arne Schneuing, Casper Goverde, Michael Bronstein, and Bruno Correia. A new age protein design empowered by deep learning. Cell Systems, 14(11):925-939 2023.
Jack Lanchantin, Angelica Chen, Janice Lan, Xian Li, Swarnadeep Saha, Tianlu Wang. Jing Xu, Ping Yu, Weizhe Yuan, Jason E Weston, et al. Bridging offline and online reinforcement learning for 1lms. arXiv preprint XXV::5506.2149. 2025.
Yong Lin, Hangyu Lin, Wei Xiong, Shizhe Diao, Jianmeng Liu, Jipeng Zhang, Rui Pan, Haoxiang Wang, Wenbin Hu, Hanning Zhang, al. Mitigating the alignment tax of rlhf. arXiv preprini 0rXiv:2309.06256. 2023.
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Allan dos Santos Costa, Maryam Faral-Zarandi, Tom Sercu, Sal Candido, et al. Language models of protein sequences the scale of evolution enable accurate structure prediction. bioRxiv, 2022.
Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference- free reward. Advances in Neural Information Processing Systems, 37:124198-1244 2024.
12

---

Published as a conference paper at ICLR 2026
Hunter Nisonoff, Junhao Xiong, Stephan Allenspach, and Jennifer Listgarten. Unlocking guidance for discrete state-space diffusion and flow models. arXiv preprint arXiv:2406.01572, 2024.
Pascal Notin, Nathan Rollins, Yarin Gal, Chris Sander, and Debora Marks. Machine learning for functional protein design. Nature biotechnology, 42(2):211-22) 228, 2024.
Christine Orengo, Alex Michie, Susan Jones, David T Jones, Mark B Swindells, and Janet M Thornton. Cath-a hierarchic classification of protein domain structures. Structure, 5(8):1093-1109, 1997.
Martin Pacesa, Lennart Nickel, Christian Schellhaas, Joseph Schmidt, Ekaterina Pyatova, Lucas Kissling, Patrick Barendse, Jagrity Choudhury, Srajan Kapoor, Ana Alcaraz-Serna, et al. Bindcraft: one-shot design of functional protein binders. bioRriv, pp. 2024-09, 2024.
Michelle E Peterson, Roy M Daniel, Michael J Danson, and Robert Eisenthal. The dependence of enzyme activity on temperature: determination and validation of parameters. Biochemical Journal, 402(2):331-337, 2007.
Jiezhong Qiu, Junde Xu, Jie Hu, Hanqun Cao, Liya Hou, Zijun Gao, Xinyi Zhou, Anni Li, Xiujuan Li, Bin Cui, Fei Yang. Shuang Peng, Ning Sun, Fangyu Wang, Aimin Pan, Jie Tang, Jieping Ye, Junyang Lin, Jin Tang, Xingxu Huang, Pheng Ann Heng, and Guangyong Chen. Instructplm: Aligning protein language models follow protein structure instructions. bioRxiv, 2024a. doi: 10. 1101/2024.04411.588664 URL hwtp://..ww......./////otttt//e. http://otttttttallr/caaaaaaa 04/20/2024. 17.56444... 589642
www.
Jiezhong Qiu, Junde Xu, Jie Hu, Hanqun Cao, Liya Hou, Zijun Gao, Xinyi Zhou, Anni Li, Xiujuan Li, Bin Cui, et al. Instructplm: Aligning protein language models to follow protein structure instructions. bioRxiv, pp. 2024 04, 2024b.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728-53744 2023.
Alexandre Rame, Guillaume Couairon, Corentin Dancette, Baan-Baptiste Gaya, Mustafa Shukor, Laure Soulier, and Matthieu Cord. Rewarded soups: towards pareto-optimal alignment by interpo- lating weights fine-tuned on diverse rewards. Advances in Neural Information Processing Systems, 36:71095-711 134, 2023.
Milong Ren, Chungong Yu, Dongbo Bu, and Haicang Zhang. Accurate and robust protein sequence design with carbondesign. Nature Machine Intelligence, (55):536-547, 2024.
Chiara Rodella, Symela Lazaridi, and Thomas Lemmin. Temberture: advancing protein thermosta- bility prediction with deep learning and attention mechanisms. Bioinformatics Advances, 4(1): vbae103, 2024.
Aliyu Salihu and Md Zahangir Alam. Solvent tolerant lipases: a review. Process Biochemistry, 50(1): 86-96, 2015.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemat- ical reasoning in open language models. arXiv preprint wxXv::2402.03300, 2024.
Ian Sillitoe, Nicola Bordin, Natalie Dawson, Vaishali P Waman, Paul Ashford, Harry M Scholes, Camilla SM Pang, Laurel Woodridge, Clemens Rauer, Neeladri Sen, et al. Cath: increased structural coverage of functional space. Nucleic acids research, 49(D1):
2021.
Zhenqiao Song. Tinglin Huang, Lei Li, and Wengong Jin, Surfpro: Functional protein design based continuous surface. arXiv preprint arXiv:24050066// 2024.
13

---

Published as a conference paper at ICLR 2026
Xiangru Tang, Xinwu Ye, Fang Wu, Daniel Shao, Dong Xu, and Mark Gerstein. Bc-design: A biochemistry-awar framework for highly accurate inverse protein folding. In ICML 2025 Genera- rive Al and Biology (GenBio) Workshop.
Yunhao Tang, Daniel Zhaohan Guo, Zeyu Zheng, Daniele Calandriello, Yuan Cao, Eugene Tarassov, Rémi Munos, Bernardo Ávila Pires, Michal Valko, Yong Cheng, et al. Understanding the perfor- mance gap between online and offline alignment algorithms. arXiv preprint arXiv:2405.0888 2444808448 2024.
Peter Vamplew, Richard Dazeley, Cameron Foale, Sally Firmin, and Jane Mummery, Human-aligned artificial intelligence a multiobjective problem. Ethics and information technology. 20(1):27-44 2018.
Kaiwen Wang, Rahul Kidambi, Ryan Sullivan, Alekh Agarwal, Christoph Dann, Andrea Michi, Marco Gelmi, Yunxuan Li, Raghav Gupta, Avinava Dubey, et al. Conditional language policy: A general framework for steerable multi-objective finetuning. arXiv preprint arXiv:2407.1576, 2024.
Ziwen Wang, Jiajun Fan, Ruihan Guo, Thao Nguyen, Heng Ji, and Ge Liu. Proteinzero: Self- improving protein generation via online reinforcement learning, arXiv preprint arXiv:2506.07459, 2025.
Joseph L Watson, David Juergens, Nathaniel R Bennett, Brian L Trippe, Jason Yim, Helen E Eisenach, Woody Ahern, Andrew Borst, Robert J Ragotte, Lukas F Milles, et al. De novo design of protein structure and function with rfdiffusion. Nature, 62097976):1089-11000 2023.
Talal Widatalla, Rafael Rafailov, and Brian Hie. Aligning protein generative models with experimental fitness via direct preference optimization. bioRxiv, pp. 2024-05 05, 2024.
Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopss, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Korbblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In International conference on machine learning, pp.
PMLR, 2022.
Fang Wu, Shuting Jin, Jianmin Wang, Zerui Xu, Jinbo Xu, Brian Hie, et al. Surfdesign: Effective protein design molecular surfaces.
Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah A Smith, Mari Ostendorf, and Hannanch Hajishirzi. Fine-grained human feedback gives better rewards for language model training. Advances in Neural Information Processing Systems, 36:59008-59033, 2023.
Junhao Xiong, Hunter Nisonoff, Maria Lukarska, Ishan Gaur, Luke M Oltrogge. David F Savage, and Jennifer Listgarten. Guide your favorite protein sequence generative model. arXiv preprint arXiv:2505.04823, 2025.
Junde Xu, Zijun Gao, Xinyi Zhou, Jie Hu, Xingyi Cheng, Le Song, Guangyong Chen, Pheng- Ann Heng, and Jiezhong Qiu. Protein inverse folding from structure feedback. arXiv preprint 2rXiv:25006030028, 2025.
Fanglei Xue, Andrew Kubaney, Zhichun Guo, Joseph K Min, Ge Liu, Yi Yang, and David Baker. Improving protein sequence design through designability preference optimization. arXiv preprint arXiv:2506.00097 2025.
Rui Yang, Xiaoman Pan, Feng Luo, Shuang Qiu, Han Zhong, Dong Yu, and Jianshu Chen. Rewards- in-context: Mblti-objective alignment of foundation models with dynamic preference adjustment. arXiv preprint 2xXv:22402.10207 2024.
Kaizhi Yue and Ken A Dill. Inverse protein folding problem: designing polymer sequences. Proceed- ings of the National Academy of Sciences, 89())4163344167 1992.
Vinicius Zambaldi, David La, Alexander E Chu, Harshnira Patani, Amy E Danson, Tristan OC Kwan, Thomas Frerix. Rosalia G Schneider, David Saxton, Ashok Thillaisuddarmm, et al. De novo design of high-affinity protein binders with alphaproteo. arXiv preprint arXiv:2409.0800 2024.
14

---

Published as a conference paper at ICLR 2026
Yang Zhang and Jeffrey Skolnick. Tm-align: protein structure alignment algorithm based on the tm-score. Nucleic acids research, 33(7):2302-2309 2005.
Zaixiang Zheng, Yifan Deng, Dongyu Xue, Yi Zhou, Fei Ye, and Quanquan Gu. Structure-informed language models are protein designers. In International conference on machine learning, pp. 42317-42338. PMLR, 2023.
Zhanhui Zhou, Jie Liu, Chao Yang, Jing Shao, Yu Liu, Xiangyu Yue, Wanli Ouyang, and Yu Qiao. Beyond mne-preference-frree Multi-objective direct preference optimization. 2023a.
Zhaoyi Zhou, Chuning Zhu, Runlong Zhou, Qiwen Cui, Abhishek Gupta, and Simon Shaolei Du. Free from bellman completeness: Trajectory stitching via model- return-conditooned supervised learning. arXiv preprint vrXiv:2310.193000 2023b.
Baiting Zhu, Meihua Dang, and Aditya Grover. Scaling pareto-efficient decision making via offline multi-objective rl. arXiv preprint arXiv:2305.00 00567, 2023.
Fengqi Zhu, Rongzhen Wang. Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji-Rong Wen, et al. Llada 1.5: Variance-reduced preference optimization for large language diffusion models. arXiv preprint arXiv:2505. /9223, 2025.
15

---

Published as a conference paper at ICLR 2026
APPENDIX
A ADDITIONAL EXPERIMENTS RESULTS
A.1 IN-DEPTH ANALYSIS OF THE GENERATED SEQUENCES
Solubility. In order to systematically evaluate the solubility-related physical properties of the generated proteins, we calculated series of quantitative indicators. These descriptors reflect different aspects of amino acid composition, surface exposure, and charge distribution, which together provide comprehensive view of protein solubility and stability. The indicators include:
Overall Hydrophilic Residue Fraction: the proportion of hydrophilic residues across the whole protein. A higher value indicates greater overall hydrophilicity and thus better solubility.
Surface Hydrophilic Residue Fraction and Surface Strong Hydrophilic Residue Frac- tion: the fraction of hydrophilic residues (or strongly hydrophilic residues such as charged side chains) exposed on the protein surface. Higher fractions suggest stronger potential for hydrogen bonding electrostatic interactions with water molecules.
the proportion of solvent-accessible surface area (SASA) contributed by hydrophilic residues, directly reflecting whether these residues exposed to solvent instead of buried within the protein core.
Surface Hydrophilic SASA Fraction:
Surface Net Charge per 100 Residues: the normalized net surface charge. Values farther from zero indicate stronger net charges, which promote electrostatic repulsion between protein molecules and reduce aggregation.
Surface Charge Distribution Uniformity: measure of how evenly charges are distributed across the protein surface. Higher uniformity implies a more balanced and ordered charge pattern, favoring stability in solution.
GRAVY Value (Grand Average of Hydropathy): an overall measure of hydropathy. Lower GRAVY values correspond higher hydrophilicity, typically associated with enhanced solubility.
Hydrophiliic
Comparison
Charge per
$.0
6672
0U101
Fracden
Madhoolikc Favticn
Mydrop & Faction
Scorg Fourtion
Surface Charge Distribution Unifermity
GRNVY Value Compartson
High Aigleelli
Sentore Oherge morthaatee indrrddy
2,000
.3.05
Figure 3: Quantative Analysis of ProteinMPNN, SolubleMPNN and MoMPNN generated sequences on hydrophilic-related metrics.
16

---

Published as a conference paper at ICLR 2026
Results of McMPNN [SoI+IG+ESM], ProteinMPNN, and SolubleMPNN based sequences gen- erated in the CATH benchmark are shown in Figure 3] Across all these indicators, MoMPNN consistently outperformed SolubleMPNN, suggesting that the proteins generated by MoMPNN not only achieves high silico score but also exhibit more favorable distributions of surface charge and hydrophilic residues. This highlights the ability of MoMPNN to design proteins with genuinely improved solubility.
Thermostability. Since thermostability more challenging property evaluate purely from an in silico perspective, we followed the analysis strategy of Ertelt et al. (024) and examined the amino acid distribution of MoMPNN [Thermo+[G+ESM], HyperMPNN, and ProteinMPNN in both surface and core regions. The results based on sequences generated in the CATH benchmark are shown in Figure f MoMPNN and HyperMPNN display almost identical redistribution patterns across residue categories, which differ systematically from ProteinMPNN.
Surface Residue Distribution by Category
0.8
ProtzinMPNN HуperMРNN
0.7
MoMPPNN
2.673
0.4
0.235
5,777
9.296
0.165
3.39]
5.115
0.19
0.1
0.000
0.033
Core Residue Distribution by Category
5.703
0.6
0.5
5,275
0.2
3.054
5,003
2.100
5:107
0.035
0,015
Positively Charged
Negatively Charged
Polar
Apolar
Others
Figure 4: Differences in amino acid composition of proteins from ProteinMPNN, HyperMPNN and MoMPNN.
Positively charged residues (Lys, Arg). Both MoMPNN and HyperMPNN show clear increase on the surface and in the core compared with ProteinMPNN. This enrichment of positively charged residues strengthens electrostatic interactions: on the surface it enhances solubility, while in the core facilitates stabilizing salt-bridge formation, contributing to thermostability.
Negatively charged residues (Asp, Glu). Slightly increased in both surface and core for MoMPNN and HyperMPNN relative to ProteinMPNN. This modest enrichment enhances polarity at the surface and contributes to more balanced charge distribution, which helps stabilize the folded structure.
Polar residues (Asn, Gln, Ser, Thr). Both models show small increase on surface and in core compared ProteinMPNN. These residues can form hydrogen bonds that stabilize secondary structures and packing, providing moderate contributions to thermostability.
Apolar residues (Iso, Leu, Met, Phe, Trp, Tyr, Val). In both MoMPNN and HyperMPNN, apolar residues markedly decreased the surface but strongly increased in the core.
17

---

Published as a conference paper at ICLR 2026
This redistribution is favorable: reduced hydrophobic exposure on the surface prevents aggregation, while enriched apolar residues in the core reinforce hydrophobic packing, a critical factor for thermostability.
Other residues (Ala, Cys, Gly, His, Pro). Both MoMPNN and HyperMPNN show clear overall decrease. As residues in this category often introduce backbone flexibility, their reduction suggests preference for more rigid and stable structural configurations.
Overall, the highly consistent residue redistribution patterns observed MoMPNN and HyperMPNN, as compared with ProteinMPNN, indicate that MoMPNN inherits the stability-orienter features of HyperMPNN while maintaining favorable balance between surface polarity and core hydropho- bicity. These trends strongly support the ability of MoMPNN to design sequences with enhanced thermostability.
A.2 PRELIMINARY EXPERIMENT RESULTS
In the early stage of our development, we conducted small scale test to verify whether the current choice of multi-objective modeling leads to better performance, comparing it to a naive weighted score method introduced in BB3) CATH4.3 was employed as the training set, which is the same as our main experiment. Evaluation results calculated by generating 16 sequences for each backbone in a curated validation set of 100 structures. All models use the default temperature for sampling as described in the main text.
In this experiment, we choose the Inital Guess and Evo. ppl as the optimization objectives. We also designed baseline method, Weighted-score DPO, which aggregates multiple optimization objectives into a single score using weights and then performs optimization following the standard single-objective DPO framework. The weights used here consistent with those of MoMPNN [IG+ESM].
According Table Weighted-score DPO achieves the best performance terms of Evo ppl., but its performance on other metrics is inferior to that of the base model ProteinMPNN, MoMPNN achieve significantly more balanced results in the small-scale test set.
Table 3: Comparison of protein sequence design methods across different evaluation metrics on de novo backbone structures from RFDiffusion. The best results and the second-best results are marked bold and bold.
Designability Metries
Method
Evo. ppl J
AAR 1
RMSD
TM t
PLDDT
ESM-IF InstructPLM
4.033
0.805
80.55
6.538
0.509
7.196
0.683
74.77
6.946
0.607
ProteinMPNN
3.658
0.823
82.09
6.843
0.441
Weighted-score DPO
3.783
0.811
80.74
5.843
0.410
MoMPNN [IG+ESM]
3.706
0.825
82.28
6.205
0.431
A.2.1 ANALYSIS OF ITERATIVE REFINEMENT
Subsequently, we evaluated the model's capacity for iterative improvement using the small-scale dataset. Specifically, we saved the model after each round training. For each saved model, we generated 16 sequences for every backbone in the dataset, then calculates the values of Initial Guess, Evo. ppl, and AAR for each round's model on the small-scale test set.
According to Figure 5 MOMPNN]G+ESMJ exhibits greater stability than Weighted-score DPO across the three metrics (Initial Guess, Evo. ppl, and AAR) during the semi-online training process. Moreover, as training rounds incrementally increase, the Weighted-score DPO model tends to converge on a single metric. For instance, as shown in the Fig. 5] MoMPNN enables the joint optimization of the two objectives (reaching values of 0.589 and 6.14, respectively) while only causing a 1% decrease AAR. In contrast, the optimization of Initial Guess for Weighted-score DPO exhibits fluctuations and starts decline from the 6th round, with more significant drop in
18

---

Published as a conference paper at ICLR 2026
AAR. This indicates that MoMPNN can effectively optimize multiple objectives simultaneously while minimizing deviations from the base model.
6.5
0.5
Figure 5: Analysis of MoMPNN [IG+ESM] and Weighted-score DPO. Initial Guess, Evo. ppl and recover rate changes across each round of iterative refinement.
B IMPLEMENT DETAILS
B. MATHMATICAL DERIVATIONS
For the theoretical completeness, we provide some definitions, lemmas, theorems and proofs of all the formulas the main text here (Zhou et al. 2023a Rafailov et al. 2023).
Firstly, the multi-objective function is:
arg max L(Te) = E D.yal(y)))
Σ wxrk(F,)))
3D Kr. (me((())))))))
(6)
Then, further derive the above equation.
max B.....Dnrsll
Σ wwrn(a,u)
(
31
n (y)a) log क [5yl (r)
max Ex~DE Bynw(
Σ wxrx(x,3))
3
गea (3)[))
log x(y12)
1
min E E. (sl
(xrr(or,))
B
t Tyy
min E E (glo)()))
log
Z(0) Teall (y) (r Ex warx(*.)) log Z(x)
Among them,
7(x) = Σ Tree Tuerly 2)e Ex (ar)(a.))
(8)
so, define that:
" Noting that ((x) and æ are independent, Eq given as follows:
(y|x))
7(x)) Tee F))(y Ex were(m.))
(9)
T,ylz)
min DI
log
Z (rec( En wark().)) log ((x)
(10)
min EraD (= log (x))
19

---

Published as a conference paper at ICLR 2026
So, get the minimum when:
1 <V/l) = n²" (y)ar)
Teet Tee Ex warn(x.))
(11)
(xx)
Then, can get that:
TT T(ylx) (y)((() + /3 log ((x)
Σ wArk(E,)) B log
(12)
where the number of properties.
Based on Eq. 3 and maximum likelihood estimation, for the k-th property can get the reward loss:
CR(FK, D&) (r., log (r (z, Mhw (N( x))]
(13)
where is the logistic function and rre y) is the implicit reward model.
can be seen from Eqs. 12 and 13 that for the k-th properties:
To (b(x))
(x(x,)) =
3 log
+ B log Z(x)
Σ
wwTw (I,
wp
Trer(y
K.
k' #/
log a 1 w/k
To (yw
3 log x))
Ta (311 x))
(14)
Cn(0;ra: Dx)
B log
Teet Teet (Jer
x))
ner (3) x))
we (1w (uw) rw ro. (2x vi) ))}
kf y'
Since there are K properties, the final multi-objective training objective is given by:
(m;(0;r;D) = Σ
w/ E (.) log o
Blog
( x))
3 3lg
To ( x
Wikk
Teer U/o x) Tret (Br
x))
K
- -w) 31)) )
(15)
Σ
ww vw)
K.
B.2 HYPERPARAMETER SETTINGS
Unless otherwise stated, all training runs utilized the Adam optimizer (31 = 0.9. 82 0.98, 10-9) with learning of 5e-6 for 20 rounds (600 training steps per rounds). Training was distributed across eight NVIDIA 4090 GPUs, with a total batch size of 64. The 3 for the DPO loss is to 0.5. The weights for each objective are set following: 0.6 for IG and TM, 0.4 for all other objectives.
B.3 BASELINES IMPLEMENTATION
ESM-IE. We employed the test script provided in the ESM GitHub repository (https //github. com/ waceboooreearcbeeeeeee. wttee/tteeeeee/e / wwa.mmanaaaaaaa mmaain/eaapeee//ieeer...
folding), with the model esm ifl _ep__ggp__uu (ee__11_112000 142M_ UR50. Aside from the parameters configured for testing (detailed above to enable comparative evaluation), all remaining parameter settings followed the default configurations supplied in the repository. The default sampling temperature is set 0.1, which is the same as ProteinMPNN.
InstructPLM. We utilized the test script provided in the GitHub repository
with all other parameters following the default settings specified
20

---

Published as a conference paper at ICLR 2026
therein. It is noted the default temperature in experiments is default 0.8 and top p is 0.9. We also report the results when adopting the same configuration as ProteinMPNN for fair comparison, i.e. temperature T 0.1 without top p, which show similar performance.
ProteinMPNN. ProteinMPNN provides multiple models based on distinct noise levels. For a more comprehensive comparison, we adopted the default ProteinMPNN model with 0.2 A noise. We use the testing scripts of ProteinMPNN from the ProteinMPPNN GitHub repository (https? All other settings are default.
//github. com/ Ptapppaaas//rtteeiaeee
ProteinDPO.. We implemented the testing scripts for ProteinDPO from the GitHub repository https://gituub.com/eee wwtpccooo//vveeeeeeeeeeee pewpevvoddsiigppppttin---po All other parameters followed the default settings specified, with the temperature set 0.1.
SolubleMPNN. We implemented the testing scripts of ProteinMPNN from the ProteinMPNN GitHub repository using the checkpoint of SolubleMPNN (https: htt///gittul com/ dauparas/ ProteinMPNN). As the same ProteinMPNN, we choose the default model with 0.2 A noise. All other settings are default.
HyperMPNN. We get the model weights for he different training settings (added backbone noise) from the HyperMPNN GitHub repository (ht / wwm.comm///////e memieez serlate ab/Hype TMPNN) and use the testing scripts of the original ProteinMPNN. Similar to ProteinMPNN, choose the default model with 0.2 A noise.
Guidance Method. We implemented a predictor- free guidance method following the approach in (Nisonoff al. (2024). The wredictor-guided rates alternatively obtained in terms of conditional R2 2 Bly and unconditional R2 (r, F), rates for x + I in the form
R ( (z. (l) R,,
(16)
As shown in this equation, the guided rates R. (1) (z, mly) generalize both the conditional [R(-=)) (z. Bly) (2, øly)] and unconditional (R(-=0) (z, periments, we used the ProteinMPNN as the unconditional model and SolubleMPNN / Hyper- MPNN the contional model. We set y 0.5, with all other settings consistent with used for ProteinMPNN. We used the reference code provided in the paper /nttps://githbb com/ hnissoofff wcreteeereuue secrrett_guiddaceeeeeee
R1 ø)] rates. In
Weighted-score DPO. We directly train a DPO model by obtaining a final score from a weighted combination of the ratings provided by different preference annotators. This aggregated score was then used to construct training pairs, on which we applied the standard DPO training procedure. All hyperparameters, including the importance weight Wy, were kept identical those used in MoMPNN.
B.4 PREFERENCE PREDICTORS
Structural Consistency. We download the ESMFold model from https://github. com/
facebookresearchh esm) esm) and TMalign (https: http///aangguuuuppoe ( wtwp:ppp..rg//MMaagg.g. used for calculating the TM score between predicted structure and the input. For AlphaFold Initial Guess, we use the implementation from design
Evolutionary Plausibility. We download the esm2 133 650M UR50D model from https: huggingface. co/ wwcebook.eoooooo.oooooooooo.oo. eam2 t33 650M UR50D and calculate the sequence pscudo perplexity following Kantroo et al. (2025).
Solubility. We download the Protein-Sol predictor from The predictor outputs a score ranging [0, 1 for each sequence.
manchester. ac. wac.uuk/sfftwas
Thermostability. We download the TemBERTure model from https: htt///ttttbb///////// com/ ibmm wwwnibe-chhTeemBae Thermophilic score ranging (0, 1] for each sequence.
and use the temBERTure, CLS mode. The model outputs
21

---

Published as a conference paper at ICLR 2026
C BENCHMARK DETAILS
C.I
CATH 4.3 BENCHMARK
Data.
We download the CATH4.3 benchmark dataset from boon/ttts://gttteeoo wtw.ttuuu.cm//ABi..... ccm/A4Bio
dat seeeteeeeeeeeeeeeeeeeeee The dataset is provided as ISONL file and we convert PDB files by extracting the sequences and backbone atom co- ordinates.
Metrics.
We evaluate the designed proteins using the following metries:
RMSD measures the average structural deviation between the designed and reference struc- tures. The two structures are first aligned using the Kabsch algorithm, and the deviation is computed on Ca atoms (for backbone). The designed structures are predicted by ESMFold.
TM score quantifies the global structural similarity between the designed and reference structures. In our experiments, we used ESMFold to predict the 3D structure the designed sequence, and then computed the TM-score against the reference crystal structure.
PLDDT represents the average per-atom confidence score. The values are extracted from the B-factor column of the ESMFold output.
Evo ppl evaluates the evolutionary plausibility of the designed amino acid sequence. Lower perplexity values indicate closer alignment with natural protein sequence patterns, thereby reducing risks of aggregation or misfolding (see Appendix B.4).
Sol estimates the solubility of the designed sequence. The scores are predicted by Protein-Sol (see Appendix [B4).
Thermo estimates the thermostability of the designed sequence. The scores are predicted by TemBERTun Ture (see Appendix B.4).
AAR measures the averaged amino acid recovery, i.e., the fraction of residues in the predicted sequence matching the original sequence:
AAR T.TT 10;
(17)
where sequence length, x; is the amino acid at position in the predicted sequence, and y/ is the corresponding residue in the reference sequence.
The metrics are reported as the average value across all generated sequences for the test set.
C.2 De novo DESIGN BENCHMARK
Data. For the de novo design benchmark, we generated protein backbones with lengths ranging from 50 to 500 residues using RFDiffusion (Watson et al.) 2023). Specifically, generated four distinct backbones for each length within this range (i.e., 50, 51, 52, . 500 residues). All backbone generations were performed using the default parameters of RFDiffusion. There are total 1,824 backbones as de novo design benchmark dataset. These generated backbones were then used as inputs for subsequent modeling and evaluation our experiments.
Metrics.
We use the same set of evaluation metrics as the CATH4.3 benchmark except AAR.
C.3 BINDER DESIGN BENCHMARK
Data. further evaluate the performance of our model, we additionally adapted several challenging target proteins benchmarks to validate our model (Pacesa et al. [2024] Zambaldi et al. 2024). There were six distinct protein targets, and binders were designed for each these targets. Suppiementary T detailed information for each target protein, including its PDB ID, relevant chain details, and the length range of the designed binders. The hotspot residues specify the desired interaction interface the target protein. Following previous works, used RFDiffusion to generate 100 unique backbones for each binder, guided by the defined hotspot residues and binder length range.
22

---

Published as a conference paper at ICLR 2026
Table 4: Input structures, hotspot settings and binder lengths for benchmarks.
Design Target
Input
Target chain residue numbers
Target Hotsppot
Binder length(for benchmarks)
PD-LI
5045
A18-133 132
A56, AILS, A123 B485, E489, E494, E500, ES05 A65, A74, A77, A82, A85, A93 A64,
50-120
SC2RBD
6M0J
E333-526
50-120
BHRFI
A2-158 prodiction AF2 prodiction
2WH6
80-120
A32--146
A
CLN1-14
A1-188
A31, A46, A55, A152
80-175
Metries. In binder design, we evaluate models by sequence success rate, backbone success rate, evolutionary perplexity, and solubility.
Inter-chain PAE measures the predicted error in the relative alignment between the binder and target chains, with lower values indicating more precise and stable binding conformations.
Overall Ca RMSD measures the structural deviation of the designed binder's backbone from reference structure, with smaller values indicating greater conformational consistency and fold stability.
Binder pLDDT measures the local confidence of each residue's 's spatial arrangement in the binder, where values above 80 typically correspond to experimentally validated, thermody- namically stable regions.
Evolutionary Perplexity measures the evolutionary plausibility of the binder's amino acid sequence, with lower values indicating closer alignment with natural protein sequence patterns.
Solubility measures the ability of the binder to dissolve in aqueous environments, an essential property for experimental manipulation and potential therapeutic applications.
The sequence success rate refers to the proportion of generated binders deemed successful, where a binder sequence is defined as successful if it meets three criteria: binder sequence pLDDT > 80. inter-chain PAE 10, and overall Co RMSD < 2 Å. The backbone success rate refers to the proportion of successfully designed backbones among all generated binder backbones, where a binder backbone is considered successful if any one of its designed sequences meets the sequence success criteria. Together, these metrics confirm that the designed binder not only folds correctly but also maintains functional utility in practical scenarios.
D DISCUSSION ON DESIGN CHOICES
Our design choices were guided by computational limits, empirical observations, and the need to keep the multi colli-oobeectvee optimization stable. For rollout number and sampling temperature, early validation suggested that changes to these hyperparameters had only limited impact final performance when the total number of training iterations was fixed in MoMPNN. This pattern allowed us to adopt a moderate configuration without extensive tuning. For property weights, placed slightly more emphasis on the primary design objective than on auxiliary ones, since reducing the weight of the latter further slowed convergence and weakened results under the same training budget.
The adaptive margin used in the LMO follows directly from the multi-objective function, and it depends only on the assigned weights and the property scores of the paired sequences. With both the dataset and the weights fixed, the margin for each pair can be precomputed. But we also could create dynamic versions of the adaptive margin, for instance, by changing the weights over time or using signals from recent optimization behavior. The dynamic variants could potentially allow the optimization follow the Pareto front more closely. But, makes the model training unstable, and also requires careful control over how different objectives influence one another. Consequently, employs fixed weights and precomputed margins, while adaptive margin schemes are left for future work.
For preference-prae pair construction, we sorted rollouts and paired the best half against the worst half, using delta threshold to remove uncertain pairs. Since all properties are estimated by predictive models, small differences often reflect noise rather than clear preferences. Filtering these cases makes the training data more reliable, but it also means the data only focuses on clear differences. More
23

---

Published as a conference paper at ICLR 2026
sophisticated sampling strategies, such as hard-negative selection uncertainty-awee aware sampling, could mitigate under sampling of the ambiguous region; however, implementing them requires careful consideration of prediction noise and uncertainty. These ideas remain interesting directions to pursue beyond the scope of the current work.
E DISCUSSION ON FUNCTIONAL PROTEIN SEQUENCE DESIGN
Recent explorations protein sequence design have begun to extend beyond backhone-basee specifications by introducing functional constraints that capture chemically relevant exterior features. Currently, advanced functional protein sequence design methodologies, such as SuriPro (Song al. 2024). BC-Design (Tang et al.), and SurfDesign (Wu et al.), explicitly integrate critical chemical features of the extenior surface (e.g.. hydrophobicity, charge activity) that govem interactions with the environment or ligands into the sequence design task. Our ProtAlign framework has the potential to further optimize these functional protein design models by not only handling explicit functional constraints but also guiding the design of sequences toward broader spectrum of desirable physicochemical properties. Objectives such as binding affinity could be incorporated to enhance the designed protein's 's interaction capability with the target ligand: likewise, adding objectives like stability would support feasibility in biological or industrial environments. Optimization from these two perspectives may enable the models to design multi-propes functional proteins that are both high-affinit and practically viable.
F LLM USAGE
LLMs were only used general-purpost tool for language editing and polishing in the preparation of this manuseript. Specifically, their role was limited to optimizing the expression fluency, refining academic terminology consistency, and adjusting sentence structures of the text; they did not partici- pate in any aspect of this research, including the conception of research ideas, design of experimental protocols, collection or analysis of experimental data, or derivation of research conclusions. All content modifications made by LLMs have been thoroughly reviewed and verified by the authors to ensure accuracy, consistency with the original research intent, and compliance with academic norms. The authors take full responsibility for the final content of this manuscript.
24


Reviews:
[{"review_id": "kyjNwYRzb8", "reviewer": "Reviewer_NtsP", "summary": "This paper applies multi-objective preference optimization to protein inverse folding, using semi-online DPO with adaptive margins to balance structural accuracy against properties like solubility and thermostability. The resulting model, MoMPNN, beats existing baselines  across several benchmarks. The approach is solid but not particularly novel—it's essentially transplanting techniques from LLM alignment into protein design. That said, the execution is strong: the experiments are thorough, the amino acid distribution analysis shows the model learns sensible patterns, and the framework appears general enough to extend to other properties. The comprehensive evaluation is strong.", "questions": "No questions.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "See summary", "weaknesses": "See summary", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "f9gWG8ybpB", "reviewer": "Reviewer_4A1d", "summary": "The paper proposes ProtAlign, a multi-objective preference-alignment framework for protein inverse folding that optimizes developability properties without compromising designability. The method uses a semi-online DPO loop: generate rollouts at higher temperature, score them with property predictors, construct pairwise preferences per property, then train offline with an adaptive preference margin to reconcile conflicts among objectives. Instantiated on ProteinMPNN as MoMPNN, the approach is evaluated on CATH 4.3, de novo backbones from RFDiffusion, and realistic de novo binders; results show developability gains while maintaining or improving structural consistency relative to strong baselines.", "questions": "- Can the weights across properties and the adaptive margin be tuned online using objective-improvement rates to more reliably approach a Pareto front across backbones and lengths?\n\n- What is the effect of the number of rollouts and sampling temperature on the stability of training and final metrics in the semi-online loop, given that the paper uses a higher temperature for exploration but evaluates at a lower temperature for ProteinMPNN-family models?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Method is simple and general: multi-objective DPO with an adaptive preference margin to mitigate conflicts across properties; the training pipeline evenly samples pairwise entries across properties and alternates rollout and training for efficiency.\n\n- Practical semi-online training decouples rollout/evaluation from optimization, enabling batch computation and easier deployment while retaining online exploration benefits.\n\n- Evaluations are broad and application-relevant: crystal redesign, de novo backbones, and realistic binder design; the study systematically integrates developability metrics into inverse-folding evaluation beyond amino acid recovery.\n\n- The presentation style is good, with nice-looking figures and easy-to-follow-up narration styles.", "weaknesses": "- Limited ablations on multi-objective weights and margin settings. It might be helpful to quantify how weights, temperature, and margin thresholds shape the Pareto front and to provide transferable default configurations as the paper heavily relies on it.\n- The adaptive preference margin m(yw,yl) is precomputed from auxiliary property deltas and then kept fixed during training. This is simple and fast, but it cannot react if the policy distribution drifts, predictors recalibrate, or property trade-offs evolve; the “right” margin may change as the frontier moves. \n- Pair construction may over-represent “easy wins” and under-sample ambiguous regions. Preference pairs are formed by sorting rollouts and pairing top-half vs. bottom-half, with a delta threshold to drop uncertain pairs. While this stabilizes supervision, it can bias learning away from the decision boundary where the frontier is decided. Active pair mining (hard-negative selection) or uncertainty-aware sampling could help learn more from the ambiguous region and reduce label imbalance across properties.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Aeag8piwmO", "reviewer": "Reviewer_qpSG", "summary": "The paper addresses the challenge that protein inverse folding models must balance designability (recovering a backbone) with developability properties (e.g., solubility, thermostability). The authors propose ProtAlign, a multi-objective preference alignment framework that fine-tunes pretrained models using a semi-online Direct Preference Optimization (DPO) strategy. The method uses a flexible preference margin to mitigate conflicts between competing objectives and constructs preference pairs using in silico property predictors. Applying this to ProteinMPNN yields MoMPNN. Experiments on CATH 4.3 crystal structures , de novo backbones , and binder design scenarios show that MoMPNN enhances developability properties without compromising structural fidelity compared to baselines.", "questions": "Why is the AAR of ProteinMPNN on CATH 4.3 test 0.39, which seems lower than most of the reproduction of this model, e.g., 0.44 on CATH 4.3 was reported in ProteinInvBench? If this AAR is not correct, does it indicate a significant compromise of AAR?\n\nRL-based preference methods like ProteinDPO for inverse folding are discussed in the related work section. Why are they compared as baselines? They are supposed to be the most related baselines.\n\nRegarding the semi-online training strategy, is the preference dataset $\\mathcal{D}_k$ at iteration t cumulative (containing all rollouts from iterations $1 \\dots t$), or is it replaced entirely by the new rollouts?\n\nThe paper provides a compelling comparison against a \"Weighted-score DPO\" baseline in Appendix A.2, showing MoMPNN is more stable. Can the authors provide more intuition on why the flexible margin (Eq. 4 ) achieves better and more stable multi-objective optimization compared to directly optimizing on a weighted sum of preference scores?\n\nThe model is trained on protein monomers but evaluated on a de novo binder design task, which involves protein complexes. Did the authors observe any specific failure modes or performance issues at the binder-target interface, given that the model was not explicitly trained for complex-specific properties?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This method improves developability metrics using a preference alignment framework , which does not require additional specific, curated datasets of experimentally-validated proteins.\n\nThe authors evaluate MoMPNN on a strong set of tasks beyond standard sequence recovery. This includes redesigning CATH 4.3 crystal structures, designing sequences for de novo generated backbones, and a practical de novo binder design scenario. This rigorous evaluation demonstrates the method's utility in realistic design workflows where other baselines show performance degradation.", "weaknesses": "It would be better to report the metrics on ground truth sequences, as these metrics are based on prediction models as approximations.\n\nFull names of abbr.’s in tables are missing in the captions.\n\nThe temperatures used in inference of different baselines are not identical, resulting in potentially unfair comparison. A fair comparison would be either the greedy strategy (without temperature), or comparing the best point on the temperature-performance curves between different methods; or at least report the results under one identical temperature.\n\nis it a typo in eq 4? k appears in the formula of m, which seems irrelevant to k.\n\nExplanation for the relationship between L and L_MO is needed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)