Learning to engineer protein flexibility
Authors: Petr Kouba
Joan Planas-Iglesias
Jiri Damborsky
Jiri Sedlar
Stanislav Mazurenko
Josef Sivic
Venue: ICLR 2025 Poster (2025)
Keywords: protein flexibility
protein flexibility prediction
protein design
protein sequence design
inverse folding
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Generative machine learning models are increasingly being used to design novel proteins. However, their major limitation is the inability to account for protein flexibility, a property crucial for protein function. Learning to engineer flexibility is difficult because the relevant data is scarce, heterogeneous, and costly to obtain using computational and experimental methods. Our contributions are three-fold. First, we perform a comprehensive comparison of methods for evaluating protein flexibility and identify relevant data for learning. Second, we overcome the data scarcity issue by leveraging a pre-trained protein language model. We design and train flexibility predictors utilizing either only sequential or both sequential and structural information on the input. Third, we introduce a method for fine-tuning a protein inverse folding model to make it steerable toward desired flexibility at specified regions. We demonstrate that our method Flexpert enables guidance of inverse folding models toward increased flexibility. This opens up a transformative possibility of engineering protein flexibility.

Published as a conference paper at ICLR 2025
LEARNING TO ENGINEER PROTEIN FLEXIBILITY
Petr Kouba 1.2
Joan Planas-Iglesias 2.3
Jiri Damborsky
Jiri Sedlar' Stanislav Mazurenko 2.3 Josef Sivic"
CCzec Institute of Informatics, Robotics and Cybernetics, Czech Technical University
2 Loschmidt Laboratories, Department of Experimental Biology and RECETOX, Masaryk University International Clinical Research Centre, St. Anne's University Hospital Brno
ABSTRACT
Generative machine learning models are increasingly being used to design novel proteins for therapeutic and biotechnological applications. However, the current methods mostly focus on the design of proteins with fixed backbone structure, which leads to their limited ability to account for protein flexibility, one of the crucial properties for protein function. Learning to engineer protein flexibility is problematic because the available data are scarce, heterogeneous, and costly to obtain using computational as well experimental methods. Our contributions address this problem are three-fold. First, we comprehensively compare methods for quantifying protein flexibility and identify data relevant learning. Second, we design and train flexibility predictors utilizing sequential or both sequential and structural information on the input. We overcome the data scarcity issue by leveraging a pre-trained protein language model. Third, we introduce method for fine-tuning a protein inverse folding model steer toward desired flexibility in specified regions. We demonstrate that our method Plexpert-Desigee enables guid- ance of inverse folding models toward increased flexibility. This opens up new possibilities for protein flexibility engineering and the development of proteins with enhanced biological activities.
INTRODUCTION
The goal of this study is to develop a tool that integrates protein flexibility into computational pro- tein design. Computational protein engineering and design are crucial technologies that have shown immense potential in creating next-generation enzymes for a wide range of applications, from the production of fine chemicals, pharmaceuticals, and food ingredients to sustainable, environmen- tally friendly solutions in green energy and biodegradation "Planas-Iglesiae et al., (02T). These approaches aim to propose new amino acid sequences, either from scratch (de novo) or by modify- ing sequences of existing natural proteins to alter their properties in a desired way. Despite the long history of these efforts, protein engineering and design remain labor-intensive and time-consuming. with uncertain outcomes in each case (Listov et al. 2024). Recent advances in generative machine learning have led to several powerful tools for designing proteins with specific topologies (the in- verse folding problem), offering the potential for universal protein design platforms (Notin et al., 2024 Kortemme] 2024). However, the number of successful case studies using these tools remains relatively low. One of the major obstacles hindering their broader application is their inability to account for protein flexibility (Chu et al. 2024 Kouba et al. 2023).
Proteins are highly dynamic biomolecules, and the presence of flexible regions is critical for their biological function (Corbella et al.) 2023) Lemay-St-Denis et al. 2022). In particular, fine-tuning conformational dynamics of loops close the active site has been shown to be an important method for modulating substrate specificitics (Romero-Riveee 2021 and pH dependency (Shen et al.) (202]) of enzymes.
et al.] (022). turnover rates (Crean et al.
addition, the biological function of many proteins often requires that small molecule is transported through their structures, for ex- ample via tunnels leading to the active site, whose dynamical properties are thus critical for protein function (Jurcik et al.) 2018). Modulating the flexibility of protein parts even far away from the ac- tive sites has been shown to be an efficient strategy for improving proteins (Karamitros et al., 2022). However, quantification of protein flexibility remains challenging despite recent progress both experimental and computational approaches. Experimental methods, such as X-ray crystallogra- phy, nuclear magnetic resonance, and hydrogen-deuterium exchange coupled to mass spectroscopy,

---

Published as a conference paper at ICLR 2025
require expensive equipment, specialized biophysical expertise, and time-consumingg sample prepa- ration and analysis (Peacock & Komives 2021). Consequently, they lack the high throughput needed for systematic protein flexibility evaluation. Computational methods provide a broad spectrum of approaches for assessing protein flexibility, from rapid coarse-grained modeling to more accurate but highly computationally intensive molecular dynamies (MD) simulations (Kmiecik al. (016). However, the available datasets are limited, and systematic comparisons of different methods across large protein datasets are thus lacking. More importantly, it remains unclear how to effectively in- tegrate these computational methods into the ofate-of-the-arr generative tools, increasingly used in protein engineering and design (Kortemme 2024).
In this work, we aim to fill these gaps by proposing a new method for designing protein flexibility. First, we perform a comprehensive comparison of methods for quantification protein flexibility and identify relevant data for learning (Section 33. Second, we address the data scarcity issue by proposing new flexibility predictors utilizing just sequential (Flexpert-Seq) or both sequential and structural (Flexpert-3D) information on the input (Section 4.1) Importantly, our goal is to create fast predictor, which is crucial for its integration into state-of-the-art ML-guided protein engineering pipelines. Third, we introduce a new method flexpert-Designne for fine-tuning a protein inverse folding model to make it steerable toward desired flexibility in specified positions (Section (.2). Our results show that is possible to predict flexibility from sequence, and that adding structural information further improves the prediction. Furthermore, using our protein flexibility predictor, we demonstrate that inverse folding models can be steered toward generating protein sequences with increased flexibility, suggesting potential strategy for engineering of protein flexibility.
2 RELATED WORK
Experimental methods for measuring protein flexibility. Common experimental methods for determining protein flexibility include X-ray crystallography, Nuclear Magnetic Resonance (NMR), and Hydrogen-Deuteriunn Exchange coupled Mass Spectroscopy (HDX-MS) (Peacock) & Komives 2021). X-ray crystallography provides flexibility estimates based on the consistency of protein conformations in a crystalline state. The regularity of atomic positions across crystal lattice cells determines the precision of atom location, quantified by the B B-factor (or temperature factor), which indicates mobility (Wlodawer et al. 2008). Details on NMR and HDX-MS are given in Appendix A
Computational methods for measuring protein flexibility. Protein flexibility can be predicted using faster, more economical in silico methods (Rueda al., (007). A common approach in- volves deriving flexibility from Molecular Dynamics (MD) simulations, which apply Newton's 's laws to atoms to compute their motion over time. From these simulations, Root Mean Square Fluctua- tions (RMSF) can be computed per residue to quantify their flexibility observed in the simula- tions. While effective, MD is time-consuming, as must explore a wide range of protein conforma- tions. Elastic Network Models (ENMs) offer a faster alternative by modeling proteins as system of beads and springs, using Normal Mode Analysis to predict motions (López-Blane Blanco & Chacón 2016 Tirion| (996). ENMs were refined by Bahar et al. to include Gaussian and anisotropic move- ments (Bahar et al. 1997; Atilgan et al., (001) and are implemented in tools like ProDy (Bakan) let al. Zhang et al. 2021). More details ENMs are given in Appendix C Recent machine learning (ML)-based methods, such as AlphaFold (Jumper et al.] (202]) and ESMFold (Lin et al.) 2022). have been adapted for flexibility prediction. Combining these models with flow matching techniques in AlphaFlow has enabled sampling of broader conformational spaces at about 10 times the speed of MD, though with reduced structural precision (Jing et al. 2024).
Protein flexibility engineering. Engineering flexibility in proteins remains a significant chal- lenge. Traditional approaches often involve quantum mechanics (QM), QM/MM. molecular dynam- ies (MD), and Monte-Carlo simulations, along with extensive experimental testing (Planas-Iglesias et al., 2021). For example, Yu & Dalby (2018) employed MD simulations to analyze correlated motions in enzyme dynamics. Similarly, Osuna's group developed the Shortest Path Map (SPM) algorithm to identify key conformational hotspots in enzymes like retro-aldolase and monoamine oxidase (Casadevall et al. 2024| Romero- Rivera et al. 2017). Kamerlin's 's lab also utilized MD to study enzyme dynamics, particularly in (Ba)s barrel enzymes Chomero-Rivers et al.) 2022). An- other approach to flexibility design involves creating chimeras, where regions of one protein are transplanted into another. Notable examples include chimerie TEM1/PSE4 fllatamaasss studied
2

---

Published as a conference paper at ICLR 2025
with NMR (Morin et al., 2010) and Kamerlin's transplantation of the WPD loop in phosphatases 2018 Shen et al. 2022). Damborsky's 's lab has focused on transplanting dynamic loops to explore flexibility and catalytic mechanisms in luciferases (Schenkmayerova et al. 2023
Moise et al.
Chaloupkova et al. 2019), leading to the development of LoopGrafter, a web tool for in silico protein flexibility design Pllaas-Iglesiee
al. 2022
Protein inverse folding. The task of predicting a sequence that folds into given backbone struc- ture (protein inverse folding) has become a key area where machine learning significantly contributes to protein design, along with structure prediction. ProtcinMPNN (Dauparas et al. 2022) ). based on the graph neural network architecture introduced by Ingraham et al. (2019), is one of the most widely used tools for this task. Recent benchmarks, such as ProteinlnvBence (Gao et al., 2023c), feature newer methods, e.g., KWDesign and PiFold (Gao et al.) 2023b a), which outperform ProteinMPNN in standard metrics such as sequence recovery on datasets like CATH4.3 (Pearl et al. 2003). Despite these advancements, ProteinMPNN remains the community standard, largely due to extensive ex- perimental validation. This highlights the limitations of such metrics as sequence recovery, which do not fully capture model's ability to design functional or stable proteins (Wang et al., (023). As a result, there is growing interest in developing more comprehensive evaluation metries and tasks that better reflect wallworld protein design challenges.
3 ANALYSIS OF DATA SOURCES FOR PROTEIN FLEXIBILITY
One of the key challenges in the engineering of protein flexibility the lack of consistent and affordable methods for flexibility measurement or estimation. This hinders the development of flex- ibility datasets useful for learning. The most reliable type of data come from experiments and from physics-based simulations, in particular molecular dynamics simulations. However, experimental data are costly, time-consuming, and highly heterogeneous due to biases and different calibrations of the experiments. Similarly, molecular dynamics data are time-consuming and subject to many human-made choices, such as the choice of force field. Our goal in this section is to analyze the available data and data generation methods for protein flexibility that could be utilized for learning a fast flexibility predictor. We first specify how we quantify protein flexibility using relevant ex- perimental and computational methods (Section 3.1. Then we critically compare these methods to identify a relevant data source for learning (Section
3.1 METHODS FOR QUANTIFICATION PROTEIN FLEXIBILITY
In this work focus on flexibility as quantity describing the ability of individual residues (mainly their backbone) to move within the protein structure. To this end, we introduce in this section the relevant methods for quantification of protein flexibility, and in particular, the metrics used with each method for reporting per-residue protein flexibility values.
MD. Molecular Dynamics (MD) simulations can be used to quantify protein flexibility by simulat- ing the atomic movements in a given modeled solvent environment. MD simulations are considered an accurate shysics-based method and we thus regard them as the gold standard for flexibility es- timation. However, their limitations should still be noted: (i) the simulation of the solvent is only approximate, affecting mainly the simulation accuracy of side-chain atoms, and (ii) potential inter- actions driven by molecules commonly present in the natural environment but not explicitly present in the simulation might affect the flexibility. For working with data MD simulation, we select the recently published ATLAS dataset (Vander Meersche et al.) 2023), particularly due to its excellent level of data curation. ATLAS consists of MD trajectories of 1390 proteins, each simulated in three replicas for 100ns per replica. For each protein and its residues, we quantify their flexibility by computing their root mean square fluctuations (RMSF) averaged over the three simulation replicas (see Appendix
B for details on RMSF).
B-factors. The crystallographic B-factor carries information protein flexibility, it describes the inherent thermal vibrations of atoms a crystal lattice. In such an organized environment, however, the natural movement of the protein is diminished, and crystal packing artifacts may lead to an underestimation of the real protein motions (Eyal et al. 2005). In this paper, we report B- factors obtained from the Protein Data Bank (PDB) (Berman al. 2000). Note that for few cases from PDB, we also kept the B-factors originating from NMR experiments, whose nature is different from those obtained in crystallographic experiments (for details, see Appendix A).
3

---

Published as a conference paper at ICLR 2025
Table I: Pearson correlation coefficients of flexibility predictions obtained by PDB B-factors and by computational methods (see Section 3.1 for details of the studied flexibility quantification methods). The reported coefficients are averaged over the 1383/1390 proteins from the ATLAS dataset (some were skipped due to missing pieces of structure resulting in NaNs from ENMs). As the table is symmetric, we show only values in the upper triangle. The best result in terms of correlation to MD simulations is highlighted in bold, and the second best is underlined.
MD
factors
AlphaFold2
ESMFold
GNM
ANM
MD
0.51
0.71
0.57
0.76
0.77
B-factors
0.52
0.38
0.55
0.51
AlphaFold2
0.66
0.55
0.56
ESMFold
0.45
0.43
GNM
0.94
ANM
AlphaFold2. The protein structure prediction model AlphaFold2 (AF2) (Jumper et al. 2 con- tains a confidence model predicting PLDDT scores. While the PLDDT score is primarily a measure of estimated confidence by AlphaFold2 (that may be induced by sequence variability in the region, for instance), it has been shown to be negatively correlated with flexibility (Ruff& & Pappu, 2021 Guo al., 2022, Saldaño et al. (022) and exploited to approximate flexibility as measured by Nuclear Magnetic Resonance (NMR) (Ma et al.) 2023). Therefore, to quantify flexibility with AF2, we use (1- PLDDT). Since the PLDDT prediction cannot be easily decoupled from the structure prediction, its runtime is the same for the structure prediction, resulting in relatively slow speed.
The protein structure prediction model ESMFold (Lin et al. 2022) also provides a PLDDT confidence model but with a significantly faster runtime than AlphaFold2. As in the case of AlphaFold2, we use (1-pLDDT) to quantify flexibility with ESMFold.
ESMFold.
GNM. Gaussian Network Models (GNM) are a type of Elastic Network Models that model the atomic motions isotropic, i.e., they model only the magnitudes of the displacements. To quantify flexibility, we use GNM to estimate the RMSF of each residue using the input structure from the ATLAS dataset that underwent a short MD relaxation. For details on the computation of RMSF for GNMs, see Appendix B
ANM. In contrast, Anisotropic Network Models (ANM) model the atomic motions as anisotropic, i.e. they model also the direction of the displacement. As in the case of GNM, we quantify flexibility using RMSF estimated from an MD relaxed structure. For details on the computation of RMSF for ANMs, see Appendix
3.2
COMPARISON OF PROTEIN FLEXIBILITY QUANTIFICATION METHODS
Next, we compare the selected methods for quantification of protein flexibility on the ATLAS dataset (see Table D. This is done via computing the correlation between the per residue flexibility profiles obtained through the different methods. The results of this analysis are discussed next.
Table shows that the B-factors obtained for the ATLAS proteins from the PDB database do not correlate well with the other methods. We hypothesize that this is due to the crystal packing effect. The root mean square fluctuations (RMSF) from MD simulations thus appear to be more reliable proxy for the true flexibility of single proteins (i.e., not in erystal). We further support this conclu- sion by training a predictor of the B-factor data, which reached only modest performance, hinting at B-factor data being insufficient quality for learning (see Appendix DD. Hence, in the rest of the paper, we regard the flexibility computed using RMSF from the MD simulations (denoted further as MD) as the gold standard, which we will try to approach using learning.
Measured by the correlation to the RMSFs from the MD simulations, the best-performing compu- tational methods are the Anisotropic Network Models (ANM), closely followed by the Gaussian Network Models (GNM) (see highlighted row in Table [T. Here, both ANM and GNM are eval- uated over the MD relaxed structures from the ATLAS dataset and both result in mutually highly correlated RMSFs (Pearson correlation coefficient 0.94).
Having each simulation in three replicas in the ATLAS dataset allows us to evaluate the level of reliability of the RMSF computed using the short ATLAS MD trajectories. To this end, we compute
4

---

Published as a conference paper at ICLR 2025
the correlation of RMSF between different simulation replicas of the same protein. We find that the different replicas mutually correlate on average with the Pearson correlation coefficient (PCC) of 0.88. We consider this number to be the indicative upper bound for the average correlation of any flexibility predictor trained and evaluated on the ATLAS dataset.
ANMs require protein (backbone) structure the input, which can be limiting factor if such a structure is not available. While the ATLAS dataset provides MD relaxed structures, we also want to know how the ANMs perform outside of the ATLAS setting, for example, on structures coming from (i) the PDB, (ii) protein folding of the original sequence by ESMFold, or (iii) protein folding of an alternative sequence obtained with ProteinMPNN (Dauparas al., 2022). Therefore, we investigate the effect of the type of input structure on the performance of ANMs in Table O in Appendix E where we show that redesigning the sequence of protein with ProteinMPNN does not have a significant effect on the performance of ANM.
METHODS
In this section, we introduce our new methods for predicting protein flexibility (Section 4. and the new Flexpert-Design framework demonstrating the applicability for the problem of engineering protein flexibility (Section 4.2)
4.1 LEARNING TO PREDICT PROTEIN FLEXIBILITY
Our above analysis of methods for protein flexibility quantification (see Section 3.2 effort to develop a new protein flexibility predictor as there is still enough room for improvement in terms of correlation to MD (compare the PCC of 0.77 for ANM with the indicative upper bound on PCC of 0.88). Here we address the challenge by introducing a novel and fast predictor of protein flexibility in two versions: Flexpert-Seq, taking protein sequence the input, and Flexpert-3D, taking both sequence and structure on the input. In both versions, the method learns to predict the RMSF of MD simulations in a supervised manner using the ATLAS dataset. The two predictors described next.
motivates the
Flexpert-Seq: Predicting flexibility from sequence. The goal is to de- velop a predictor of protein flexibility that uses only protein sequence on the input and learns to predict the pro- tein sequence flexibility as close as possible to the predictions from MD trajectories. In detail, given a dataset Dseg {($), y4) JED of protein se- quences E A: and per residue flexibility labels yı E R.W, where D is the set indexing the protein in the dataset, AA is the alphabet of 20 nat- ural amino acids and N, is the num- ber of amino acids of protein i, the goal train regression model Fseg N. - R to predict the per residue flexibility from protein se- quence. To overcome the limitations coming from the small amount of an- notated data (1390 proteins in the AT- LAS dataset), we leverage the pre- trained protein language model Prot- Trans (Elnaggar et al. 2022) to obtain per-residue embeddings, top of which learn a linear layer regress single scalar value for each residue. An alternative choice for the backbone protein language model could be the ESM-2 model, which was shown to perform similarly to ProtTrans in fine-tuning for diverse downstream
Flexpert-3D
Input
Flexpert-Sec
Output flexibility
Prott'rans LoRA
Rogressoo
0.4
0.3
Input structure
Antsotrupic
CNN Adaptor
Network
Pixed modules
Trainable modules
Frozen layers
Undrozen layers
Figure 1: Overview of the architecture of our protein flex- ibility prediction methods Flexpert-Seq (the red box) and Flexpert-3I) (the whole architecture). Icons of snowflake and flame denote frozen and unfrozen layers, respectively, in modules containing trainable parameters. Trainable (or- ange) and fixed (blue) modules denote whether any trainable unfrozen parameters are present in the module.
5

---

Published as a conference paper at ICLR 2025
tasks (Schmirler et al.) 2024). Following the procedure proposed by Schmmireer al. (2024),
employ LoRA (Hu et al. 202 lay to fine-tune ProtTrans together with training the regression layer for our downstream task. See Figure T for overview of the method.
Flexpert-3D: Predicting flexibility from sequence and structure. Since inverse folding pipelines usually have access protein backbones the input, we also explore the strategy of using both protein sequence and protein backbone structure on the input to predict the protein se- quence flexibility as close as possible the predictions from MD trajectories. More formally, given a dataset Dap = [[*+ Xt. Us) of protein sequences & E ,
protein backbone struc- tures X E R4x3N; (4 atoms per residue backbone) and per residue flexibility labels yı RR where D is the set indexing the protein in the dataset, A is the alphabet of 20 natural amino acids and N; is the number of amino acids of protein i, the goal is to train a regression model FAD: A. x RA43N,, + R to predict the per residue flexibility from protein sequence and back- bone structure. As ANMs can be run relatively fast if the protein backbone available and they were the best performing method as evaluated in Section 3.2 we tackle the task by combining our sequence-basee based method Flexpert-See -See with ANM-based flexibility values. This strategy is thus akin to learning how to correct the crude flexibility annotations returned by ANMs to match more accu- rate values derived from MD trajectories. To this end, we enhance the architecture of Flexpert-See by CNN head acting
adaptor of the ANM-predicted flexibility into the embedding space of Flexpert-Seg (see Figure T. We train the CNN Adaptor together with the LoRA layers by fine- tuning the ProtTrans Encoder of Flexpert-Seq and, together with the regression layer, mapping the combined embeddings to per-residue flexibility.
4.2 FLEXPERT-DESIGN FLEXIBILITY ENGINEERING VIA FLEXIBILITY-AWAAR INVERSE FOLDING
In this section, we apply our flexibility predictor Flexpert-3D for the task of engineering protein flexibility. The goal is to generate a protein sequence that respects given protein backbone structure and set of flexibility instructions.
More formally, we are given a dataset Dab = {(s. Xi) of protein sequences 8; .A% and protein backbone atom coordinates Xi E 1x33.. (4 atoms per residue backbone), where D is the set indexing the protein in the dataset, A is the alphabet of 20 natural amino acids and N, is the number of amino acids of protein i. We are also given set of flexibility instructions F = {fs heD,
where fa E N. The goal is to train model PF RN x IR R4x3N + A which takes ficxibility instructions and backbone structure of protein of length N and maps it to protein sequence of the length N, respecting the input backbone and the flexibility instructions.
We tackle the problem by leveraging the established protein inverse folding model ProteinMPNN and modifying it to take the flexibility instruction on the input. We then devise a learning strategy where we utilize our flexibility predictor Flexpert-3D fine-tune the inverse folding model toward respecting the flexibility instructions. Details are given next.
Teaching inverse folding model to engineer flexibility. The critical aspects of our flexibility engineering task are the choice of the flexibility engineering instructions F and the design of the learning strategy steering the inverse folding model follow the instructions.
Our novel training strategy is to inform the inverse folding model about the flexibility of the native sequence and to optimize the model toward generating sequences with flexibility preserved with re- spect to the input. We hypothesize that such training makes the model pay attention to the flexibility instructions f F provided on the input. During inference, this awareness enables us to exploit the flexibility input of the model to pass our actual instructions for modifying the flexibility with respect to the native one.
For learning the flexibility of native sequences, we propose the following sequence of steps. First, we construct the set Fnative of flexibility pseudolabels obtained with Flexpert-3D as
(1) where the FDexpert-3D predictor is used to predict flexibility for all tuples of sequences s, and backbone coordinates X; from dataset D These predicted flexibilities f.aaive, which we call "native", used in training inform the inverse folding model about flexibility.
pnation_ {f f enatinv FaD (s:, x:). (s;, Xi) E Dis. E D),
6

---

Published as a conference paper at ICLR 2025
Flexbbility Instructions
Flexpert-Desee
0.6
0.2
Flexibiling Input
Litee . .
Flexpert-3D
0.4
Flexibility recovery loss
ProteinMPNN (vanilla)
probabilities (per node)
Graph node Initialization
Predicted sequence
Input structure
Graph encoder
Sequence decoder
Gumbel- Gumbel-Softmax Softmax
sampling
Graph edge featuriser
Leee(
SCT
Sequence recovery
Fixed modules
Trainable modules
Frozen layers
Unfrozen layers
Loss
Figure 2: Overview of the Flexpert-Design method for fine-tuning of inverse folding models toward stecrability by requirements on protein flexibilityy. The inverse folding model ProteinMPNN, taking protein backbone structure the input, modified additionally take on the input flexibility instructions. This is implemented by adding a flexibility instruction to the zero-initialized node feature for each residue. The predicted sequence sampled in a differentiable way using Gumbel- Softmax and passed to Flexpert-3D together with the input structure. Note that we are making use of the assumption that ProteinMPNN does not modify backbone structure as we are matching together the newly predicted sequence with the ground truth backbone structure. The flexibility of the sequence predicted by Flexpert-3D passed to the loss function Lftez which computes the distance between the input flexibility instructions and the flexibility of the predicted sequence. Icons snowflake and flame denote frozen and unfrozen layers, respectively, in modules containing trainable parameters. Trainable (orange) and fixed (blue) modules denote whether any trainable unfrozen parameters are present in the module.
Second, we take a vanilla ProteinMPNN inverse folding model R 4x3N - AN which maps protein backbones of length N to protein sequences of the same length, and modify it to flexibility- aware ProNeinMPNN model PF R.N R433 AN which accepts flexibility instructions f F C R the input well (see Figure
Third, we introduce new loss function LFiez for guidance of the inverse folding model PF toward preservation of the input flexibility by matching the input flexibility instructions f to the output flexibility FSD x):
CFle L(FaD(&, x).
where L is a regression loss (MSE L1), X is the input backbone structure, and & Pr(S x) is the sequence predicted by the flexibility aware inverse folding model Pp. We illustrate our method, including the loss function LFies. in Figure Using the pseudolabels of flexibility Fnative ob- tained by Flexpert-3D as the flexibility instructions in training, the Lytez loss ensures in the training process that the model learns to preserve flexibility it has seen on the input. During inference, learned flexibility preservation is exploited to pass non-native flexibility instructions the model to steer the inverse folding toward generating sequences with the required flexibility.
Training to recover flexibility and sequence. To train ProteinMPNN toward flexibility aware- ness, we still need to ensure it also works for its original objective of generating a protein sequence first train ProteinMPNN using its standard Cross-Entroy
that fits the input backbone. Therefore, loss S and optimize for maximal sequence recovery. Taking the pre-traines ProteinMPNN model weights, we further fine-tune them using the combined Lriezpert loss:
Lrterpert = 0 L. + (1 - 0) Lseqy
(3)
where E [0, 1] is the parameter mixing the two losses L and Lseg into a single loss. The choice of 0 and the sequence recovery of the trained models discussed in Appendix

---

Published as a conference paper at ICLR 2025
Table 2: Pearson correlation coefficients between fast sequence-based haselines (AlphaFold2 and ESMFold) and our method Flexpert-Seq (columns) and the RMSF values from MD simulations and from Anisotropic Network Models (ANM) evaluated over MD relaxed structures (rows). Note that an upper bound on correlation to MD is estimated to be 0.88. Evaluation performed for 139 proteins from ATLAS testset. Our approach outperforms the AlphaFold2 and ESMFold baselines and obtains the best correlation with the flexibility estimates obtained from Molecular Dynamics (see the value in bold). Interestingly, the correlation coefficient of our approach and Anisotropic Network Models (ANM) is just 0.66, suggesting that both our approach and ANM might be complementary, motivating the development a predictor combining the sequence- based predictor with ANM.
AF2
ESMFold
Flexpert-Seq (ours)
MD
0.72
0.58
0.78
ANM
0.56
0.43
0.66
Preserving differentiability through discrete language model input. An interesting challenge in utilizing the protein language model-based predictor Flexpert-3D inside of an end-to-end training pipeline is the question of preserving differentiaaility at the interface of the inverse folding model output (continuous probability distribution) and the language model input (discrete tokens) (see Figure 2 Our solution this problem is sample the learned distribution of amino acids using the Gumbel-Softmax trick (Jang et al. 2017). In particular, we use its straight-through variant, which takes the argmax of the input distribution in the forward pass but preserves the gradients by approximating the argmax with the Gumbel-Softman distribution in the backward pass. This enables us sample discrete sequence for the language model while preserving the gradients and enabling end-to-end training of the entire pipeline.
5 RESULTS
In this section, we report the performance of our methods Flexpert-Seq and Flexpert-3D in terms of protein flexibility prediction (Section (5.) and the performance of Flexpert-Design in terms of increasing flexibility in engineered protein regions (Section 5.2)
5.1 FLEXIBILITY PREDICTION
Experimental setup. We use the sequences and MD relaxed structures from the ATLAS dataset as our inputs. The flexibility labels are the per residue RMSFs averaged over the 3 MD replicas for each protein. We split the dataset using topology splitting, with the additional requirement that no topologies present in the CATH4.3 test set are present in our ATLAS training set. The reason is to prevent potential leakage when combining the flexibility predictors into the Flexpert-Design pipeline which is trained using CATH4.3. We evaluate the Pearson correlation coefficient (PCC) between the predicted flexibility and the ground truth flexibility (RMSF) for each protein in the ATLAS test set. We report the average PCC across the test set.
Results. Our predictors Flexpert-Seq and Flexpert-3D outperform the basclines when using only the sequence on the input and when also using the structure, respectively. See Table 2 for the sequence case and Table 3 for the case when using sequence and structure. Note that our predictor Flexpert-3D, which partly relies on ANM, improves over them significantly and does not rely on them exclusively (see PCC of 0.78 to ANM).
We further tested our predictors Flexpert-Seq and Flexpert-3D on separate dataset called mdCATH (Mirarchi et al.] 2024), which features MD trajectories obtained at various simulation temperatures ranging from 320 K to 450 K. Using this data, we test the effect of the simulation temperature on the performance of our models trained on the ATLAS dataset simulated at physiological temperature of 300 K. The experiment confirmed that the further the temperature is from the simulation temperature in the training set, the lower the Pearson correlation coefficients are between the model predictions and the RMSFs, see Appendix F
In Appendix G. we provide qualitative examples of Flexpert- and Flexpert-3D predictions and how these predictions compare the ground truth RMSF and to the predictions of ANM.
8

---

Published as a conference paper at ICLR 2025
Table 3: Pearson correlation coefficients between fast structure-based baselines (Anisotropic Net- work Models and Gaussian Network Models) and our method Flexpert-3D (columns) and the RMSF values from MD simulations and from Anisotropic Network Models (ANM) evaluated over MD re- laxed structures (rows). Evaluation is performed for 139 proteins from ATLAS test set. Note that an upper bound on correlation to MD is estimated to be 0.88. Also, note that our approach outperforms the ANM and GNM baselines and obtains the best correlation with the flexibility estimates obtained from Molecular Dynamies (row MD). The correlation coefficient of our model with ANM is 0.78 (see the bottom row), indicating that model relies ANM input only partially and actually learns novel information from the sequence.
ANM
GNM
Flexpert-3D (ours)
MD
0.76
0.76
0.83
ANM
1.00
0.93
0.78
5.2 PROTEINMPNN CAN BE STEERED BY FLEXIBILITY INSTRUCTIONS
Experimental setup. The evaluation is done using the CATH4.3 dataset. The set of flexibility- increasing instructions FT is constructed in the following way: (i) The flexibility instructions are initialized by the native flexibilities Fnativc (as defined in Equation (I::
(ii) For each protein independently. a random contiguous segment S of length |S] is subselected from the sequence. In case |S| 2 N, where N the length of the protein, the whole protein is selected as the segment; (iii) For each residue i in each selected segment S, its flexibility instruction fi is incremented by T > 0, aiming at increasing the residues flexibility:
fi f f + -ies.
(4)
where 1 Lis is the indicator function for the set of amino acids corresponding to the segment S.
We then take the set of flexibility increasing instructions FT together with the corresponding back- bones Das from the CATH4.3 test set and pass it to the flexibility-awarc ProteinMPNN Pr trained using our Dessen method to obtain predicted protein sequences &:
6j Pr X1). V(f,, F x Dus-
(5)
We evaluate the predicted sequences using the metries described next. For each predicted sequence 8j. we identify the mutated residues Mj C S, inside the segment S engineered for increased flexibility. For such residues, we define the flexibility enrichment ratio riy:
Fan( Xi);
Tsj
fij frative
(6)
where 8j is the predicted sequence for protein j with backbone Xj. i E Mj indexes the mutated residues in the engineered region Sj. and faative is the native flexibility of the residue (see Equa- tion (T). The flexibility enrichment ratios Tiy are then used to report the median enrichment ratio and the proportion of fexibiiity-inerrasinn mutations, i.e., those for which rij 1.
Results. Table shows that our method Flexpert-Design successfully managed to engineer in- creased flexibility in the engineered region. This result is presented in more detail in Figure 3 which shows that Flexpert-Design can shift the flexibility distribution in the engineered segment toward higher flexibility. Furthermore, Figure B shows the change in the distribution of amino acid types in the engineered region. The observation that the proportion of Alanine (A) and Glycine (G) increases goes line with the biochemical understanding that these amino acids are smaller and tend to be more flexible. The sensitivity of Flexpert- Dessgn the size of the engineered region [S]
and the value of the flcxibility increasing instruction 7 is studied in Appendix J
Interestingly, the most contributing component of Flexpert-Design is the inclusion of flexibility input and training with the Flexpert-3D generated flexibility pseudolabels. The additional optimization using LFiez brings only relatively modest improvements in steerability (see Table 4 and Figure B). Please compare "Feexpent-Design (without loss)", which does not use Cyters and the full model denoted "Flexpert-Design"" The results also clearly demonstrate significant improvements of our

---

Published as a conference paper at ICLR 2025
10
1.07
1..43
1.52
Frequeency (thousamds)
25
Prote (vanilla]
Flespert-Design (wo loss)
Proportion [%)]
20
Flexpert- Deepeett--eignn (ours)
15
10
5
5.
0.5
1.0
1.5
2.0
2.5
2 3.0
LAEV P MYMYKE D PESPTIN N 6
Enrichment natio Tij
Amino acid type
Figure 3: Comparison of the enrichment ratio distributions (left) and the amino acid type distribu- tions (right) for the Our Flexpert-Design model is compared to its variant that was trained without the Leter loss but with the flexibility on the input. Vanilla ProteinMPNN is presented as a baseline. On the left, the median values for each distribution are denoted with a vertical line. On the right, the native amino acid distributions in the examined segments are presented for reference.
experiment with segment lengths |S| = 50 and 5.
Table 4: Comparison of the ability of flexibility engineering models to increase flexibility of engi- neered segments. The presented median enrichment ratios Med(ri,) and the proportion of flexibility- increasing mutations were obtained using the CATH4.3 test set with engineered segments of length |S| 50, and flexibility engineering instructions corresponding native flexibility incremented by r = 5 in the engineered segment. The best results in each category are highlighted in bold, second best underlined.
Median enrich, ratio Med(rsy) f Prop. of flexibility iner. mutations
Mexxpert-eeiign (ours) Flexpert-Design (w/o loss) ProteinMPNN (baseline)
1.52
0.83
1.43
0.80
1.07
0.61
model in the ability engineer flexibility compared to the ProteinMPNN baseline. Additional results in Section of the Appendix also demonstrate virtually no loss in sequence recovery metrics of our approach compared the ProteinMPNN baseline. The analysis of the structure preservation of the sequences with engineered increased flexibility is given in Appendix
We also investigated the use of our model for engineering of decrease in flexibility, results which we discuss in Appendix K
CONCLUSIONS
In this work, we address the problem of engineering protein flexibility-a highly relevant task due to its potential impact on the field of protein design. Firstly, we analyze the available data obtained in wet-lab experiments, physics- based simulations, or as byproduct of machine structure prediction models. We identify the flcxibility estimate by molecular dynamics simulation as the best learning target, set upper bound the potential performance of the learned predictor, and analyze the effect of the type of input protein structure on the performance of Anisotropic Net- work Models. Secondly, we develop a protein language model-based predictor of protein flexibility working either with protein sequence (Flexpert-Seq) or with both sequence and protein backbone structure on the input (Flexpert-3D) and show that our predictors outperform the relevant baselines. Thirdly, we propose a learning strategy Trexpert-Design, which uses the Flexpert-3D flexibility pre- dictor to steer an inverse folding model toward generation of sequences with increased flexibility. We demonstrate that inverse folding models can be steered toward generating protein sequences with increased flexibility as measured by our protein flexibility predictor. This opens up new possibilities for development of proteins with enhanced biological activities. In some practical protein engineer- ing tasks, it would also be of interest to engineer protein sequences with decreased flexibility, where the ability of our current Seexpert-Desight model seems limited (see Appendix K). This motivates fu- ture development of new models, which would provide a tighter control of the designed sequence's 's instructed to in-
barning-based
flexibility and which would be successful regardless of whether the flexibility crease or decrease. An interesting future direction could be to use Flexpert-31 3D or Flexpert- Seq for guidance of discrete flow matching models.
10

---

Published as a conference paper at ICLR 2025
REPRODUCIBILITY STATEMENT
The code, together with the instructions on how to download the data and trained weights for the reproduction of this work, can be found at httt:s///sttaaa..co////oootte///
ACKNOWLEDGEMENTS
This work was supported by the Ministry of Education, Youth and Sports of the Czech Repub- lic through projects ELIXIR (No. LM2023055), ESFRI RECETOX RI (No. LM2023069), and OP JAK CLARA (No. 02.23_023 _022). We acknowledge VSB Technical University of Ostrava, ITInnovvations National Supercomputing Center, Czech Republic, for awarding this project cess to the LUMI supercomputer, owned by the EuroHPC Joint Undertaking, hosted by CSC (Finland) and the LUMI consortium through the Ministry of Education, Youth and Sports of the Czech Republic through the e-INFRA CZ (No. 90254). This work was also supported by the European Union (ERC project FRONTIER (No. 101097822). ELIAS (No. 101 101237).. CLARA (No. 101 133667)), the Technology Agency of the Czech Republic under the NCC Programme from the state budget (No. RETEMED TN02000122 22) and under the NRP from the EU RRF (No. TEREP N102000122/0013) and the CETOCOEN EXCELLENCE (No. 857560) projects supported from the European Union's Horizon 2020 research and innovation programme. Views and opinions ex- pressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council. Neither the European Union nor the granting authority can be held responsible for them. Petr Kouba is a holder of the Bro Ph.D. Talent scholarship funded by the Brno City Municipality and the JCMM.
REFERENCES
A R Atilgan, S R Durell, R Jernigan, M C Demirel, O Keskin, and Bahar. Anisotropy of fluctuation dynamics of proteins with an elastic network model. Biophys. J., 80(1):505-515, January 2001.
I Bahar, A R Atilgan, and B Erman. Direct evaluation of thermal fluctuations in proteins using single-paramett harmonic potential. Fold. Des., 2(3):173-181, 1997.
Ahmet Bakan, Lidio M Meireles, and Ivet Bahar. ProDy: protein dynamics inferred from theory and experiments. Bioinformatics, 27(1 (7(11))575-1177 June 2011.
M Berman, Westbrook, Z Feng. Gilliland, T N Bhat, H Weissig, IN Shindyalov, and Bourne. The protein data bank. Nucleic Acids Res., 28(1):235-244 January 2000.
Guillem Casadevall, Jordi Casadevall, Cristina Duran, and Sílvia Osuna. The shortest path method (SPM) webserver for computational enzyme design. Protein Eng. Des. Sel., 37:gzae005,, January 2024.
Radka Chaloupkova, Veronika Liskova, Martin Toul, Klara Markova, Eva Sebestova, Lenka Herny- chova, Martin Marek, Gaspar P Pinto, Daniel Pluskal, Jitka Waterman, Zbynek Prokop, and Jiri Damborsky. Light-emitting dehalogenases: Reconstruction of multifunctional biocatalysts. ACS Catal., (6))48 1000:48104440 (8200048230000 June 2019.
Alexander E Chu, Tianyu Lu, and Po-Ssu Huang. Sparks of function by de novo protein design. Nat. Biotechnol., 42(2):203-215, February 2024.
Marina Corbella, Gaspar P Pinto, and Shina C L Kamerlin. Loop dynamics and the evolution of enzyme activity. Nat. Rev. Chem., 7(8):336-547, August 2023.
Rory M Crean, Michal Biler, Mare W van der Kamp, Alvan C Hengge, and Shina C L Kamerlin. Loop dynamics and enzyme catalysis in protein tyrosine phosphatases. Am. Chem. Soc., 143 (10):3830-3844 March 2021.
J Dauparas, I Anishchenko, N Bennett, H Bai, R Ragotte, L F Milles, B I M Wicky, A Courbet, R de Haas, N Bethel, P J Y Leung. T F Huddy. S Pellock. D Tischer, F Chan, B Koepnick, H Nguyen, A Kang, B Sankaran, A K Bera, N King, and D Baker. Robust deep tearning-basee protein sequence design using ProteinMPNN. Science, (78(6615):49-45 2022.
11

---

Published as a conference paper at ICLR 2025
Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang. Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, Debsindhu Bhowmik, and Burkhard Rost. ProtTrans: Toward understanding the language of life through self-supervised learning. IEEE Trans. Pattern Anal. Mach. Intell., (4(10)7112-711 October 2022.
Agustí Emperador, Tim Meyer, and Modesto Orozco. Protein flexibility from discrete molecular dynamics simulations using quasi-physical potentials. Proteins, 78(1):83-94, January 2010.
Eran Eyal, Sergey Gerzon, Vladimir Potapov, Marvin Edelman, and Vladimir Sobolev. The limit of accuracy of protein modeling: influence of crystal packing on protein structure. J. Mol. Biol., 351 (2):431-442) August 2005.
Vincent Frappier and Rafael Najmanovich. A coarse-grained elastic network atom contact model and its use in the simulation of protein dynamics and the prediction the effect of mutations. PLoS Comput. Biol., (0(4): April 2014.
Zhangyang Gao, Cheng Tan, and Stan 2. Li. Pifold: Toward effective and efficient protein inverse folding. In International Conference Learning Representations, 2023a. URL https://
Zhangyang Gao, Cheng Tan, and Stan Z. Li. Knowledge-design: Pushing the limit of protein design via knowledge refinement, 2023b. URL httts::/aarxi..or///att
15151
Zhangyang Gao, Cheng Tan, Yijie Zhang. Xingran Chen, Lirong Wu, and Stan Z. Li. Proteinin- vbench: Benchmarking protein inverse folding on diverse tasks, models, and metrics. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 68207-68220. Curran Associates, Inc., 2023c. URL. ht httpps///prrcceedinnng...errrrre ccppappe http://aaaaeapae/2/////////// 07975447796337722333332778888883 Datasets and Benchmarks
pdf
Hao-Bo Guo, Alexander Perminov, Selemon Bekele, Gary Kedziora, Sanaz Farajollahi, Vanessa Varaljay, Kevin Hinkle, Valeria Molinero, Konrad Meister, Chia Hung, Patrick Dennis, Nancy Kelley-Loughnane, and Rajiv Berry. AlphaFold2 models indicate that protein sequence deter- mines both structure and dynamics. Sci. Rep., 12(1):10696, June 2022.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. arXiv CLS, June 2021a.
Yunfei Hu, Kai Cheng, Lichun He, Xu Zhang, Bin Jiang, Ling Jiang, Conggang Li, Guan Wang. Yunhuang Yang, and Maili Liu. NMR-based methods for protein analysis. Anal. Chem., 93(4); 1866-1879, February 2021b.
John Ingraham, Vikas Garg, Regina Barzilay, and Tommi Jaakkola. Generative models for graph- based protein design. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buu E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32, Cur- Associates, Inc., 2019. URL /ppphttpss//prroceednngg... www.neuwr.....ppppeeeeeepppeeeee fi les/ pappa
Eric Jang. Shixiang Gu, and Ben Poole. Categorical reparammeetmettiiitooonn with gumbel-softmax In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL htt.:httts//ppereeeee met/fose Eorum? 2oorumm1dd=rkEy88ee
Bowen Jing, Bonnie Berger, and Tommi Jaakkola. AlphaFold meets flow matching for generating protein ensembles. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 22277-22300 PMLR, 21-27 Jul 2024. URL https: http:::///pccceddings .mlr. http:/innnmmi...s/s./.ee press/
v235/ htt./iig244.... html
12

---

Published as a conference paper at ICLR 2025
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A Kohl, Andrew J Ballard, Andrew Cowie, Bernardino Romera- Paredes, Stanislav Nikolov, Rishub Jain, Jonas Adler, Trevor Back, Stig Petersen, David Reiman, Ellen Clancy, Michal Zielinski, Martin Steinegger, Michalina Pacholska, Tamas Berghammer, Sebastian Bodenstein, David Silver, Oriol Vinyals, Andrew W Senior, Koray Kavukeuoglu, Push- meet Kohli, and Demis Hassabis. Highly accurate protein structure prediction with AlphaFold. Nature, 596(7873):588) 58888589,,88.88 August 2021.
Adam Jurcik, David Bednar, Jan Byska, Sergio Marques, Katarina Furmanova, Lukas Daniel, Pïaa Kokkonen, Jan Brezovsky, Ondrej Stradd, Jan Stourac, Antonin Pavelka, Martin Manak, Jiri Damborsky, and Barbora Kozlikova. CAVER analyst 2.0: analysis and visualization of channels and tunnels in protein structures and molecular dynamics trajectories. Bioinformatics, 34(20): 3586-3588, October 2018.
Christos S Karamitros, Kyle Murray, Brent Winemiller, Candice Lamb, Everett M Stone, Sheena D' D'Arcy, Kenneth A Johnson, and George Georgiou. Leveraging intrinsic flexibility to engineer enhanced enzyme catalytic activity. Proc. Nat. Acad. Sci. U. S. A., (19(2)):2118877)) 197911 19, June 2022.
Sebastian Kmiecik, Dominik Gront, Michal Kolinski, Lukasz Wieteska, Aleksandra Elzbieta Dawid, and Andrzej Kolinski. Coarse- protein models and their applications. Chem. Rev. 116(14); 7898-7936, July 2016.
Tanja Kortemme, De novo protein design-from new structures programmable functions. Cell, 187(3):526-5444 544, February 2024.
Petr Kouba, Pavel Kohout, Faranch Haddadi, Anton Bushuiev, Raman Samusevich, Jiri Sedlar, Jiri Damborsky, Tomas Pluskal, Josef Sivic, and Stanislav Mazurenko. Machine learning- guided protein engineering. ACS Catal., pp. 13863-13895, October 2023.
Claudèle Lemay-St-een Nicolas Doucet, and Joelle N Pelletier. Integrating dynamics into enzyme engineering. Protein Eng. Des. Sel., 35, February 2022.
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Allan dos San- tos Costa, Maryam Fuzel-Zarande Tom Sercu, Sal Candido, and Alexander Rives. Language models of protein sequences the scale of evolution enable accurate structure prediction. bioRxiv, pp. 2022.07. 20.5000900 July 2022.
Dina Listov, Casper A Goverde, Bruno E Correia, and Sarel Jacob Fleishman. Opportunities and challenges in design and optimization of protein function. Nat. Rev. Mol. Cell Biol., 25(8):639- 653. August 2024.
José Ramón López- López-Blanco and Pablo Chacón. New generation of elastic network models. Curr. Opin. Struct. Biol., 37:46-53, 53, April 2016.
Puyi Ma, Da- Wei Li, and Rafael Brüschweiler. Predicting protein flexibility with AlphaFold. Pro- leins, 9146):847-855, June 2023.
Antonio Mirarchi, Toni Giorgino, and Gianni De Fabritiis. mdcath: A large-scale md dataset for data-driven computational biophysics, 2024. URL https::/arxiv..or wwwpaawwwww.gg///////...////
14794
Milot Mirdita, Konstantin Schütze, Yoshitaka Moriwaki, Lim Heo, Sergey Ovchinnikov, and Martin Steinegger. Colabfold: making protein folding accessible to all. Nature Methods, 1966):679-682.
Jun 2022. ISSN 1548-7105.
doi: 10. 1038//4 192-002-011888---111 URL
10. 1038 s4 1221-992-22 022 01488-1
Gwendolyn Moise, Yalemi Morales, Victor Beaumont, Timothy Caradonna, J Patrick Loria, Sean Johnson, and Alvan C Hengge. A YopH PTPIB chimera shows the importance of the WPD-loop sequence to the activity, structure, and dynamics of protein tyrosine phosphatases. Biochemistry, 57(36):531555326 September 2018.
13

---

Published as a conference paper at ICLR 2025
Sébastien Morin, Christopher M Clouthier, Sophie Gobeil, Joelle N Pelletier, and Stéphane M Gagné. Backbone resonance assignments an artificially engineered TEM-1/PSE-4 class a 3- lactamase chimera. Biomol. NMR Assign., 4(2):127-130, October 2010.
Pascal Notin, Nathan Rollins, Yarin Gal, Chris Sander, and Debora Marks. Machine learning for functional protein design. Nat. Biotechnol, 42(2))216-222 February 2024.
Riley Peacock and Elizabeth A Komives. Hydrogen/deuterium exchange and nuclear magnetic resonance spectroscopy reveal dynamic allostery multiple time scales in the serine protease thrombin. Biochemistry, 60(46):34413344
November 2021.
M G Pearl, C Bennett, J E Bray, A P Harrison, N Martin, A Shepherd, Sillitoe, J Thornton, and C A Orengo. The CATH database: an extended protein family resource for structural and functional genomics. Nucleic Acids Res., (1 (1(())452-4555 January 2003.
Joan Planas-Iglesias, Sérgio M Marques, Gaspar P Pinto, Milos Musil, Jan Stourac, Jiri Damborsky, and David Bednar. Computational design of enzymes for biotechnological applications. Biotech- nol. Adv. 47(107696):107666 March 2021.
Joan Planas- Ianas-Iglesias, Filip Opaleny, Pavol Ulbrich, Jan Stourac, Zainab Sanusi, Gaspar P Pinto, Andrea Schenkmayerova, Jan Byska, Jiri Damborsky, Barbora Kozlikova, and David Bednar. LoopGrafter: a web tool for transplanting dynamical loops for protein engineering. Nucleic Acids
Res., 50451):W465-W473 July 2022.
Carlos H M Rodrigues, Douglas E V Pires, and David B Ascher. DynaMut2: Assessing changes in stability and flexibility upon single and multiple point missense mutations. Protein Sci., 30(1): 60 60-69, January 2021.
Adrian Comero-Rivera. Marc Garcia-Borria and Sílvia Osuna. Role of conformational dynamics in the evolution of retro-aldolase activity. ACS Catal., 7(12):8524-8522, December 2017.
Adrian Homero--iverr Marina Corbella, Antonietta Parracino. Wayne M Patrick, and Shina Car- oline Lynn Kamerlin. Complex loop dynamics underpin activity, specificity, and evolvability in the (8a)8 barrel enzymes of histidine and tryptophan biosynthesis. JACS Au, 2(4)) April 2022.
Manuel Rueda, Pablo Chacón, and Modesto Orozco. Thorough validation of protein normal mode analysis: comparative study with essential dynamics. Structure, 15(5):665-575, May 2007.
Kiersten M Ruff and Rohit V Pappu. AlphaFold and implications for intrinsically disordered pro- teins. J. Mol. Biol., 133000)) October 2021.
Tadeo Saldaño, Nahuel Escobedo, Julia Marchetti, Diego Javier Zea, Juan Mac Donagh, Ana Ju- lia Velez Rueda, Eduardo Gonik, Agustina García Melani, Julieta Novomisky Nechcoff, Martín N Salas, Tomás Peters, Nicolás Demitroff, Sebastian Fernandez Alberti, Nicolas Palopoli, Maria Silvina Fornasari, and Gustavo Parisi. Impact of protein conformational diversity Al- phaFold predictions. Bioinformatics, 38(10):7722-2748, May 2022.
Andrea Schenkmayerova, Martin Toul, Daniel Pluskal, Racha Baatallah, Glwadys Gagnot, Gaspar Pinto. Vinicius T Santana, Marketa Stuchla, Petr Neugebauer, Pimchai Chaiyen, Jiri Damborsky, David Bednar, Yves L Janin, Zbynek Prokop, and Martin Marek. Catalytic mechanism for renilla- type luciferases. Nat. Catal., 6(1):23-38 January 2023.
Robert Schmirler, Michael Heinzinger, and Burkhard Rost. Fine-tuning protein language models boosts predictions across diverse tasks. Nat. Commun., 15(1):7407, August 2024.
Ruidan Shen, Rory M Crean, Sean Johnson, Shina C L Kamerlin, and Alvan Hengge. Single residue the WPD-loop affeets the pH dependency of catalysis in protein tyrosine phosphatases. JACS Au, 655,,,,6,,,, May 2021.
Ruidan Shen, Rory M Crean, Keith J Olsen, Marina Corbella, Ana R Calixto, Teisha Richan, Tiago A S Brandão, Ryan D Berry. Alex Tolman, J Patrick Loria, Sean J Johnson, Shina C L Kamer- lin, and Alvan Hengge. Insights into the importance of WPD-loop sequence for activity and structure in protein tyrosine phosphatases. Chem. Sci., 13(44) 135400 November 2022.
14

---

Published as a conference paper at ICLR 2025
M M Tirion. Large amplitude elastic motions in proteins from single-paramee atomic analysis. Phys. Rev. Lett., 77(9):1055-1903; August 1996.
Lukas Uhrik, Tomas Henek, Joan Planas-Iglesias, Josef Kucera, Jiri Damborsky, Martin Marek, and Lenka Hemyccova. Study of protein conformational dynamics using hydrogen/deuterium exchange mass spectrometry. Methods Mol. Biol., 2652:293-318, 2023.
Yann Vander Meersche, Gabriel Cretin, Aria Gheeraert, Jean- Christhist Gelly, and Tatiana Ga- lochkina. ATLAS: protein flexibility description from atomistic molecular dynamics simu- lations. Nucleic Acids Research, 52(DI))D344DD392 11 2023. ISSN 0305-1048. doi:
10.1093/mar/gkaad0044 URL /ttt://tpee///oo...or////0/////0
Chuanrui Wang, Bozitao Zhong, Zuobai Zhang, Narendra Chaudhary, Sanchit Misra, and Jian Tang. Pdb-struct: A comprehensive benchmark for dtructure-based protein design, 2023. httpst:::
//arssv.orrg/abs/22222.000
Alexander Wlodawer, Wladek Minor, Zbigniew Dauter, and Mariusz Jaskolski. Protein crystallog- raphy for non-crystallographers, how to get the best (but not more) from published macro- molecular structures. FEBS J., 275(1):1-21, January 2008.
Lei Yang. Guang Song. and Robert L Jermigan. Protein elastic network models and the ranges of cooperativity. Proceedings of the National Academy of Sciences, 106730):1234--12255, 2009.
Haoran Yu and Paul A Dalby. Coupled molecular dynamics mediate long- and short-range epistasis between mutations that affect stability and aggregation kinetics. Proc. Natl. Acad. Sci. U. S. A.,
115477)E11143--11055 November 2018.
She Zhang, James M Krieger, Yan Zhang, Cihan Kaya, Burak Kaynak, Karolina Mikulska- Ruminska, Pemra Doruker, Hongchun Li, and Ivet Bahar. ProDy 2.0: increased scale and scope after 10 years of protein dynamics modelling with python. Bioinformatics, 37(20):357-3355, October 2021.
15

---

Published as a conference paper at ICLR 2025
APPENDIX
A EXPERIMENTAL MEASUREMENT OF PROTEIN FLEXIBILITY BY NMR AND HDX-MS
In contrast to crystallographic methods directly obtaining the B-factors (temperature factors), NMR reads chemical shift perturbations, in other words, how the proximity of different atoms affects the chemical environment of each other. As a product of these measurements, atom-to-atom distance restraints can be inferred. Protein flexibility can be derived as the change in the intensity and ampli- tude of the read signals over time; thus, different sets of restraints can be inferred for different time points (Hu et al.) (0211b. Hence, NMR usually provides an ensemble of structural models, from which atomistic Root Mean Square Fluctuations (RMSFs) can be derived, which can be converted to B- B-factors by the following equation:
8A 2
B:
3 (RMSF))²
where B is the B-factors and i represents individual atom (Emperador et al. 2010).
The HDX-MS relies on the exchange of isotopic hydrogen in a deuterium-rich medium over a given span of time to detect the protein segments that are more exposed to solvent. Flexible regions (which often lay on the protein surface away from the hydrophobic tightly packed protein core) allow for higher solvent penetrance and, thus, attain higher deuteration values when compared to less flexible areas (Uhrik et al. 2023).
B RoOT MEAN SQUARE FLUCTUATIONS (RMSF) AS A MEASURE OF FLEXIBILITY
The Root Mean Square Fluctuations (RMSF) are a way of quantifying protein flexibility, typically based on simulated data dynamies or based on simplified models of dynamies such as the Elastic Network Models (ENM). In this work, we use RMSF computed on top of both MD simulations and ENM (Anisotropic Network Models and Gaussian Network Models). These RMSFs were obtained in the following way:
RMSF from MD simulations. We used the values provided directly with the ATLAS dataset, which provides the per residue RMSF for each of its simulation replicas. To represent each protein's 's flexibility profile, we used the per residue RMSF averaged over all 3 simulation replicas of each protein. The values provided by the ATLAS dataset were computed as standard deviation of the a-carbon displacements, using the GROMACS software. That is, the RMSF for MD simulations were computed using the following formula:
Pt V (1lrs - (1))
where denotes the RMSF of residue i, r, stands for the coordinates of the a-carbon of in the global reference frame, the angle brackets ) stand for the average over all the frames of the simulation and - 1|2 denotes the Fuclidean norm.
residue
RMSF for Elastic Network Models (Anisotropic Network Models), We used the ProDy (Bakan al. 2011) package to compute the ENMs and the corresponding RMSF values. Firstly, we used ProDy to build the Hessian matrix, based on the c--carbon coordinates of our input structure and the cutoff of 16 Angstrom, then the non-trivial modes of the Hessian were calculated, and consequently these modes were used to solve the problem of finding the inverse of the Hessian to compute the fluctuations according to the formula for the ENMs:
P: = V
16

---

Published as a conference paper at ICLR 2025
Pearson correlation coefficients of MD flexibility and Flexpert-Seq predictions, when trained on B-factors (B) and per protein normalized B-factors (B
Table 5:
Flexpert-Seg (trained on B)
Flexpert-Seq
(trained on Bnorm)
MD
0.47
0,56
where P; denotes the RMSF of residue i, C' denotes constant that is typically equal to the product of the Boltzmann constant and the temperature (C = 7g but for us this is arbitrary as we are interested in the correlations of the fluctuations and the correlations are not affected by the scaling by constant. The Tr in the formula stands for the trace of the matrix [H which is the 3x3 submatrix of the inverse Hessian corresponding to the residue (the 3x3 submatrix comes from the fact that the a-carbon positions are represented in 3D).
RMSF for Elastic Network Models (Gaussian Network Models), For Gaussian Network mod- els, the procedure is analogous to the of Anisotropic Network Models. The only difference is that instead of Hessian matrix, the Kirchoff matrix is used:
'la),
(10)
where is the Kirchhoff matrix constructed for the a-carbon coordinates of the input structure considering the cutoff of 16 Angstrom.
More details on calculation of fluctuations from ENMs (both Anisotropic and Gaussian Network
Models) can be found in (Yang al. 2009
C FINER-GRAINED ELASTIC NETWORK MODELS
The complexity of the ENM model can increase by considering different types beads and springs (i.c.. each amino acid and all the possible combinations of pairs) or making them finer-grained (i.c., considering more than one bead per amino acid), approaching the model the atomistic resolution. The order of the algorithm increases at least quadratically with the complexity of the model, mak- ing these solutions more accurate but less efficient. Some implementations modified ENMs are available in ENCoM (Frappier & Najmanovich 2014) and DynaMut (Rodrigues et al. 2021).
D THE DIFFICULTY OF LEARNING FROM B-FACTORS COMPARED TO MD TRAJECTORIES
We also investigated the potential of B-factors to be used as data for learning to predict protein flexibility. To this end, retrained Flexpert-Seq using B-factors instead of the RMSFs from MD trajectories. Because the absolute values of B-factors strongly depend on the calibration of the experiment, we trained both on the raw B-factors well as on the B-factors standardized over each protein chain separately, using the following equation:
B; B B
Buorm
(11)
oBB)
where B; is the raw B-factor of residue i, B E By is the average raw B-factor of a protein
of size N and oB= = V N- N B, B) is the standard deviation of the raw B-factors across
protein i.
Table reports the performance of Flexpert-Seg trained to predict B-factors and normalized B- factors. We the performance is significantly compromised compared to the results of Flexpert- Seq trained on MD data (Pearson correlation coefficient to MD of 0.76). We suspect that the exper- imental data suffer from the crystal packing effect, and they are too noisy reliably learn from.
17

---

Published as a conference paper at ICLR 2025
Table 6: Pearson correlation coefficients between mean square fluctuations (RMSF) from molecular dynamics (MD) simulations (row) and flexibility predictions obtained by Anisotropic Network Models (ANM) over differently processed input structures (columns).
ANM (MD str.)
ANM
ANM
ANM (pMPNN ESMFold str.)
(PDB str.)
(ESMFold str.)
MD
0.77
0.73
0.73
0.71
E THE EFFECT OF TYPE OF INPUT STRUCTURE ON THE PERFORMANCE OF ANISOTROPIC NETWORK MODELS
Table 6 shows that the PDB structure obscures some of the flexibility, probably due to the crystal packing effect present in most cases. Similarly, the ANM performance is diminished when struc- ture obtained by ESMFold is given on the input. Interestingly, preceding the ESMFold by applying ProteinMPNN to generate an alternative sequence leads to only small additional drop in perfor- mance. This is explained by the fact that the standard ANMs implemented in ProDy (Bakan et al.) 201T], which we used, do not consider the amino-acid identities and only work with the coordinates of the backbone. And since ProteinMPNN is optimized to preserve the backbone geometry, it has only a marginal effect on the performance of ANMs.
F EVALUATION OF FLEXPERT-3D AND FLEXPERT-SEE ON MDCATH DATASET
To evaluate the generalization capability of Flexpert-3D and Flexpert-Seq predictors outside of the ATLAS dataset, we performed an additional experiment evaluating the predictors on the mdCATH dataset (Mirarchi al. 2024).
The recently introduced mdCATH dataset presents a challenging test of generalizability for our pre- dictors trained on the ATLAS dataset. The mdCATH dataset larger in terms of the number of data points (5398 compared to 1390 in ATLAS) and the simulated data correspond to individual protein domains (i.e., they are shorter proteins, on average with 137 residues compared 236 residues in ATLAS). Furthermore, the simulation times in mdCATH are longer than in ATLAS (on average 464 ns for mdCATH compared 100 ns for ATLAS) and the simulations were performed different temperatures (320 K, 348 K, 379 K, 413 K, and 450 K) than the simulations in the ATLAS dataset (300 K). The results of the evaluation are presented Table
Table 7:
Evaluation of Flexpert-3D and Flexpert- predictors on MD simulations from the md- CATH dataset. Both predictors were evaluated the full dataset (5398 proteins, columns "Full data") as well as on its subset excluding topologies present in the ATLAS training set (4013 pro- teins, columns "Topo. filtered")). The reported numbers are Pearson correlation coefficients between the model predictions and the RMSF from the dataset averaged over all the proteins in the dataset.
Flexpert-3D
Flexpert-Seq
Simulation temperature
Full data / Topo. filtered Full data / Topo. filtered
T 320 K
0.69 / 0.66
0.64 / 0.64
T = 348 K
0.64 / 0.63
0.63 / 0.63
T 379 K
0.59 / 0.57
0.60 / 0.59
T = 413 K
0.49 / 0.48
0.52 / 0.52
450 K
0.34 0.33
0.38 / 0.38
Despite some performance drop with respect to the performance achieved on the ATLAS dataset (0.82 for Flexpert-3D and 0.76 for Flexpert-Seq). the performance our predictors remains rea- sonable for the simulations performed at the temperature T = 320 K (0.66 for Flexpert-3D and 0.64 for Flexpert-Seq. for the topology filtered dataset). Considering that the temperature in this part of the mdCATH dataset is 20 K away from the temperature of the simulations in the ATLAS training
18

---

Published as a conference paper at ICLR 2025
Flexible
RMSF (Ground Truth)
ANM (R=0.78)
Flexpert-Seq (R-0.91)
Flexpert-3D (R-0.95)
Rigid
Figure 4: ATLAS datapoint Sjca.S. Flexpert-See and Flexpert-3D seem to improve over the predic- tion of ANMs mainly by correctly capturing the higher flexibility of the helix at the terminus.
set, this appears as reasonable generalization. We also notice that Flexpert-3D experiences more dramatic decrease in performance than Flexpert-Seq for higher temperatures in mdCATH simula- tions, suggesting that the sole reliance on sequence information makes the method more robust to the change the simulation temperature. In addition, the proteins are possibly getting unfolded in the high-temperature simulations of mdCATH, leading to non-physiologica degrees of flexibility.
G QUALITATIVE ANALYSIS OF FLEXPERT-SEO AND FLEXPERT-3D FLEXIBILITY PREDICTIONS
In this section, present some qualitative examples of Flexpert-Seq and Flexpert-3D predictions in the form of visualization of protein structures colored by the flexibility. The visualized flexibility predicted by Flexpert-Seq and Flexpert-3D estimated by MD (considered the ground truth) or by Anisotropic Network Models (ANM). See Figures to
H BALANCING SEQUENCE RECOVERY AND FLEXIBILITY AWARENESS IN TLEXPERT-DESICH TRAINING
where 0 E [0, 1] is the parameter balancing the losses. We studied the effect of 0 sequence recovery of the fine-tuned ProteinMPNN model. On one hand, we want f to be as high as possible to focus on learning flexibility awareness; on the other hand, we still need the fine-tuned inverse folding model to retain the sequence recovery so that it still has its inverse folding ability. We observed drop of only one percentage point terms of sequence recovery, when we trained with 0 0.8, with the recovery further dropping for higher 0, see Table g Therefore, we selected 0.8 as the parameter that still sufficiently maintains the original performance of the vanilla ProteinMPNN.
When training Flexpert-Desigee using the combined loss LFleapert = 0- O--
When balancing the Lyteapert loss, also tried to "effectively normalize" the L and Lseg losses to have better control with the parameter 6. This we did by normalization of the gradients in the backward pass and then using the parameter 0 to mix the level of gradients instead of the level of losses. Such training, however, exhibited high instability, so we did not use for the final version.
19

---

Published as a conference paper at ICLR 2025
Flexible
RMSF (Ground Truth)
ANM (R=0.54)
Flexpert-Seq (R=0.80)
Flexpert-31 (R=0.77)
Rigid
Figure 5: ATLAS datapoint 1c52. A. Anisotropic Network Models perform relatively poorly, which likely propagates to Flexpert-3D causing it to underperform (Pearson R 0.77) with respect to Flexpert-Seq (Pearson R = 0.8).
Flexible
RMSF (Ground Truth)
ANM (R=0.88)
Flexpert-Sea (R-0.28)
Flexpert-311 (R=0.59)
Rigid
Figure 6: ATLAS datapoint 6o2v A. This failure case for Flexpert-Seq (Pearson 0.28), which predicts one beta strand together with the terminal coil to be more flexible than what was observed in MD simulation (see RMSF subfigure). Anisotropic Network Models perform better in this case, possibly helping Flexpert-3D improve over Flexpert-Seq.
20

---

Published as a conference paper at ICLR 2025
Flexible
RMSF (Ground Truth)
ANM (R=0.87)
mutmuu
AuAAAn
Flexpert-Seq (R=0.75)
Flexpert-3D (R=0.92)
mumati
Rigid
Figure 7: ATLAS datapoint legw_B. BB. This datapoint features relatively high variance in the flexibil- ity, posing challenge for Flexpert-Seq (Pearson R 0.75). The Anisotropic Network Models work better (Pearson R=0.87) and presumably help Flexpert-3D to excel (Pearson R=0.92).
Table 8: Effect of the parameter 0 for mixing of sequence recovery and flexibility awareness on the sequence recovery the fine-tuned ProteinMPNN model. The vanilla ProteinMPNN is denoted as P. p denotes Flexpert-Design trained with 0 value of X.
P
po-u
Pf PB-0.95
Sequence recovery
0.49
0.48
0.48
0.47
0.47
Training and fine-tuning. We tried train Flexpert-Design from scratch using the Lleeperrt loss, but it seemed that the LPter loss was not informative in the early stages. Therefore, we decided to first train ProteinMPNN in a standard way for 100 epochs on topology split CATH4.3 dataset. Second, we finetuned the model for 12 hours on GPU using the Lilexpert which resulted in additional 8-1 epochs.
I EVALUATION OF STRUCTURE PRESERVATION IN FLEXPERT-DESIGN GENERATED SEQUENCES WITH INCREASED FLEXIBILITY
To evaluate how Flexpert-Design alters the structure of the proteins by the engineering of the in- creased flexibility, we predicted the structures of the sequences with engineered flexibility using AlphaFold2 inside Colabfold (Jumper et al. 2021 Mirdita al. 2022). We compared the predicted backbone structures with the input backbone structure and evaluated the confidence of the structure prediction. The experiment was run for 700 randomly selected proteins from the CATH test set, for which ran the Flexpert-Design predictions with the flexibility increasing instructions and the segment length S=50 (the same choice as in the experiment reported in Figure and Table
The predicted sequences root mean square deviation between the structures (RMSD) (Table 9). Furthermore, the pLDDT confidence overall across the whole protein and separately inside and outside the engineered region (where the flexibility increasing instructions were provided) was analyzed, see Table
compared to the ground truth structures at the level of C., using the
We that the feexpert-Designn generated sequences show lower recovery of the ground truth struc- ture compared to ProteinMPNN, which could suggest that Flexpert-Desie produces less stable structures that it introduces notable conformational changes. A particularly significant drop in
21

---

Published as a conference paper at ICLR 2025
Table 9: Evaluation of the Flexpert-Design generated protein sequences using the alignment of the folded backbone structures to the ground truth backbone structures (RMSD column) and the AF2 predicted pLDDT of the whole protein, the engineered region (eng. region) and of the outside the engineered region (non-eng. region).
Model
RMSD (Å)
1 (whole protein)
PLDDTT
9 pLDDT's 1 (non-eng. region)
(eng. region)
ProteinMPNN
2.00
0.77
0.81
0.73
Flexpert-Dessee (no loss)
3.31
0.61
0.60
0.62
Bexpert-Desipee (ours)
3.36
0.60
0.58
0.63
Median cnrichment ratio Med (ri))
1.5
1.5
1.4
1.4
1.3
1.3
ProteinMPNN (vanilia)
Fiexpert-Dees Sespert-Design (ours)
1.2
1.2
Flexpert-Dee Deeernn (wlo loss)
1.1
1.1
0
2
4
6
8
10
20
40
60
80
100
Parameter
Segment length (S]
Figure 8: The effect of the flexibility increasing parameter 7 (left) on the median enrichment ratio of our Flexpert-Design model and its variant trained without Fite compared to the vanilla Protein- MPNN as a baseline. The effect of the length of the engineered segment on the median enrichment ratio (right).
the PLDDT is in the engineered regions (which tended be well-structured in the ground truth, as suggested by the high PLDDT for ProteinMPNN sequences). which might hint at the increased flexibility (which was shown by our predictor Figure and Table 4 causing the drop in PLDDT and increase in RMSD.
J THE EFFECT OF FLEXIBILITY INSTRUCTION VALUE AND THE SIZE OF THE ENGINBERED SEGMENT ON FLEXIBILITY ENRICHMENT
The effect of the size of the engineered segment |S] and of the parameter used for increasing the flexibility is presented in Figure The graphs suggest that our model achieves the best median enrichment ratio in the setting
At the same time, for parameters within the range E [2.5, 10) and |S| 110, 100], the results of our model are comparable to the optimal setting demonstrating reasonable robustness of the model setting these parameters.
S 50 and 7
K LIMITATION: ENGINEERING DECREASE IN FLEXIBILITY
Analogically to the flexibiliy-innrraaiigg increasing experiment presented in Section 5.2 also investigated
the possibility of decreasing flexibility using our approach with parameter 7 0. We present the results for [S] 50 and -10 in Table 10
22

---

Published as a conference paper at ICLR 2025
Table 10: Comparing the ability of flexibility engineering models decrease the flexibility of engi- neered segments. The presented median enrichment ratios Med (r(y)) and the proportion of flexibility decreasing mutations were obtained using the CATH4.3 test set with engineered segments of length |S| = 50, and flexibility engineering instructions corresponding to native flexibility incremented by 10 in the engineered segment. The best results each category are highlighted in bold, second best are underlined.
Median enrich. ratio 4.
Prop. of flexibility decreasing mutations
Flexpert-Design (MSE loss) Flexpert-Design (L1 loss) Flexpert-Design (w/ loss)
1.06
0.40
0.99
0.51
1.04
0.46
ProteinMPNN (vanilla)
1.07
0.39
Table 10 also presents the variant of Flexpert-Design trained using LI loss instead of MSE, which seems to work better for the case of decreasing flexibility. However, despite achieving modest decrease in flexibility, we consider this task as a limitation of Flexpert-Design. Our explanation is that Flexpert-Design is biased toward positive flexibility engineering instructions because it is trained on the native distribution of flexibilities, which are mostly positive as they approximate the RMSF quantity, which by definition non-negative. Similarly, we suspect that the Flexpert- 3D predictor might be slightly biased toward overpredicting flexibility since it predicts a flexibility increase even for the vanilla ProteinMPNN model. The possible reason is the effect of outliers in the training set. Outliers can exist in the dataset for positive values, but cannot exist for negative due to the non-negative nature of RMSF. Such disbalance can potentially skew the learning distribution toward higher values.
23


Reviews:
[{"review_id": "hWSDiGqqTZ", "reviewer": "Reviewer_fhjb", "summary": "This paper proposes a method called Flexpert-Design to enable guidance of inverse folding models toward increased flexibility. The authors first quantify protein flexibility and identify data relevant to learning, then train flexibility predictors with protein language models. Finally, they fine-tune inverse folding models to steer it toward desired flexibility.", "questions": "Here are some questions **from weaknesses**.\n\n(1) Since estimation with Gumbel-Softmax trick is not extremely accurate, how does this method compared to other RL-based methods with the L\\_flex as the reward?\n\n(2) Why you choose ProtTrans as the backbone, do you try ESM series?\n\n(3) Can the proposed method be applied to any inverse folding models?\n\nAnd here are questions **about experiment details and considerations**.\n\n(4) In Table 1, how do you match the RMSF of MD with other methods’ metrics (like plddt of AF2 and ESMFold2)? And in Sec. 3.1, you mention that MD is considered an accurate physics-based method. Is that a common sense? If so, why include the whole experiment in Table 1?\n\n(5) How do you get the average PCC upper bound of 0.88? Is that for MD? Does it make sense to set an average value as the upper bound?\n\n(6) How do you prove that the protein sequences designed by Flexpert-Design are natural (or useable) other than with good flexibility only? Since you mentioned that the loss function makes a balance between flexibility and sequence recovery, is it worth sacrificing the overall quality of proteins to enhance their flexibility?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- **New field scenario**: Protein flexibility is rarely considered by current protein engineering methods.\n\n- **Reasonable baseline settings**: The authors arange a reasonable set of flexibility-related baselines.\n\n- **Quantified metric evaluation**: The authors apply several methods to quantify protein flexibility comprehensively.", "weaknesses": "- The technical novelty is somewhat limited. While the problem is important, the overall framework is simple and trivial.\n\n- Flexpert-Design uses Gumbel-Softmax trick to enable back propagation. However, this estimation is not extremely accurate.\n\n- Undiscussed choice consideration for the backbone. \n\n- The experimental results are not comprehensive. Only flexibility-related metrics are shown. Besides, only ProteinMPNN is considered as the inverse folding model.\n\n- Lack of evaluations on the overall protein quality (reliability, plausibility, etc.) for the sequences generated by Flexpert-Design. This can be an important proof of its practical application ability\n\n- No clear explanation for the engagement of the proportion of flexibility-increasing mutations (What is its meaning? What properties and biological representations does it reflect?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TT6ymziFWQ", "reviewer": "Reviewer_i3SF", "summary": "the paper address an interesting problem where one wan to generate novel proteins while accounting for protein flexibility—a feature that is critical for protein function but has proven challenging to model and integrate due to data scarcity, heterogeneity, and the high costs of data generation.  The paper provide benchmark dataset and ways to quantify flexibility. They  adapt the inverse folding model for flexibility, guiding protein sequences towards targeted flexibility in specific regions.", "questions": "1. Given the relatively small ATLAS dataset, how does the predictor generalize to other protein datasets? Has the model been tested on proteins outside the ATLAS dataset to evaluate robustness?\n2. I am wondering if the proteins generated  by this model (due to the flexibility-aware nature of the model) show evidence of functional improvements (e.g., improved stability, interaction specificity), or is the flexibility change primarily theoretical?\n3. The Flexpert loss function combines flexibility and sequence objectives with a mixing parameter, θ. How sensitive is the final model performance to different values of θ, and how was the optimal value chosen?", "limitations": "", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The study provides a robust review of various protein flexibility quantification methods and assesses the utility of diverse data sources. \n2. The development of Flexpert-Design to adjust flexibility in specified regions is significant, especially for applications in enzyme engineering, drug design, and biocatalysis.", "weaknesses": "1. A major limitation is the lack of extensive experimental validation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E8Smw4XKhZ", "reviewer": "Reviewer_DNnV", "summary": "In this paper the authors investigate the correlation of various in silico proxies to root mean square fluctuations of protein structures derived from molecular dynamics calculations. They posit that augmenting inverse folding methods for protein design to guide for RMSF as a proxy for flexibility is desirable for design applications. To this end, they introduce Flexpert-Seq, Flexpert-3D, and Flexpert-Design. They demonstrate that supervised models outperform other methods for predicting RMSF from sequence alone, and that they can guide inverse folding design to favor more flexible residues.", "questions": "Can the authors support the statement in the abstract that accounting for flexibility is the major limitation of current protein design methods? There are many limitations of current methods; flexibility is certainly an important one, but it is not clear that it is a singular limitation.\nCan the authors show some failure cases where predictions from Flexpert-Seq do not agree well on the ATLAS dataset? I’m curious if there are certain classes of protein or biophysical features that are difficult to predict well.\nThe enrichment for A, S, T, and G residues and depletion of bulkier residues on flexibility guidance makes sense - is there a “self-consistency” check that ensures that the guided designs retain the desired predicted backbone structure? How does Flexpert-design compare to running ProteinMPNN and using protein language model likelihoods to identify residues where mutations to smaller, more flexible residues are likely to be tolerated?\nRelatedly, per-residue RMSF seems to be a “local” notion of flexibility, which causes a straightforward enrichment of a few specific, small residues. Can the authors imagine ways to include a more non-local measure of flexibility that may induce larger or smaller fluctuations across a biomolecule, without dramatically changing the conditional amino acid distribution?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The paper introduces an interesting problem, explores the problem and relevant dataset before diving into the methodology, and shows that simulated dynamical properties of proteins can be predicted from sequence alone, or sequence and simple structural featurizations, offering important advantages in speed and use-of-use compared to other methods.\nThe discussion of sequence recovery as an incomplete evaluation of inverse folding methods, and the predominant usage of ProteinMPNN due to its extensive wet lab validation, is appreciated.\nThe Table 3 and 4 results are nice, clear demonstrations that the authors’ hypothesis is correct, and that a supervised model outperforms more indirect methods like pLDDT and ANM for predicting RMSF values.", "weaknesses": "The authors might provide a concise definition of “flexibility” earlier on in the paper. Flexibility could mean many things and be quantified and measured different ways, experimentally and in silico, as detailed in Section 3.1. \nSome discussion is needed of the incompleteness of the methods introduced in Section 3.1. pLDDT and B-factors can be confounded by many other covariates that are not flexibility, and RMSF calculations from MD may not accurately reflect the real world biophysics of dynamic proteins in a physiological context, where interactions with solvent and other biomolecules will affect effective flexibility. \nA small nit, say “MD trajectories” rather than MDs to clarify the source of reference datasets.\nI would recommend capturing all the newly introduced methods under subheadings under a Methods section heading, and then going into results. With the current structure I expected to see Results after Section 4, but Section 5 introduces another method.\nWithout simple baselines (some ideas provided in the Questions below) it is difficult to assess the results presented in the design section.\nThe Conclusions section restates what was done, which is covered in the intro. Discussion of limitations and future directions would be appreciated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OpHq3EFih6", "reviewer": "Reviewer_R4S7", "summary": "The manuscript presents an deep learning based approach to predict protein flexibility and engineer protein sequence towards more flexibility. There are three steps in the manuscript. The first step is to compare metrics that quantify protein flexibility. The second step is to repurpose pre-trained protein language model for flexibility prediction. Here, the authors trained Flexpert-Seq (no 3D structure input) and Flexpert-3D (with 3D structure input) for the regression task. Last but not the least, the authors proposed a framework to train ProteinMPNN with flexibility instructions to generate more flexible protein sequence given the a input 3D structure.", "questions": "1. How is RMSF calculated? It would be better to note the RMSF equation in the manuscript.\n2. Line 152-153, author claims higher pLDDT uncertainty indicates lower flexibility, which is kind of counter-intuitive. Can authors elaborate?\n3. When benchmarking the flexibility prediction ability, how are AF2 and ESMFold used? Based on my understanding of the manuscript, the flexibility prediction is a regression task. Did authors attach a prediction head to baseline models? Please elaborate.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The idea of predicting the flexibility of protein given structure or sequence is rather novel. The comparison between the metrics for evaluating protein flexibility is valuable. The framework proposed to inverse fold for better flexibility, Flexpert-Design, can be valuable for specific applications requiring more flexible proteins.", "weaknesses": "1. Writing can be improved. For example, in line 79-80, \"From these simulations, Root Mean Square Fluctuations (RMSF).\" is not a full sentence. Presentation can also be improved. In table 3 and 4, only MD results are bolded, whereas the ANM (MD str.) results are not. Authors can also add arrows to indicate if each metric is the higher the better or the lower the better.\n2. The benchmark needs more explanation, especially the flexibility prediction one. It is not clear to me how AF2 and ESMFold are used for flexibility prediction (the regression task). The interpretation of the benchmarking results (table 3 and 4) is also vague.\n3. There is not enough elaboration on why more flexible protein is needed. One major contribution of this paper is the Flexpert-Design framework, which tunes ProteinMPNN to generate more flexible protein given the prediction of a flexibility predictor. However, it is not clear to me (from reading the manuscript) the purpose of modifying the flexibility of a protein. The authors can focus more on the motivation. \n4. The Flexpert-Design can introduce bias from Flexpert-3D into the ProteinMPNN model through the flex loss. Also, the correlation between the input structure and the flexibility instruction can be low when the flexibility instruction is artificially increased, i.e. the input structure can never have high flexibility given by the flexibility instruction. In such case, the prediction of both flexibility instructed ProteinMPNN and Flexpert-3D can be unreliable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)