EnzymeFlow: Generating Reaction-specific Enzyme Catalytic Pockets through Flow Matching and Co-Evolutionary Dynamics

Authors: Chenqing Hua, Yong Liu, Dinghuai Zhang, Odin Zhang, Sitao Luan, Kevin K Yang, Guy Wolf, Doina Precup, Shuangjia Zheng
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
Keywords: enzyme design; flow model; protein design; protein evolution

Abstract:
Enzyme design is a critical area in biotechnology, with applications ranging from drug development to synthetic biology. Traditional methods for enzyme function prediction or protein binding pocket design often fall short in capturing the dynamic and complex nature of enzyme-substrate interactions, particularly in catalytic processes. To address the challenges, we introduce EnzymeFlow, a generative model that employs flow matching with hierarchical pre-training and enzyme-reaction co-evolution to generate catalytic pockets for specific substrates and catalytic reactions. Additionally, we introduce a large-scale, experimentally validated dataset of enzyme-reaction pairs, specifically designed for the catalytic pocket generation task, comprising a total of $328,192$ pairs. By incorporating evolutionary dynamics and reaction-specific adaptations, EnzymeFlow becomes a powerful model for designing enzyme pockets, which is capable of catalyzing a wide range of biochemical reactions. Experiments on the new dataset demonstrate the model's effectiveness in designing high-quality, functional enzyme catalytic pockets, paving the way for advancements in enzyme engineering and synthetic biology. The EnzymeFlow code can be found at https://anonymous.4open.science/r/EnzymeFlow-7420.

Decision: reject
Reviewer scores: 3, 6, 6, 3

Reviews and discussion:
<thread>
creation date:
1730381189778

thread_type:
official_review

<thread_object>
creator: reviewer_zztw

<content>
summary:
This paper proposes a new model named EnzymeFlow, designed to generate enzyme catalytic pockets for specific substrates through flow matching and enzyme-reaction co-evolution. The model employs a hierarchical pre-training strategy, initially pre-trained on protein backbones and ligand-binding pockets, followed by fine-tuning on enzyme catalytic reactions. The authors also constructed a large-scale dataset named EnzymeFill, containing over 320,000 enzyme-reaction pairs, to support model training. Experimental results indicate that EnzymeFlow performs well in generating high-quality catalytic pockets, showing its potential in enzyme engineering and synthetic biology.

strengths:
This paper provides a dataset for enzyme pocket design tasks and details the workflow for collecting and cleaning the dataset. It adapts and innovates upon the previous ideas from PocketFlow for the enzyme pocket design context by incorporating 3D and 2D molecular information of substrates and product molecules at the encoding level, thereby describing the dynamic catalytic process. The paper also introduces a new approach to constructing and encoding MSA co-evolutionary information for enzyme catalysis, offering new ideas for future developments in this field.

weaknesses:
The design of certain modules and the loss functions raises questions about their logical significance. Specifically, Equation (5) describes the score function loss for enzyme-reaction co-evolution information. Reconstructing MSA from its noisy encoded form would certainly make sense, but in the code implementation, the MSA encoding is only activated during MSA diffusion. Ultimately, it compresses $N_{MSA}$ into a single sequence representation (MSA + bb_lig_fusion_emb) to reconstruct $N_{MSA}$, raising doubts about the validity of reconstructing high-dimensional data from such a low-dimensional form.

Moreover, both MSA and EC class, as implemented in the code, function more like two output heads in a multi-task setup. These significant conditional features are not integrated early into the IPA network but are instead fused after the IPA outputs the updated structure, making it difficult for these features to directly influence IPA’s backbone design. This makes the description in Equation (6) misleading, as it may lead readers to believe that these conditional features have a substantial multimodal influence on backbone design. If the authors had included an architecture of the model's computation flow, it would be clear that the feature paths of MSA and EC class are almost entirely separate from the core backbone + substrate + product molecule paths. In Figure 3.3, the influence of EC class and co-evolution on the gradient is minimal, more akin to decoding pre-designed structure embeddings to predict MSA and EC class.

Additionally, a long-standing issue in this field is that enzyme pocket design is distinct from general protein design, and in silico metrics alone are insufficient to establish the practicality and reliability of newly designed enzymes. Only wet lab experimental results serve as the gold standard for this task.

questions:
1. According to the code, it appears that providing MSA and EC class during sampling does not influence IPA's backbone design, nor are these features fed back as self-conditioning. Do these tasks merely help the model learn a better landscape during training?
2. Why aren't the features of MSA and EC class integrated into the backbone embedding before entering IPA?
3. Continuing from question 1, how is the multi-label classification for EC class specifically conducted? EnzymeFlow uses ec_pred for class prediction, while the comparison method uses the CLEAN model to predict EC class from the generated structure. Is this a fair comparison?
4. Continuing from question 3, the ec_embed feature of EC class does not enter IPA, so how is it ensured that the generated structure matches the given EC class during sampling?
5. How were the comparative methods set up? Does Table 2 include the performance of PocketFlow fine-tuned on enzyme data?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730561967083

thread_type:
official_review

<thread_object>
creator: reviewer_xpne

<content>
summary:
This paper proposes a model to design enzyme pockets conditioned on reactions. It is a flow-matching model to co-generate enzyme pocket structures, sequence, ec class, and MSAs of enzymes and reaction smiles. Their model is pre-trained on the protein structure dataset and protein-ligand binding dataset. Empirical study shows that their method outperforms other baselines in generation quality and function prediction.

strengths:
(1) This work effectively integrates various resources - such as data, pre-training, and co-evolutionary information - to enhance enzyme design performance. 

(2) The authors curate a dataset tailored for enzyme pocket design, which could serve as a crucial resource for advancing predictive models in this field.

weaknesses:
(1) mismatch in introduction and method

The introduction emphasizes the importance of capturing chemical changes specific to enzyme generation, distinguishing it from docking tasks that do not consider reactions. However, the method section lacks sufficient detail on how reaction-specific modeling is incorporated. The approach appears closer to a docking setup, focusing primarily on backbone structures, with limited attention to sidechain interactions critical for catalytic activity.

(2) concerns about co-evolutionary dynamics

The sequence alignment approach applied to SMILES representations raises concerns, as small molecules are inherently non-sequential, and different SMILES can produce varying alignments. 

(3) evaluation of functions

The rationale behind the ECAcc evaluation metric requires further clarification, as its relevance to enzyme function is not immediately clear. 

(4) evaluation of structures

The curated enzyme pocket dataset is a valuable resource. However,  If I did not misunderstand the curation process,  the binding structures were computationally predicted, and there may be inherent biases from the prediction model. Including evaluations on non-synthetic datasets could help mitigate any potential bias and validate the robustness of the proposed methods.

questions:
* About the ECAcc

First, the use of ECAcc as a metric for evaluating enzyme functionality requires additional justification. It is unclear how ECAcc is directly correlated with catalytic ability, and including references that support this metric’s relevance to enzymatic activity would help strengthen its validity as a functional evaluation.

Second, it would be insightful to understand the model’s performance when the EC class is not co-generated during training. If this information is not utilized by other baselines, the comparison may not be entirely fair.

Third, the reported accuracy (~0.2) appears relatively low, suggesting potential challenges in accurately predicting EC classes. An explanation of these challenges would be helpful.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730635881387

thread_type:
official_review

<thread_object>
creator: reviewer_xrvg

<content>
summary:
This paper introduces a novel approach to enzyme pocket design, emphasizing functional relevance from the outset. The authors propose EnzymeFlow, a flow matching model conditioned on both substrates and products, trained to predict enzyme pocket structure, sequence, EC-class, and enzyme-reaction co-evolution concurrently. They introduce EnzymeFill, a newly curated dataset tailored for the enzyme pocket design task, comprising enzyme-reaction pairs with precise pocket structures. Experimental results provide substantial evidence of EnzymeFlow's effectiveness in generating functionally reliable enzyme pockets that align with catalytic requirements.

strengths:
- EnzymeFlow shifts the focus from static ligand-protein interactions to more complex enzyme-substrate interactions, a more functionally relevant framework for catalysis. This paradigm is well-motivated.
- EnzymeFlow proposes a distinct formulation for enzyme design that departs from traditional protein design workflows, emphasising the importance of function in modelling.
- The proposed method is technically sound.

weaknesses:
- Despite the promising performance on pocket design, the utility of EnzymeFlow in whole enzyme design remains under-explored, limiting insight into its full application potential.
- As the experimental results show, although EnzymeFlow achieves better functional performance compared to other baseline methods, the ECacc is still somewhat low to conclude that EnzymeFlow adequately captures the catalytic requirements.

questions:
1. Could the authors clarify the rationale for focusing on a pathway from pocket to motif-scaffolding rather than complete enzyme design?
2. Testing the full protein configurations by integrating EnzymeFlow with recent motif-scaffolding methods or inpainting methods and comment on the self-consistency of the designed pockets would be helpful in understanding the application potential.
3. In Table 2, the comparison with RFDiffusionAA may not be ideal as RFDiffusionAA is designed for whole proteins rather than specific pockets. It may be more appropriate to extract the pocket from a whole protein generated by RFDiffusionAA and compare it with EnzymeFlow.
4. Could the authors provide an analysis of EnzymeFlow's performance across different substrate classes?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730662254220

thread_type:
official_review

<thread_object>
creator: reviewer_yzem

<content>
summary:
EnzymeFlow introduces a new approach to conditionally generate enzyme pockets conditioned on the reaction of interest. They use a modified MultiFlow approach that is conditioned on both substrates and products, they use an auxiliary EC flow, and they use a novel reaction-protein coevolution MSA to enhance the generation process. The evaluate the model across a number of metrics that are considered relevant for enzymatic activity including cRMSD and TM-score of the pockets; predicted binding affinities, chai score, amino acid recovery and predicted EC accuracy.

strengths:
The paper attempts to solve a key task on the road to de novo enzyme design -- namely generating enzyme pockets which are conditioned on the reaction of interest. The paper has a number of strengths:
1. The authors present an interesting idea -- that enzymes and their associated reactions co-evolve. They attempt to use this hypothesis to learn representations of this co-evolution to enhance generation.
2. They introduce a new dataset of enzyme pockets. 
3. They combine a number of methods applied in different applications on a new application.

weaknesses:
The main weakness with the paper is the results:
1. It is unclear whether the improvements presented by the authors are significant or not. Please provide confidence intervals such that it is easier to evaluate the significance of the results. 
2. It is unclear to me whether these scores meaningfully correlate with enzyme function. For example, TM-scores of below 0.3 represent quite a significant difference from the ground truth structure, meaning that the generated pockets are still quite different. Why is this? Furthermore, it seems that the generated pockets recover the correct amino acids a small fraction of the time and additionally only have the same predicted EC a small fraction of the time. A strong result would be that generated structures are similar to ground truth, have close or exact amino acid identities and (if they are truly functionally similar) should have the same predicted EC numbers.
3. The novel contributions of the author seem to also have very little impact on performance -- are the results w/o coevolution, pretraining or both statistically significant? 
4. While I rather like your attempt to incorporate co-evolutionary reaction-enzyme information, I am not convinced that SMILES of reactions is a good approach as small changes in molecules can have large effects on SMILE strings. Are there alternative ways of doing this? In how many cases do SMILES strings change significantly across different proteins in the same MSA? 
5. Size of evaluation set.

questions:
I will be happy to review my score if the authors can answer the above questions convincingly as well as provide the following additional information:
1. Please clearly indicate what novel contributions come from the authors versus have been taken from other papers and combined. For example, it seems that most of the generative model is based on MultiFlow and not introduced by the authors. Is the approach for ligand-conditioning developed by you or by others? The co-evolution for example (to the best of my knowledge) is a novel contribution of the authors. 
2. Please provide a convincing argument to the significance of the results and an explanation as to why the numbers being so low is acceptable. For example, would you expect pockets generated by your model to be active? 
3. You claim that other approaches simply consider static behavior of proteins and their ligands, but I do not see any consideration of dynamic behavior in your model beyond considering both substrate and product. Do you have reason to believe that your model learns this implicitly? If so please provide evidence of this.
4. How does your model perform using a standard protein MSA compared to your special coevolution MSA?
5. Please provide results for vanilla MultiFlow without any additional elements as a baseline.
6. Please provide a TM-score for the active site against the entire protein, not just the pocket.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper seems to have several significant weaknesses. The most important are
- The unclear role of the EC class as a conditioning factor. As mentioned by some reviewers, the actual source code suggests that  MSA and EC class appear more like two multi-task learning tasks, which also raises questions about the extent to which these "conditional" features can have a substantial influence on the backbone design. 
- The unclear role and importance of a rather general concept such as cross attention for modeling details in chemical reactions.  
- The unclear relevance of experimental results for predicting enzyme functionality.
- The unclear statistical significance of several results.
While some of these concerns could be addressed to some degree in the authors' rebuttal, in the end just too many open questions remained. Therefore, I conclude that this paper in its present form is not ready for publication, and I recommend rejection of this paper.