MetaEnzyme: Meta Pan-Enzyme Learning for Task-Adaptive Redesign

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-21 14:27:22

Abstract: Enzyme design plays a crucial role in both industrial production and biology. However, this field faces challenges due to the lack of comprehensive benchmarks and the complexity of enzyme design tasks, leading to a dearth of systematic research. Consequently, computational enzyme design is relatively overlooked within the broader protein domain and remains in its early stages. In this work, we address these challenges by introducing MetaEnzyme, a staged and unified enzyme design framework. We begin by employing a cross-modal structure-to-sequence transformation architecture, as the feature-driven starting point to obtain initial robust protein representation. Subsequently, we leverage domain adaptive techniques to generalize specific enzyme design tasks under low-resource conditions. MetaEnzyme focuses on three fundamental low-resource enzyme redesign tasks: functional design (FuncDesign), mutation design (MutDesign), and sequence generation design (SeqDesign). Through novel unified paradigm and enhanced representation capabilities, MetaEnzyme demonstrates adaptability to diverse enzyme design tasks, yielding outstanding results. Wet lab experiments further validate these findings, reinforcing the efficacy of the redesign process.

Main paper: 
 introduction: 
 Enzymes, distinguished as specialized proteins, serve as biological catalysts, expediting chemical reactions. Their capacity to catalyze reactions ensures specificity and enables operation under mild conditions, thus playing a crucial role in various industries. [1, 34, 47, 49] . Positioned at the forefront of industrial production and the biological domain, enzyme design involves the deliberate creation of modified variants through functional design, commonly known as protein redesign [6, 21, 37] , based on known structures or sequences. Despite its critical role, computational enzyme design is still in its early stages within the broader protein field. The scarcity of comprehensive enzyme data, coupled with the diversity of enzyme tasks and models [30, 52] , has resulted in a lack of systematic research and oversight in computational enzyme design. The inherent complexity of tasks and the vast diversity of data pose challenges for widespread adoption, contributing to relatively low attention in the enzyme field.To address the challenges inherent in enzyme design, it is imperative to confront issues stemming from data scarcity and model generalization. Enzyme datasets often suffer from smaller scales compared to the broader protein domain due to specific functional categorizations, leading to inadequate model training. To mitigate this challenge, we propose leveraging pretrained universal protein models as intermediaries, replacing direct training of task-specific enzyme functional models. These universal models benefit from more extensive datasets in the general domain, resulting in stronger representation capabilities in both sequence and structural modalities. This could facilitates better domain adaptation to downstream tasks through transfer learning manner.Furthermore, functional enzyme tasks exhibit complex diversities, necessitating different modalities and data requirements, which may require multi-modality pretrained model architectures as drivers. However, this approach is not the more efficient or unified solution. Given our concentration on essential low-resource enzyme redesign tasks-functional design (FuncDesign), mutation design (MutDesign), and sequence generation design (SeqDesign)encompassing the primary tasks and paradigms of contemporary AI research in enzymes, we underscore the significance of incorporating structural modality. This emphasis stems from the recognized principle that structures dictate functions. Taking these considerations into account, we aim to pursue both robust representation and generalization while simplifying the enzyme design framework.Through observing commonalities across all tasks, we propose a novel MetaEnzyme framework, as illustrated in Figure 1 . MetaEnzyme consists of a foundational universal protein design network (UniProt-Net) and downstream enzyme redesign modules. In the first stage, UniProt-Net undergoes pretraining to acquire robust representation capabilities. We instantiate UniProt-Net with a crossmodal structure-to-sequence network, as illustrated in Figure 2 (a), to bridge the gaps between various redesign tasks. To augment structural and contextual representations, we introduce geometryequivariant modules, implicit energy-motivated data augmentation, and multi-modal fusion, significantly enhancing generalization capabilities for low-resource scenarios. In the second stage, all enzyme redesign modes are driven by the cross-modal UniProt-Net. Based on characteristics such as autoregressive (AR) vs. nonautoregressive (NAR), non-parametric vs. parametric, etc., we rationalize different tasks and adopt meta-learning and domain-adaptive techniques in low-cost and high-efficiency manners.The main contributions are outlined as follows:• The innovative unified enzyme design framework, MetaEnzyme, consolidates patterns across mainstream tasks, effectively leveraging commonalities for enhanced adaptability. The framework facilitates seamless transitions between universal protein design network and in-domain design evaluation tasks.• The proposed geometry-enhanced module, combined with techniques such as data augmentation, multi-modal fusion, significantly enhances representation capabilities for low-resource settings, along with the extension of multiple enzyme tasks.• A novel decoupled mutation scoring method is introduced for evaluating mutation effects, greatly improving efficiency.• Additional wet lab experiments for computational validation. 
universal protein design network 2.1 problem statement: 
 As shown in Figure 2 𝑌 = {𝑌 1 , 𝑌 2 , • • • , 𝑌 𝑛 } ∈ R 𝑛 . And Ŷ denotes the native sequence Ŷ = { Ŷ1 , Ŷ2 , • • • , Ŷ𝑛 } ∈ R 𝑛 .∈ 𝐸 (3), a geometric network 𝜑 is E(3)-equivariant if 𝜑 (T • 𝑋 ) = T • 𝜑 (𝑋 ), and 𝜑 is E(3)-invariant if 𝜑 (T • 𝑋 ) = 𝜑 (𝑋 ).The core component of the GeoStruc-Encoder is the lightweight GVP module [19] , in which the vanilla GVP layers are overall rotation-invariant for rigid bodies since GVP outputs a rotationequivariant vector feature 𝑣 through an equivariant function 𝑓 , and a rotation-invariant scalar feature 𝑠 through an invariant function 𝑔 for each amino acid, thus for any arbitrary rotation R: R 𝑓 (𝑣) = 𝑓 (R𝑣), 𝑔(𝑠) = 𝑔(R𝑠), as shown in Figure 2(b) .Then to make the entire EnAd-Module rotation-invariant, a local reference frame 𝑣 𝑙 [15] is further introduced to fuse with the rotating vector feature 𝑣 ′ , resulting in enhanced rotation-invariant features, which is rotation-invariant for any rotation R: 𝑓 (𝑣 ′ ⊕𝑣 𝑙 ) = 𝑓 (R (𝑣 ′ ⊕ 𝑣 𝑙 )). Additionally, the input features are translationinvariant [20] , making the overall EnAd-Module also translationinvariant. Therefore, the concatenated features are invariant to translations and rotations on the input coordinates. Thus for any rotation/translation T of the input, the output of the entire EnAd-Module 𝑝 𝑒𝑛𝑎𝑑 can be invariant: 𝑝 𝑒𝑛𝑎𝑑 (𝑣, 𝑠) = 𝑝 𝑒𝑛𝑎𝑑 (T (𝑣, 𝑠)). To obtain a better structural representation, we initialize the EnAd-Module parameters as [15] which is trained on millions of proteins.Guided by biochemistry principles, atomic interactions are regulated by forces and energy. This insight inspires the incorporation of energy and force concepts into data augmentation [17, 18] . To achieve this, we introduce an energy-driven Riemann-Gaussian geometric technique, illustrated in Figure 2 (c), which ensures that structural variations maintain the energy dynamics of proteins, preserving pairing relationships and geometric characteristics. In essence, the augmented structures guarantee structural invariance for the spatial distribution of energy or force. In a nutshell, we obtain a noisy sample 𝑋 ′ from 𝑋 according to a certain conditional distribution, i.e., 𝑋 ′ ∼ 𝑝 𝑛𝑜𝑖𝑠𝑒 (𝑋 ′ |𝑋 ). Unlike conventional denoising methods applied to images or other Euclidean data, the introduced noise in our 3D geometry is tailored to be geometry-aware rather than conformationaware, i.e., 𝑝 (𝑋 ′ |𝑋 ) should possess doubly E(3)-invariant:EQUATIONThis is consistent with the observation that the behavior of proteins with the same geometry should be independent of different conformations. A conventional choice of 𝑝 𝑛𝑜𝑖𝑠𝑒 (𝑋 ′ |𝑋 ) is utilizing the standard Gaussian with noise scale 𝜎 as 𝑝 𝑛𝑜𝑖𝑠𝑒 (𝑋 ′ |𝑋 ) = N (𝑋, 𝜎 2 𝐼 ).But this naive form fails to meet the doubly E(3)-invariant property in Eq. 1. Specifically, considering the derived force target∇ 𝑋 ′ log𝑝 (𝑋 ′ |𝑋 ) = -𝑋 ′ -𝑋 𝜎 2 where 𝑋 ′ = R • 𝑋 s.t. rotation R ≠ 𝐼 , then ∇ 𝑋 ′ log𝑝 (𝑋 ′ |𝑋 ) = -1𝜎 2 (R -𝐼 )𝑋 ≠ 0, which imply that the force between 𝑋 ′ = R • 𝑋 and 𝑋 is not equal although the same geometry is shared. Hence, to devise the form with the symmetry in Eq. 1, we instead resort to Riemann-Gaussian [7] as:EQUATIONwhere 𝜁 (𝜎) is the normalization term, and 𝛿 is the metric that calculates the difference between 𝑋 ′ and 𝑋 . Riemann-Gaussian is a generalization version of typical Gaussian, by choosing various distances 𝛿 beyond the Euclidean metric. To pursue the constraint in Eq. 1: where 𝑋 𝑟 = 𝑋 -𝜇 (𝑋 ) shifts 𝑋 to zero mean (𝜇 (𝑋 ) is the mean of 𝑋 's columns). The same transformation applies to 𝑋 ′ 𝑟 . The distance function 𝛿 adheres to the doubly E(3)-invariance as in Eq. 1. Notably, 𝛿 is permutation-invariant concerning the order of the columns in 𝑋 ′ and 𝑋 .EQUATIONThe initialization of the Context-Module, as shown in Figure 2 (d), aims to incorporate prior language knowledge into the modules [51] . The entire Context-Module functions as an encoder-decoder Transformer [45] , denoted as 𝑝 𝑐𝑜𝑛𝑡𝑒𝑥𝑡 with linear output layers. To initialize the Context-Module exclusively based on sequence data from the training set, we employ a sequence-to-sequence recovery task using an autoencoder (AE) mode. This allows the Context-Module to acquire contextual semantic knowledge, utilizing cross-entropy (CE) loss for learning. Formally, given an input protein sequence for the encoder𝑆 in = {𝑠 1 , 𝑠 2 , • • • , 𝑠 𝑛 }with 𝑛 amino acids, and the reference native sequence as𝑆 native = 𝑆 in = {𝑠 1 , 𝑠 2 , • • • , 𝑠 𝑛 }, the AE-based objec-tive is to generate a sequence 𝑆 𝑟 to recover 𝑆 native as accurately as possible:L AE = CE 𝑝 𝑐𝑜𝑛𝑡𝑒𝑥𝑡 (logits 𝑆 𝑟 |𝑆 in ), 𝑆 native .(4)We initialize the parameters of UniProt-Net based on prior knowledge. To enhance the dataset, we incorporate Riemann-Gaussian data augmentation, combining the original protein dataset 𝐷 𝑜𝑟𝑖 with the augmented dataset denoted as 𝐷 𝑟𝑖𝑒 , resulting in a new dataset 𝐷 𝑎𝑢𝑔 = 𝐷 𝑜𝑟𝑖 ∪𝐷 𝑟𝑖𝑒 . In the processing flow, the protein backbone 𝑋 ∼ 𝐷 𝑎𝑢𝑔 is initially input into the EnAd-Module, producing geometric context features. These features are then fed into the decoder to generate the sequence distribution logits 𝑌 and derive the generated sequence 𝑌 as:EQUATIONand cross-entropy loss is used for training: We introduce a task-specific meta-learning framework [5, 11] for the task distribution 𝑇 , outlined in Algorithm 1. The UniProt-Net here is regarded as a function 𝑓 𝜃 with an initial pretrained parameter 𝜃 𝑚𝑒𝑡𝑎 involving linear classification layers. The primary objective of the meta-learner is to acquire updated parameters of 𝜃 𝑚𝑒𝑡𝑎 containing high-order meta-information, facilitating swift adaptation to a new task drawn from 𝑇 . Upon adapting the meta-learner to a new task 𝑇 𝑖 ∼ 𝑇 , the 𝜃 𝑚𝑒𝑡𝑎 undergoes updates, transforming into the taskspecific parameter 𝜃 𝑖 through a few gradient descent iterations. These parameter updates, executed based on the support set 𝑆 𝑖 for 𝑇 𝑖 , are referred to as inner loops. The number of inner loops is denoted as 𝐿 𝑖𝑛 . And loss function J 𝑚𝑒𝑡𝑎 is utilized in the inner-loop updates as:EQUATION 
domain adaption for low-resource enzyme design tasks: 
 𝑖 = 𝜃 -ℓ • ∇ 𝜃 L 𝑇 𝑖 (𝑓 𝜃 ); end for Update 𝜃 ← 𝜃 -ℓ ′ • ∇ 𝜃 𝑇 𝑖 ∼𝑇 L 𝑇 𝑖 (𝑓 𝜃 ′ 𝑖 ); 𝐿 𝑖𝑛 -= 1; end if 𝐿 𝑜𝑢𝑡 -= 1; until 𝐿 𝑜𝑢𝑡 == 0;EQUATIONwhere 𝜃 (𝑡 =0) 𝑖 equals to 𝜃 𝑚𝑒𝑡𝑎 , and 𝜃 (𝑡 ) 𝑖 denotes the fold-specific parameters after 𝑡 inner loops. ℓ is the learning rate. 𝑋 𝑖 , Ŷ𝑖 represents the native structure-sequence pairs, with ĉ𝑖 as labels in 𝑆 𝑖 . The optimization of 𝜃 𝑚𝑒𝑡𝑎 occurs on the query set 𝑄 𝑖 ∼ 𝑄 across tasks 𝑇 during outer loops. Consequently, the update for 𝜃 𝑚𝑒𝑡𝑎 at one step as:EQUATIONwhere ℓ ′ is the learning rate of the outer loop. 𝑋 ′ 𝑖 , Ŷ ′ 𝑖 represent protein pairs with ĉ′ 𝑖 labels in 𝑄 𝑖 , similarly. The meta-objective function J 𝑚𝑒𝑡𝑎 varies across functional tasks. For instance, in binary categorization, it can be expressed as:EQUATIONwhere 𝛾 > 0 denotes an adjustable factor. While in a multi-classification task, it can be expressed as :EQUATION)where 𝐾 is the number of classification categories.Mutation effect prediction is a non-parametric zero-shot learning in the NAR manner. Given the unique architecture of our MetaEnzyme, we introduce a novel mutation effect scoring method that decouples mutation scoring approaches, as illustrated in Figure 3 . Formally, considering a wild-type enzyme structure 𝑋 𝑤𝑖𝑙𝑑 ∈ R 𝑛×3×3 with its corresponding wild-type sequence 𝑌 𝑤𝑖𝑙𝑑 ∈ R 𝑛 , the non-parametric mutation effect prediction task aims to query the mutation-related ranking for the mutation sequence 𝑍 𝑚𝑢𝑡 ∈ R 𝑛 , which is scored based on the marginal probabilities logits 𝑤𝑖𝑙𝑑 and the reference distribution logits 𝑚𝑢𝑡 as:EQUATIONwhere 𝛿 𝑚𝑢𝑡 is a distance measure between distributions of the wild-type and mutant. M=20 corresponds to 20 commonly used amino acid types. 𝑂𝑛𝑒𝐻𝑜𝑡 is one-hot encoding, with 𝑧 𝑖𝑘 ∈ {0, 1} s.t. 𝑧 𝑖𝑘 ∼ 𝑧 𝑖 , 0 ≤ 𝑘 < 𝑀. 𝐿𝑆 denotes the label smoothing function with a constant 𝜖 = 0.1 as:EQUATIONInstantiating 𝛿 𝑚𝑢𝑡 with a weighted cross-entropy function:EQUATIONwhereEQUATIONwhere 𝑀𝑢𝑡𝑆𝑒𝑡 ← (𝑌 𝑤𝑖𝑙𝑑 == 𝑍 𝑚𝑢𝑡 ) ∈ R 𝑛 , ∀𝑀𝑢𝑡𝑆𝑒𝑡 𝑖 ∈ {0, 1}, and 𝑀𝑢𝑡𝑆𝑒𝑡 implies all mutant positions. 𝛼 = 0.5. From Eq. 11, we observe that: 1) the computation of wild-type and mutant proteins is independent; 2) deep network flow (such as UniProt-Net) predicts only the wild-type proteins, requiring a single inference due to the insight that mutant sequences stem from unique wild-type proteins through evolution or modification; 3) large-scale mutant sequences do not necessitate a network model, and they can be converted to a distribution representation based on one-hot encoding simply. This decoupling method provides a significant advantage in terms of speed, particularly for sequence datasets exceeding a million entries and for scenarios involving multi-site/higher-order mutations. Additionally, as the computation process relies entirely on the pre-trained parameters of UniProt-Net and makes no distribution assumptions about the query mutant enzymes, it is non-parametric. Section 4.3.2 presents detailed analysis.For the conditional sequence generation, we shift the entire framework to an AR mode. Specifically, when provided with a candidate structure backbone 𝑋 𝑐 , the objective is to generate an unknown protein sequence 𝑌 𝑠𝑎𝑚𝑝𝑙𝑒 as:EQUATION)This density is represented using a Vanilla Transformer decoding paradigm. AR decoding typically conducts SeqDesign tasks for general proteins and functional enzymes. 
in silico experiments 4.1 setups: 
 Training Set. CATH dataset [16] is divided into training, validation, and testing sets, containing 18,204, 608, and 1,120 structuresequence pairs, respectively. The training set is utilized to train the UniProt-Net in the pretraining stage. Evaluation Sets. FuncDB 𝑏𝑖 : Approximately 700 enzymes with experimentally determined structure-sequence pairs across 10 folds from RCSB are collected. Each fold is crafted with an equal number of enzymes and balanced with non-enzyme negative samples. This dataset is used for enzyme function prediction and design. FuncDB 𝑚𝑢𝑙𝑡𝑖 is a subset of FuncDB 𝑏𝑖 dataset, comprising solely positive samples, for multi-class functional prediction tasks-precisely predicting the fold level to which an enzyme belongs. MutDB 𝑃𝑟𝑜𝑡𝐺 𝑦𝑚 [36] is intricately designed for protein fitness, encompassing an expansive collection of over 217 deep mutational scanning assays and millions of mutant sequences. SeqDesignDB 𝑃𝑒𝑡 comprises 178 proteins identified for the capability to degrade plastics [12] . SeqDesignDB 𝐻 𝑦𝑏𝑟𝑖𝑑 is the alias of FuncDB 𝑚𝑢𝑙𝑡𝑖 for differentiation. SeqDesignDB 𝐴𝐿𝐿 is extracted from the testing set of CATH. SeqDesignDB 𝑇 𝑠50 and SeqDesignDB 𝑇 𝑠500 correspond to Ts50 and Ts500 [26] for validating generalization.Implementation. The AdamW optimizer with a batch size of 5 and a learning rate of 1e-3 to train the UniProt-Net. The GeoStruc-Encoder consists of 4 layers with a dropout of 0.1. The node hidden dimensions of scalars and vectors are 1024 and 256. The adapter and decoder have 8 multi-head self-attention layers, with an embedding dimension of 512 and an attention dropout of 0.1. For meta-learning configures, the number of inner loops is set to 5, and the batch sizes of the support set and query set each are both 10. The updated learning rates 𝑙 ′ and 𝑙 are set to 1e-3, and the Adam optimizer is used. The parameters of EnAd-Module are fixed. And 1 NVIDIA A100 80GB GPU is used.(FuncDesign)4.2.1 Fold-independent Prediction. Meta Learning for Fold independent prediction aims to predict unseen enzyme fold classes.Based on FuncDB 𝑏𝑖 , data from 9 folds are used as the training set, leaving one fold as the evaluation. Due to the limited samples in each fold and their mutual independence, we treat each fold as an independent task, leading to the application of meta-learning for task-based learning. 'Fold' here denotes the three-dimensional configuration of secondary structural elements, such as alpha helices and beta sheets, that define a specific protein or protein group. Proteins sharing similar folds usually exhibit substantial structural resemblances, despite variations in their sequences and functions. Classifying folds can provide valuable insights into the evolutionary connections between proteins. As shown in the table 1, Ours(M) involves meta-leaner as a base, followed by meta-finetuning for further refinement. And Ours(Z) directly employs zero-shot learning for trained meta-learner. Ours(M) yields superior results, highlighting the significant improvement brought about by meta-finetuning.Fold-agnostic Prediction. Ours(F) serves as a control group, primarily comparing few-shot learning with conventional finetuning methods. It represents fold-agnostic prediction, excluding meta-learning learning. We combine positive and negative samples from 9 folds, randomly allocating 80% as the training set and 20% as the evaluation set, treating the remaining 1 fold as an outof-distribution generalization. For unseen folds, we observed that task-specific learning based on meta-learning is more effective with the average value of Ours(M) at 0.903 exceeding that of Ours(F) at 0.857. Additionally, ESM-IF, acting as a baseline due to its outstanding representation and similar modalities, clearly demonstrates the superiority of our model in terms of generalization.Recognition. Furthermore, we conduct predictions on more challenging fold types based on FuncDB 𝑚𝑢𝑙𝑡𝑖 , essentially involving a less data-intensive multi-task enzyme function prediction, as shown in Figure 4 . This setup aims to predict, within enzymes with mixed folds, the specific fold to which an enzyme belongs. We select ESM-IF and GVP as structure-to-sequence models as baselines, as they also utilize N, C 𝛼 , and C as standard inputs. These comparisons validate the superior structural representation capabilities by a large margin (Ours: 17.3%P@1,31.7%P@2,41.0%P@3). We employ ESM-2 as a language modality input for comparison to confirm its sequence-only representation capabilities. Although ESM-2 (11.5%P@1,24.5%P@2,36.5%P@3) is a little better than the earlier GVP, but is far below the performance of our models, which demonstrates enhanced generalization ability due to the incorporation of structural information.Overall, while we have undertaken initial explorations in the few-shot setting, the overall accuracy remains relatively low. This is attributed to limitations in the quantity of available data and the diversity of fold types. This compels us to collect data encompassing a broader range of categories.Mutation Effects (MutDesign) Comparison among protein language models and inverse folding models.To ensure a comprehensive and unbiased comparison, we selected prominent protein language models and inverse folding models as benchmarks, as illustrated in Figure 5 (a). All baseline models employ zero-shot learning for mutation effects evaluation through the MutDB 𝑃𝑟𝑜𝑡𝐺 𝑦𝑚 , which encompasses millions of mutations. Note that our model consists of two configurations: Ours(w/o finetune) directly infers based on UniProt-Net, yielding an average 𝜌 of 43.9%; And Ours(w/ finetune) utilizes the meta-finetuned UniProt-Net, resulting in an average 𝜌 of 44.5%, which consistently achieves the best matching rank, implying that the in-domain finetuning on enzyme data contributes to enhanced overall performance in mutation prediction. In contrast, the optimal protein language model (VESPA) [31] achieved an average 𝜌 of 43.7%, while the optimal inverse folding model (ESM-IF) scored 42.2%. This performance superiority can be attributed to our model's dual advantage, encompassing both contextual and structural transfer learning within the proposed training paradigm. Table 2 : Space and time complexity analyses conducted with an average input protein length 500. Throughputs: samples per second; FLOPs: Gigabyte.Mechanism. The complexity analysis was carried out utilizing a single NVIDIA A100 80GB GPU, as outlined in Table 2 . We utilize the widely accepted fitness scoring metric, as referenced in ESM-1v [32] , as our baseline. By maintaining a fixed batch size of 1 and an average protein length of 500 amino acids, we vary the number of mutant proteins from 500 to 5000. Space complexity is governed by the number of parameters, where both the baseline and our decoupling method exhibit relatively small parameter counts (124.82M). Regarding time complexity, reflected through Throughputs and FLOPs, both the baseline and our method showcase nearly identical FLOPs (52.7k). As we increment the average number of mutant proteins from 500 to 5000, our decoupling method's advantage further amplifies, although this ascent in advantage slows down due to hardware memory limitations.Our approach requires only one inference step for the wild-type protein to acquire its probability distribution, subsequently facilitating swift calculation of mutant differences between mutant sequences and the inferred wild-type distributions. This significantly expedites the process. In contrast, the baseline necessitates model inference for each mutant protein, resulting in reduced speed. Overall, our model demonstrates markedly enhanced throughput performance, especially noticeable under conditions involving a substantial number of mutations. Compared to the mainstream baseline scoring method, ours boasts a speed advantage ranging from 4k to 30k times faster. This presents a promising avenue for future research, particularly in multi-site/high-order mutation inference, thereby expanding the possibilities for exhaustive exploration.Switching to the NAR mode and coordinating with the lower triangular mechanism, MetaEnzyme is employed for SeqDesign. The amino acid recovery (AAR) metric is adopted. In-domain Protein Assessment. In the evaluation of the comprehensive SeqDesignDB 𝐴𝐿𝐿 ('ALL'), our MetaEnzyme demonstrates superior performances, achieving an impressive AAR score of 54.94%. This surpasses the mainstream ProteinMPNN, PiFold, and ESM-IF consistently by a substantial margin. In alignment with [16] , we further assess specific subsets within 'ALL', namely the 'Short' dataset (comprising protein sequences with a length ≤ 100 residues) and the 'Single-chain' dataset (consisting of single-chain proteins cataloged in the Protein Data Bank). Notably, MetaEnzyme exhibits outstanding performance on both 'Single-chain' (39.17%) and 'Short' datasets (40.92%) compared to alternative methods.Out-of-domain Generalization. For a comprehensive comparison of the generalizability to out-of-domain datasets, we present results for the SeqDesignDB 𝑇 𝑠50 and SeqDesignDB 𝑇 𝑠500 datasets. Even in these diverse contexts, MetaEnzyme consistently demonstrates improvements, achieving a remarkable AAR score of 62.68% in Ts50 and a breakthrough AAR score of 60.77% in Ts500.Enzymes. Conditional SeqDesign for PET Enzymes. Polyethylene terephthalate (PET) is a widely used synthetic plastic polymer globally, known for its chemical inertness due to ester bonds and aromatic nuclei. This makes PET resistant to degradation, raising environmental concerns. Enzymatic degradation offers a promising and eco-friendly solution to address the ecological challenge of plastic waste, particularly polyester waste recycling. Despite the potential, our understanding of PET-degrading enzymes is limited. Machine learning-aid techniques [9, 29, 33] might accelerate the discovery of PET hydrolases. To support this effort, we curated the SeqDesignDB 𝑃𝑒𝑡 dataset for SeqDesign analysis, serving as a valuable resource. GVP). This suggests strong generalization capabilities for unseen functional enzymes, paving the way for future analysis and PETenzyme redesign.Conditional SeqDesign for Fold-aware Enzymes. In the examination of enzyme performance across diverse fold levels within the SeqDesign task, we systematically categorized them into 10 distinct classes utilizing SeqDesignDB 𝐻 𝑦𝑏𝑟𝑖𝑑 , as presented in Figure 7 (HybridEnz). When compared to alternative SeqDesign models, our attained AAR performance of 61.83% consistently outperforms the baseline scores (60.31% and 44.68%). This noteworthy result underscores a persistently elevated overall performance across a spectrum of hybrid fold types, emphasizing the robustness and effectiveness of our SeqDesign model.To underscore the significance of pretrained modules, we conducted an ablation study to assess the impact of the pretrained EnAd-Module (PEM) and the pretrained Context-Module (PCM). The results, outlined in and #1 vs. #2, it is evident that the pretrained structural features play a pivotal role in boosting performance. This aligns seamlessly with our rationale for incorporating prior structural knowledge. Moreover, given that the PCM module operates downstream of PSM, it is likely to directly benefit from the representation capabilities of PSM. Surprisingly, in the isolated comparison of #1 vs. #2, significant performance is enhanced even when utilizing only the PCM without the initialization parameters of the PEM. This intriguing finding, previously unexplored in SeqDesign studies, suggests that the PCM has acquired substantial knowledge of the protein language, thereby contributing significantly to improved SeqDesign performance. This observation gains further support in the comparison between #3 and #4, solidifying the conclusion that the PCM's acquired knowledge plays a vital role, diminishing the reliance on initialization parameters. The comprehensive analysis presented here highlights the logical and noteworthy advancements brought about by these pretrained modules in protein design. To further validate the reliability, we selected a commercially relevant enzyme with potential applications in production, the reversible glycine cleavage system (rGCS) [28, 39, 48] , for wet lab experimentation. rGCS, known for efficiently fixing carbon to produce glycine in vitro, currently exhibits relatively low yields in glycine production. Traditional rational enzyme design for rGCS has shown limited improvements in catalytic activity and other attributes. The rGCS, consisting of three proteins (P-protein decarboxylase, T-protein aminomethyltransferase, and H-protein shuttle), can fix two different one-carbon carbon sources to synthesize the two-carbon compound glycine. We focus on P-protein redesign in this work. For details on the structure and mutation preparation processes, please refer to Appendix. 
in vitro wet experimental validation: 
 Utilizing rational design principles, we strategically chose mutation sites with anticipated substantial influence on mutation effects. Wet lab experiments assessing carbon fixation were conducted for approximately 40 guided mutations, and their effects are illustrated in Figure 8 . The wet lab outcomes exhibit close alignment with in silico predictions, demonstrating a notable Spearman's 𝜌 ranking correlation of 70.1%. These results reinforce the practical efficacy of MetaEnzyme, pinpointing crucial sites for subsequent iterations in enhancing the P-protein's carbon-fixing capabilities. 
related work: 
 In the dynamic landscape of protein design, AI-driven approaches, exemplified by inverse folding [2, 13, 38, 50] , have made significant strides. Tools such as Structured Transformer [16] and GVP-GNN [19] have pioneered conditional protein generation, while recent models like ProteinMPNN [8] , PiFold [8] , and ESM-IF [15] showcase advancements in sequence recovery. AlphaFold2 [20] , RosettaFold [3] , OmegaFold [46] , helixfold [10] , ESMFold [27] stand out as influential structure prediction models. In the niche domain of enzyme engineering [4, 14, 21-25, 40, 43] , language and structure models such as DeepSequence [44] ,ESM-1v [32, 41, 42] , Tranception [35] , ESM-2 [27] employ few/zero-shot learning for mutation fitness. Our work extends these innovations, leveraging learned structural and linguistic insights, to encompass comprehensive design tasks, addressing challenges in general proteins and functional enzymes. Notably, the field grapples with the need for broader generalization across diverse proteins and presents opportunities for advancing functionality prediction. 
limitations and conclusions: 
 This work presents a pioneering MetaEnzyme framework tailored for crucial enzyme tasks, unveiling an architectural breakthrough with far-reaching implications across diverse protein engineering applications. MetaEnzyme underscores the prospect of transitioning from a universal to a unified enzyme design, enabling seamless adaptation across various functionalities through straightforward architectural modifications or lightweight adjustments. Despite the resource-intensive wet lab validation in this study, certain limitations are acknowledged: 1) restricted datasets of functional enzymes; 2) the necessity for more robust models to enhance generalization; and 3) a desire for additional wet lab experiments. Overcoming these challenges is crucial for advancing the field, and bolstering the applicability of enzyme design models.  
 
 Appendix:  


Decision: This paper proposes MetaEnzyme for computational enzyme design. It employs a cross-modal structure-to-sequence transformation architecture to establish robust protein representations and utilizes domain adaptive techniques for various enzyme design tasks under low-resource conditions. Specifically, MetaEnzyme focuses on functional design, mutation design, and sequence generation design. Wet lab experiments validate its efficacy, highlighting its potential impact on enzyme redesign.

Initially, the paper got 1 br and 2 ba. The main concerns are about the lacking experiments and its generalization ability. All reviewers have not give the final rating. The AC have read the authors rebuttal, which provide experiments on more datasets, the quantitative analysis, and more method details. The AC find the main idea of the MetaEnzyme interesting, and the experiments are sufficient and supportive.

Reviews: review: paper_summary: paper_summary: This paper introduces MetaEnzyme, a new framework for computational enzyme design that addresses the challenges of data scarcity and model generalization in the enzyme design field. MetaEnzyme leverages a cross-modal structure-to-sequence transformation architecture to obtain robust protein representations and employs domain adaptive techniques to generalize enzyme design tasks under low-resource conditions. It focuses on three fundamental low-resource enzyme redesign tasks: functional design (FuncDesign), mutation design (MutDesign), and sequence generation design (SeqDesign). It propose geometry-enhanced module, combined with techniques such as data augmentation and multi-modal fusion to enhance generalization in low-resource scenarios. Results demonstrate MetaEnzyme's adaptability and effectiveness across diverse enzyme design tasks. And wet lab experiments further validates its findings.
strength_weakness: strength_weakness: 1. The proposed MetaEnzyme framework can handle different tasks where systematic research has been lacking. It  contributes to the broader field of enzyme engineering.
2. The paper is well-structured, with clear sections and a logical flow of information. Lots of formula expressions make it easy for readers to understand the methods.
3. The proposed mutation effect scoring method is novel and effective, as it offers significant computational efficiency
  especially for sequence datasets with a large number of entries.
limitations: limitations: 1. As the datasets are limited, more experiments and analysis with extensive datasets are needed to clarify its effectiveness and generalization.
2. The method relies on pretrained universal protein models, which could potentially limit the model's performance on tasks that differ significantly from the data the pretrained models were exposed to.
paper_summary: paper_summary: Enzyme design is crucial in both industrial production and biology. However, due to the lack of comprehensive benchmarks and the complexity of enzyme design tasks, computational enzyme design remains in its early stages. In this paper, we introduce MetaEnzyme, a staged and unified enzyme design framework, to address these challenges. MetaEnzyme employs a cross-modal structure-to-sequence transformation architecture to obtain robust protein representation, and leverages domain adaptive techniques to generalize specific enzyme design tasks under low-resource conditions. The framework focuses on three fundamental low-resource enzyme redesign tasks: functional design (FuncDesign), mutation design (MutDesign), and sequence generation design (SeqDesign). Through novel unified paradigm and enhanced representation capabilities, MetaEnzyme demonstrates adaptability to diverse enzyme design tasks, yielding outstanding results. Wet lab experiments further validate these findings, reinforcing the efficacy of the redesign process.
strength_weakness: strength_weakness: 1.	Innovation: The MetaEnzyme framework has been proposed, which effectively solves the problems of data scarcity and insufficient model generalization in protein design through a cross modal structure to sequence transformation architecture and domain adaptation technology.
2.	Universality: The MetaEnzyme framework is suitable for various enzyme design tasks in low resource environments, including functional design, mutation design, and sequence generation design.
3.	Experimental verification: The effectiveness of the MetaEnzyme framework was verified through wet experiments, further demonstrating the reliability and efficiency of the redesign process.
limitations: limitations: 1.	Experimental comparison: The lack of comparison with traditional protein design methods, as well as more standard methods, limits a comprehensive understanding of the advantages of the MetaEnzyme framework in practice.
2.	Quantitative analysis: Lack of deeper quantitative analysis of experimental results, such as error analysis or trend analysis of model performance with data size changes, limits the interpretation and promotion of results.
3.	Terminology explanation: Some terminology explanations are not clear enough, which may cause confusion for readers. It is necessary to supplement some terminology tables to explain the meanings of professional terms.
paper_summary: paper_summary: ## Summary
"MetaEnzyme: Meta Pan-Enzyme Learning for Task-Adaptive Redesign" presents an innovative framework that addresses significant challenges in the field of enzyme design. By leveraging a meta-learning approach, the paper introduces a unified, task-adaptive enzyme design process that integrates cross-modal structure-to-sequence transformation architectures. The framework focuses on three key enzyme design tasks: functional design (FuncDesign), mutation effect prediction (MutDesign), and sequence generation design (SeqDesign). The results demonstrate substantial improvements in enzyme design performance across various tasks, showcasing the adaptability and efficiency of MetaEnzyme.

# Questions:
1. How does MetaEnzyme incorporate structural similarity metrics when designing and evaluating enzymes? Specifically, are metrics such as RMSD or TM-score used to ensure that the designed enzymes retain the necessary structural integrity and functional conformation?
2. Given the importance of the active site configuration for catalytic activity, how does the model ensure that structural modifications do not adversely affect the enzyme's active site geometry?
strength_weakness: strength_weakness: **Unified Framework**: MetaEnzyme effectively consolidates multiple enzyme design tasks into a single, cohesive framework, facilitating better task adaptation and efficiency.

**Comprehensive Evaluation**: The framework is thoroughly evaluated on multiple datasets, including CATH and SeqDesignDB, showing notable improvements in enzyme function prediction, mutation effect prediction, and sequence generation
limitations: limitations: The paper does not provide sufficient details on how structural similarity metrics, such as RMSD (Root Mean Square Deviation) or TM-score, are utilized in the design and evaluation of enzymes. Considering the critical role of enzyme structure in catalytic function, the incorporation of these metrics would be essential for ensuring the designed enzymes retain functional conformations.

The framework heavily relies on pre-trained modules. While these pre-trained components enhance performance, they also introduce a dependency on the availability and quality of pre-trained data, which may not always be accessible or sufficiently comprehensive for all enzyme design tasks.

While the paper provides an overview of the algorithms used, a more detailed explanation of the specific techniques and their implementations would help in understanding the precise mechanisms through which MetaEnzyme achieves its results. This includes a deeper dive into the meta-learning and domain adaptation strategies employed.