Generalized Protein Pocket Generation with Prior-Informed Flow Matching

Authors: zaixi  zhang
marinka  zitnik
qi  liu
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-07 15:52:17

Abstract: Designing ligand-binding proteins, such as enzymes and biosensors, is essential in bioengineering and protein biology. One critical step in this process involves designing protein pockets, the protein interface binding with the ligand. Current approaches to pocket generation often suffer from time-intensive physical computations or template-based methods, as well as compromised generation quality due to the overlooking of domain knowledge. To tackle these challenges, we propose PocketFlow, a generative model that incorporates protein-ligand interaction priors based on flow matching. During training, PocketFlow learns to model key types of protein-ligand interactions, such as hydrogen bonds. In the sampling, PocketFlow leverages multi-granularity guidance (overall binding affinity and interaction geometry constraints) to facilitate generating high-affinity and valid pockets. Extensive experiments show that PocketFlow outperforms baselines on multiple benchmarks, e.g., achieving an average improvement of 1.29 in Vina Score and 0.05 in scRMSD. Moreover, modeling interactions make PocketFlow a generalized generative model across multiple ligand modalities, including small molecules, peptides, and RNA.

Main paper: 
 introduction: 
 Proteins are the fundamental building blocks of living organisms, often interacting with ligands (e.g., small molecules, nucleic acids, and peptides) to execute their functions. Recently, computational methods have played critical roles in designing functional proteins binding with ligands with broad applications in bio-engineering and therapeutics [76, 52, 50, 51, 67, 8, 58] . For example, Polizzi et al., [65] leverage template-matching methods to design de novo proteins binding with the drug apixaban [15] ; Yeh et al., [83] use deep learning methods to generate efficient light-emitting enzyme luciferases with selective substrate catalysis capabilities. To design such ligand-binding proteins, an essential step is to design protein pockets, the protein interface interacting with binding ligands [68, 39, 12, 28] . However, the complexity of ligand-protein interactions, the variability of protein sidechains, and sequence-structure relationships pose great challenges for pocket design [25, 51, 48] .Traditional methods for pocket design mainly focus on physics modeling or template-matching [12, 28, 65, 19, 62] . However, the involved physical energy calculation or substructure enumeration could be quite time-consuming. Recent advancements in pocket design have benefited a lot from deep learningbased approaches [73, 92, 83, 47, 51, 48] . However, these innovative approaches often overlook essential domain knowledge, such as the protein-ligand interactions and the geometric constraints governing them. Though they can efficiently generate many candidates, further screening/optimization is required to get valid and high-affinity pockets. Moreover, most methods are restricted to small molecule ligands, omitting other important ligand types such as nucleic acids [8] and peptides [53] .To tackle the aforementioned challenges, we propose PocketFlow, a protein-ligand interaction priorinformed flow matching model for protein pocket generation. Firstly, we define conditional flows for diverse data modalities in the protein-ligand complex including backbone frames, sidechain torsions, and residue/interaction types. We choose flow matching as the generative framework because of its efficiency and flexibility [17, 13, 57, 57] . Furthermore, PocketFlow explicitly learns the dominant protein-ligand interaction types including hydrogen bonds [35] , salt bridges [24] , hydrophobic interactions [61] , and π -π stacking [36] , which are crucial for strong binding stability and affinity of protein-ligand pairs [2] . In the sampling process, binding affinity and interaction geometry guidance are adopted to encourage generating pockets with high affinity and validity. Specifically, we leverage a lightweight binding affinity predictor to predict the affinity of the generated complex and apply distance and angle constraints to promote desirable protein-ligand interactions. To circumvent the non-differentiability issues associated with residue type sampling, we employ a novel sidechain ensemble method for interaction geometry calculations. Extensive experiments show that PocketFlow provides a generalized framework for high-quality protein pocket generation across various ligand modalities (small molecules, RNA, peptides, etc.,). The code is provided at https://github.com/zaixizhang/PocketFlow. Our main contributions are summarized as:• Generalized tasks: Our study broadens the scope of protein pocket generation tasks to include various ligand modalities such as small molecules, nucleic acids, and peptides.• Novel method: PocketFlow combines the recent progress of flow-matching-based generative models and physical/chemical interaction priors (affinity guidance and interaction geometry guidance) to generate protein pockets with enhanced affinity and structural validity.• Strong performance: PocketFlow outperforms existing methods on various benchmarks of pocket generation, producing an average improvement of 1.29 in Vina score and 0.05 in scRMSD. Further interaction analysis highlights the model's ability to foster beneficial protein-ligand interactions, e.g., an average of 4.12 hydrogen bonds, while markedly reducing steric clashes (an average of 1.21 in generated pockets v.s. 4.59 in the test set).2 Related Works 
generative models for protein generation: 
 Recent advancements in deep generative models have significantly advanced the field of de novo protein structure generation, enabling researchers to create proteins with specific desired properties [81, 37, 84, 86, 13, 95, 94, 91] . For example, RFDiffusion [81] employs denoising diffusion probabilistic models [33] in conjunction with RoseTTAFold [7] for de novo protein structure generation. It achieved notable success by generating proteins validated in wet lab experiments. Chroma [37] leverages a similar diffusion process with efficient neural architecture for molecular systems that enables long-range reasoning with sub-quadratic scaling. It also demonstrates strong capabilities to satisfy constraints including symmetries, substructure, shape, semantics, and simple naturallanguage prompts. Recently, models leveraging flow matching frameworks have shown promising results on protein generation [86, 13, 85, 40, 53] . For example, FoldFlow [13] proposed a series of flow-matching-based generative models for protein backbones with improved training stability and efficiency than diffusion-based models. FrameFlow [84, 85] further shows sampling efficiency and achieves success on motif-scaffolding tasks with flow matching. However, these protein generation methods are not directly applicable to protein pocket generation that requires protein-ligand interaction modeling.Protein pockets are the protein interface where the ligand binds to the protein and pocket design is a critical task for bioengineering [68, 39, 12, 28] . Traditional methods for pocket design focus on physics modeling or template-matching [12, 28, 65, 19, 62, 93] . For example, PocketOptimizer [62] predicts mutations in protein pockets to increase binding affinity based on physical energy calculation, which may bring a large time burden. The recent progress in protein pocket design has been facilitated by deep generative models [73, 92, 83, 93, 48] . For instance, FAIR [92] co-designs pocket structures and sequences using a two-stage coarse-to-fine refinement approach. RFDiffusion All-Atom [48] extends RFDiffusion for joint modeling of protein and ligand structure to generate ligand-binding protein and further leverages ProteinMPNN [21] /LigandMPNN [22] for sequence design. However, deep-learning methods lacking physical/chemical prior guidance may be less accurate and generalizable. In PocketFlow, we aim to design prior-guided pocket generative models. Notations. As shown in Figure 1 (a), we model protein-ligand complex as C = {P, G} consisting of protein P and ligand G (small molecule as an example). Protein P is composed of a sequence of residues (amino acids) with residue types denoted c (i) ∈ R 20 . Consistent with [92, 87] , the protein pocket R ⊂ P is defined as the subset of residues closest to the ligand atoms under a threshold δ (e.g., 3.5 Å). In a residue, the backbone structure (consisting of C α , N , C, O) is parameterized with C α position x (i) ∈ R 3 and a frame orientation matrix O (i) ∈ SO(3) following [43, 84] . The sidechain is parameterized with maximal 4 torsion angles χ (i) = {χ i1 , χ i2 , χ i3 , χ i4 } ∈ [0, 2π) 4 . Given these key parameters, the full atom protein structure can be derived with the ideal frame coordinates and the sidechain bond length/angles [43] . The protein-ligand interaction type for each residue is marked as I (i) ∈ R 5 (Hydrogen bond, Salt bridge, Hydrophobic, π-π stacking, no interaction). A pocket with N r residues can be compactly represented as R = {c (i) , x (i) , O (i) , χ (i) , I (i) } Nr i=1 . As for the ligand, we use a generalized atom-level representation that accommodates various modalities including small molecules, peptides, and RNA. The atom types and bonding information between atoms are given and PocketFlow predicts the N l ligand atom coordinates (also denoted as x (i) for conciseness).Problem Formulation. PocketFlow co-designs residue types and 3D structures of the protein pocket conditioned on the ligand (could be small molecules, nucleic acids, peptides, etc.) and protein scaffold (the other parts of protein besides the pocket region, i.e., P \ R). The ligand structure G is also predicted. Formally, PocketFlow aims to learn a conditional generative model p θ (R, G|P \ R).Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: In Sec.5.5, we clearly describe the limitations of the work and the potential ways to reduce the limitations in future works. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
preliminaries on flow matching: 
 Flow matching (FM) [57] is a simulation-free method for learning continuous normalizing flows (CNFs) [18] that generates data by integrating an ordinary differential equation (ODE) over a learned vector field. Here, we first give an overview of Riemannian flow matching [17] . On a manifold M, the CNF ψ t (•) : M → M is defined by integrating along a time-dependent vector field u t (x) ∈ T x M where T x M denotes the tangent space of the manifold at x ∈ M:d dt ψ t (x) = u t (ψ t (x)), ψ 0 (x) = x, t ∈ [0, 1].The flow transforms a simple distribution p 0 towards the data distribution p 1 . In FM, the target is to learn a neural network v θ (x, t) that approximates u t (x), while the vanilla regression loss:L F M (θ) = E t∼U [0,1],pt(x) ∥v θ (x, t) -u t (x)∥ 2g is hard to compute in practice. Here, U[0, 1] is the uniform distribution between 0 and 1, and ∥ • ∥ 2 g is the norm induced by the Riemannian metric g. Instead, it is tractable to define conditional vector field u t (x|x 1 ) and obtain the conditional FM objective: [57, 17] . In the inference, ODE solvers are applied to solve the ODE, e.g., x 1 = ODESolve(x 0 , v θ , 0, 1) where x 0 is the initialized data and x 1 is the generated data.L CF M (θ) = E t∼U [0,1],p1(x1),pt(x|x1) ∥v θ (x, t) -u t (x|x 1 )∥ 2 g . It has been proved that ∇ θ L F M (θ) = ∇ θ L CF M (θ) 
pocketflow: 
 PocketFlow is an interaction prior-informed flow-matching model for pocket design. In this section, we first define PocketFlow for different components in the protein-ligand complex (backbone in Sec. 4.1, sidechain in Sec. 4.2, and residue/interaction types in Sec.4.3). Then we show the prior-informed training and sampling in Sec. 4.4 and 4.5.As introduced in Sec. 3.1, each residue frame can be parameterized by a rigid transformation T = (x (i) , O (i) ) within SE(3) space. The backbone with N r residues can thus be described by a set of transformations [T (1) , . . . , T (Nr) ] belonging to SE(3) Nr and constitutes a product space. The following deduction focuses on a single frame but can be generalized to the whole protein backbone. The C α coordinates x (i) are initialized with linear interpolation and extrapolation based on the coordinates of neighboring scaffold residues following [92] . The prior distribution of O (i) is chosen as the uniform distribution on SO (3) . Following previous works [17, 84] , the conditional flow for x (i) and O (i) are defined as x(i) t = (1 -t)x (i) 0 + tx (i) 1 and O (i) t = exp O (i) 0 (t log O (i) 0 (O (i) 1 )) respectively, which are geodesic paths in R 3 and SO (3) . The exponential map exp O0 can be computed using Rodrigues' formula and the logarithmic map log O0 is similarly easy to compute with its Lie algebra so(3) [84] . The loss function of PocketFlow on SE(3) is the summation of the two losses below:EQUATIONEQUATIONwhere we additionally consider N l ligand atom coordinates in L coord (θ), for which we use Gaussian distribution at the center of ligand mass as the prior distribution.As described in Sec. 3.1, the sidechain conformation of each residue can be represented as maximally four torsion angles 4 . In a pocket with N r residues, the sidechain torsion angles form a hypertorus T 4Nr , which is the quotient space R 4Nr /2πZ 4Nr with the equivalence relation: χ = (χ 1 , . . . , χ 4Nr ) ∼ (χ 1 + 2π, . . . , χ 4Nr ) ∼ (χ 1 , . . . , χ 4Nr + 2π) [41, 90] . Following [42] , the prior distribution is chosen as a uniform distribution over T 4Nr . We regard the torsion angles as mutually independent and use interpolation paths as:χ (i) = {χ i1 , χ i2 , χ i3 , χ i4 } ∈ [0, 2π)χ t = (1 -t)χ 0 + t(χ ′ 1 -χ 0 ) where χ ′ 1 = (χ 1 -χ 0 + π) mod (2π) -π + χ 0 .The loss for the torsion angles is defined as:L tor (θ) = E t,p1(χ1),p0(χ0),pt(χt|χ0,χ1) 1 N r Nr i=1 v (i) θ (χ (i) t , t) -χ ′(i) 1 + χ (i) 0 2 2 .(3)Each residue is assigned a probability vector with 20 dimensions: c (i) ∈ R 20 . The prior distribution is set as the uniform distribution and the conditional flow is defined as the Euclidean interpolation between c 0 and c 1 (one hot vector indicating residue type). c t is a probability vector because its summation over all types equals 1. We leverage the cross-entropy loss CE(•, •) following [53, 73, 16] :EQUATIONwhich measures the difference between the true probability and the inferred oneĉ(i) 1 = c (i) t + (1 - t)v (i) θ (c (i)t , t). We also note the recent progress of the sequential flow matching methods [74, 16] , which can be seamlessly integrated into PocketFlow and are left for future works.It has been shown that modeling Protein-ligand interactions explicitly in biomolecular generative models can effectively enhance the generalizability [89, 97] . We used the protein-ligand interaction profiler (PLIP) [69] to detect and annotate the protein-ligand interactions for each residue by analyzing their binding structure. Following [97] , 4 dominant interactions are considered including salt bridges, π-π stacking, hydrogen bonds, and hydrophobic interactions. For simplicity, if a residue has more than one interaction, we take the one with the highest rank, which considers both the contribution to the binding affinity and the frequencies (see Appendix. B). Similar to residue types, interactions are modeled as category data: I = {I (i) } Nr i=1 . Besides the 4 interaction types, we also consider an unknown/none type. Similar to Equ. 4, we have the interaction loss:L inter = E t∼U (0,1),p1(I1),p0(I0),pt(I|I0,I1) Nr i=1 CE I (i) t + (1 -t)v (i) θ (I (i) t , t), I (i) 1.(5)Network Architecture. To design the binding protein pocket R = {c (i) ,x (i) , O (i) , χ (i) , I (i) } Nr i=1and update the binding ligand coordinates {x (i) } N l i=1 , we utilize an architecture modified from the FrameDiff [86] which incorporates Invariant Point Attention (IPA) from AF2 [43] to encode spatial features combined with transformer layers [79] to encode sequence-level features. To achieve a unified representation of both protein residues and ligand atoms, we follow the approach used in RoseTTAFold All-Atom [49] , where each ligand atom is treated as an individual residue. Initial representations are based on atom element type embeddings, and the frame orientations are set as identity matrices. To further model the covalent bonding information (single bond, double bond, triple bond, or aromatic bond), we also add the bond embeddings to the 2D track. We use additional MLPs based on the residue embeddings to predict the residue types, interaction types, and sidechain torsion angles. Instead of directly predicting the vector field, we let the model predict the final structure at t = 1 and interpolate to obtain the vector field. More details are introduced in the Appendix. C.Overall Training Loss. The overall training loss of PocketFlow is the summation of Equ. 1, 2, 3, 4, and 5. To fully utilize the protein-ligand context information, we use the whole protein-ligand complex structure at t, i.e., C t = P t ∪ G t as the inputs of v θ (•, t).Equivariance. Following [84, 53] , we perform all training and sampling within the zero center of mass (CoM) subspace by subtracting the CoM of the scaffold from the initialized structure. PocketFlow has the ideal SE(3)-equivariance property of geometric generative models: Theorem 1. Denote the SE(3)-transformation as T g , PocketFLow p θ (R, G|P \ R) is SE(3) equivariant i.e., p θ (T g (R, G)|T g (P \ R)) = p θ (R, G|P \ R), where R denotes the designed pocket, G is the binding ligand, and P \ R is the protein scaffold.The main idea is that the SE(3)-invariant prior and SE(3)-equivariant neural network lead to an SE(3)-equivariant generative process of PocketFlow. We give the full proof in the Appendix. D.To improve the binding affinity and structural validity of the generated protein pocket, we proposed a novel domain-knowledge-guided sampling scheme. Generally, we use classifier-guided sampling [23] and consider overall binding affinity guidance and interaction geometry guidance. To encourage the generated protein-ligand complex to satisfy a specific condition y, we apply the Bayes rule [23, 30] :EQUATIONwhere ∇ Ct log p(C t ) is the unconditional vector field v θ (C t , t) and ∇ Ct log p(y|C t ) is the guidance term to constrain the generated complex in a specific condition y.Binding Affinity Guidance. To generate protein pockets with higher binding affinity to the target ligand, we train a separate lightweight affinity predictor for guidance (More details of the predictor in Appendix. E.1). Specifically, the data points in the training set are annotated 1 if their affinity is higher than the average score of the dataset, otherwise 0 [66] . Because the intermediate structure is noisy, we take the expected structure at t = 1, i.e., Ĉ1 (C t ) from the model output and feed it into the predictor. Then we have the classifier-guided velocity field ṽθ (C t , t):EQUATIONwhere we add a scaling factor γ > 0 that controls the gradient strength. p θ is the affinity predictor and v b ∈ {0, 1} is the binary label of binding affinity.Interaction Geometry Guidance. Inspired by [97, 89] , we considered 4 dominant non-covalent interaction types in PocketFlow, including salt bridges, π-π stacking, hydrogen bonds, and hydrophobic interactions. The local geometries need to satisfy a series of distance/angle constraints to form strong interactions [69] . For example, for hydrogen bonds, the distances between donor and acceptor atoms need to be less than 4.1 Å and larger than 2 Å to reduce steric clashes [35] . The following inequality is a necessary condition for residues in Ĉ1 (C t ) with predicted interaction label Î1 as hydrogen bond:EQUATIONwhere l min and l max are distance constraints;A (k)hbond denote the k-th residue in the set of residues with predicted hydrogen bonds. With a little abuse of notations, x (i) and x (j) denote the candidate atom coordinates in the residue and ligand respectively. The distance guidance can be derived as:EQUATIONwhered (k) = min i∈A (k) hbond ,j∈G x (i) -x (j)2 and ξ 1 , ξ 2 > 0 are constant coefficients that control the strength of guidance. The detailed deduction is included in the Appendix. E.2. Besides the distance constraint, the hydrogen bond needs to satisfy the acceptor/donor angle constraint [69] , e.g., the donor/acceptor angle needs to be larger than 100 • . The angle guidance is presented as follows:EQUATIONwhere ϕ (k) = max i∈A (k) hbond ,j∈G hangle(x (i) , x (j) ) and hangle(•, •) calculates the acceptor/donor angle in Figure . 4. ξ 3 > 0 is the guidance coefficient. The guidance for the other interactions is discussed in Appendix. E. We note that the residue type/side chain structure of the pocket is not determined during the sampling. Directly sampling from the residue type distribution makes the model not differentiable [38] . We propose the Sidechain Ensemble for the interaction geometry calculation, i.e., the weighted sum of geometric guidance with respect to residue types (Figure . 6 ).Sampling. With the initialized data, the sampling process is the integration of the ODE dCt dt = v θ (C t , t) from t = 0 to t = 1 with an Euler solver [14] . γ, ξ 1 , ξ 2 , and ξ 3 are set as 1 in the default setting. To apply the guidance, we use ṽθ which is v θ plus guidance terms (Equ. 7, 9, and 10):EQUATIONxEQUATIONEQUATION)where ∆t is the time step; v θ (•; t) denotes the subcomponent of the vector field for different variables. norm(•) means normalizing the vector to a probability vector such that the summation is 1, and reg(•) means regularizing the torsion angles by reg(τ ) = (τ + π) mod (2π) -π. 
experimental settings: 
 Datasets. Following previous works [29, 71, 92] we consider two widely used protein-small molecule binding datasets for experimental evaluations: CrossDocked dataset [27] is generated through crossdocking and is split with mmseqs2 [75] at 30% sequence identity, leading to train/val/test set of 100k/100/100 complexes. Binding MOAD dataset [34] contains experimentally determined proteinsmall molecule complexes and is split based on the proteins' enzyme commission number [9] , resulting in 40k protein-small molecule pairs for training, 100 pairs for validation, and 100 pairs for testing. To test the generalizability of PocketFlow to other ligand modalities, we further consider PPDBench [3] , which contains 133 non-redundant complexes of protein-peptides and PDBBind RNA [80] , which contains 56 protein-RNA pairs by filtering the PDBBind nucleic acid subset. More details of data preprocessing are included in the Appendix. A. Considering the distance ranges of protein-ligand interactions [60] , we redesign all the protein residues that contain atoms within 3.5 Å of any binding ligand atoms, i.e., the pocket area. The number of designed residues is set the same as the reference pocket. We sample 100 pockets for each complex in the test set for evaluation.Baselines and Implementation. PocketFlow is compared with four state-of-the-art representative baseline methods. DEPACT [19] is a template-matching method [88] for pocket design by searching and grafting residues. dyMEAN [47] and FAIR [92] are deep-learning methods based on equivariant translation and iterative refinement. RFDiffusionAA [48] is the latest version of RFDiffusion [81] , which can directly generate protein pocket structures conditioned on the ligand. Different from the sequence-structure co-design scheme in dyMEAN and FAIR, the residue types and sidechain structures in RFDiffusionAA are determined with LigandMPNN [22] in a post-hoc manner.For experiments on PPDBench and PDBBind RNA, we pretrain PocketFlow and baseline models on the combination of the CrossDocked and Binding MOAD dataset, in which we carefully eliminate all complexes with the same PDB IDs to avoid potential data leakage. Then we represent the peptide and RNA similar to small molecules (atom coordinates/types and covalent bonds) and input to PocketFlow for sampling without fine-tuning. For simplicity, the structure of peptide/RNA is set fixed. All the baselines are run on the same Tesla A100 GPU.Performance Metrics. We employ the following metrics to comprehensively evaluate the validity of the generated pocket sequence and structure. Amino Acid Recovery (AAR) is defined as the overlapping ratio between the predicted and ground truth residue types. In bioengineering, mutating too many residues may lead to instability or failure to fold [4] . Therefore, a larger AAR is more favorable. scRMSD refers to self-consistency Root Mean Squared Deviation between the generated and the predicted pocket's backbone atoms to reflect structural validity. Following established pipelines [ 77, 54] , for each generated protein structure, eight sequences are firstly derived by ProteinMPNN [21] and the folded to structures with ESMFold [56] . we report the minimum scRMSD for the predicted structures. To measure the binding affinity for protein-small molecule pairs, we calculate Vina Score with AutoDock Vina [78] following [64, 92] . For protein-peptide and protein-RNA pairs, we calculate Rosetta ∆∆G [5] and Rosetta-Vienna RNP-∆∆G [44] respectively that measure the binding affinity change. The unit is kcal/mol and a lower Vina score/∆∆G indicates higher affinity.Table 1 shows the results of different methods on the CrossDocked and Binding MOAD dataset for small-molecule-binding pocket design. We can observe that PocketFlow overperforms baseline models with a clear margin on AAR, scRMSD, and Vina scores, demonstrating the strong ability of PocketFlow to design pockets with high validity and affinity. The average improvements over the RFDiffusionAA on AAR, scRMSD, and Vina Score are 3.3%, 0.05, and 1.29 respectively. PocketFlow also predicts more aligned sidechain angles with ground truth as evidenced in Table . 5. Compared with baselines, PocketFlow enjoys the advantage of powerful flow-matching architecture and effective physical/chemical prior guidance. Different from the post-hoc manner of deriving sequences in RFDiffusionAA [48] , the co-design scheme also encourages sequence-structure consistency. We also compare the generation efficiency of different models in Figure . 7. Considering the pocket quality improvement brought by PocketFlow, the time overhead is acceptable.We also perform ablation studies in Table 3 : Interaction analysis of the generated protein pockets on the CrossDocked dataset. We measure the average number of steric clashes (Clash), hydrogen bonds (HB), salt bridges (Salt), hydrophobic interactions (Hydro), and π-π stacking (π -π) per protein-ligand complex. More results on the variants of PocketFlow are included in Appendix F.Besides small molecules, the binding of protein with other ligand modalities such as peptides and nucleic acids play critical roles in biomedicine [82, 8] . However, the available dataset size compared with small molecules-protein complexes is quite limited (e.g., ∼ 100 in PPDBench v.s. over 100k in CrossDocked). Here, we explore whether the pretrained PocketFlow on the combination of CrossDocked and Binding MOAD can generalize to peptide and RNA-binding pocket design in Table . 2. The peptide and RNA ligands are represented as molecules (atoms and covalent bonds) to fit into the pretrained models. We have observed that PocketFlow achieves performance comparable to the state-of-the-art baseline, RFDiffusionAA, with prior guidance significantly enhancing its generalizability. Our hypothesis is that the protein-ligand interactions and fundamental physical laws learned by PocketFlow are applicable universally across various biomolecular domains [89, 97] . By explicitly incorporating physical and chemical priors into the generative model, PocketFlow not only aligns with these universal principles but also gains a marked advantage of generalizability.We adopt PLIP [69] and posecheck [31] to detect the protein-ligand interactions in the generated pockets. In Table . 3, we show the average number of steric clashes, hydrogen bond donors, acceptors, and hydrophobic interactions (without redocking). We observe that PocketFlow can generate pockets with fewer clashes and more favorable interactions. For example, the average steric clashes for RFDiffusionAA and PocketFlow are 3.58 and 1.21 respectively. The average number of Hydrogen Bonds for RFDiffusionAA and PocketFlow are 3.76 and 4.12 respectively. These improvements can be attributed to the model's affinity/geometry guidance and its enhanced modeling of pocket/ligand flexibility, both of which promote the formation of advantageous protein-ligand interactions while minimizing clashes. Some interaction types such as π-π stacking in PocketFlow are a little less than the reference, which may be due to the low frequency of these interactions in the dataset. PocketFlow recovers most residue types and changes several key residues to achieve higher binding affinity. The overall structure of the pocket, including the sidechains, is generally well-maintained.While PocketFlow is a powerful generative method for pocket generation, we find the following limitations for further improvement. First, PocketFlow is only trained on protein-small molecule datasets in the paper. In the future, incorporating protein-peptides/nucleic acids/metal datasets, even the generated data from AlphaFold3 [2] would be promising directions. Second, the integration of pretrained protein language models [55] and structure models [96] could significantly enhance PocketFlow's performance. Additionally, wet lab experiments to verify PocketFlow's efficacy are planned. Potential negative impacts may include the misuse of PocketFlow for creating harmful biomolecules [32] . Rigorous oversight and screening access to the model should be considered. 
conclusion: 
 In this paper, we proposed PocketFlow, a protein-ligand interaction prior-informed flow matching model for protein pocket generation. We define multimodal flow matching for protein backbone frames, sidechain torsion angles, and residue/interaction types to appropriately represent the proteinligand complex. The binding affinity and interaction geometry guidance effectively improve the validity and affinity of the generated pockets. Moreover, PocketFlow offers a unified framework covering small-molecule, nucleic acids, and peptides-binding protein pocket generation. 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: The paper discussed potential societal impacts.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA]Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: The existing assets are properly cited and credited.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators.13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [NA] Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: We consider two widely used datasets for benchmark evaluation: CrossDocked dataset [27] contains 22.5 million protein-molecule pairs generated through cross-docking. Following previous works [59, 64, 92] , we filter out data points with binding pose RMSD greater than 1 Å, leading to a refined subset with around 180k data points. For data splitting, we use mmseqs2 [75] to cluster data at 30% sequence identity, and randomly draw 100k protein-ligand structure pairs for training and 100 pairs from the remaining clusters for testing and validation, respectively; Binding MOAD dataset [34] contains around 41k experimentally determined protein-ligand complexes. Following previous work [71] , we keep pockets with valid and moderately 'drug-like' ligands with QED score ≥ 0.3. We further filter the dataset to discard molecules containing atom types / ∈ {C, N, O, S, B, Br, Cl, P, I, F } as well as binding pockets with non-standard amino acids. Then, we randomly sample and split the filtered dataset based on the Enzyme Commission Number (EC Number) [9] to ensure different sets do not contain proteins from the same EC Number main class. Finally, we have 40k protein-ligand pairs for training, 100 pairs for validation, and 100 pairs for testing. For all the benchmark tasks in this paper, PocketFlow and all the other baseline methods are trained with the same data split for a fair comparison.To test the generalizability of PocketFlow to other ligand modalities, we further consider PPDBench [3] , which contains 133 non-redundant complexes of protein-peptides and PDBBind RNA [80] , which contains 56 protein-RNA pairs by filtering the PDBBind nucleic acid subset with RNA sequence lengths longer than 5 and less than 15. Distance between two centers of charges in salt bridges [10] Following [97] , we considered 4 dominant non-covalent interaction types in PocketFlow, including salt bridges, π-π stacking, hydrogen bonds, and hydrophobic interactions (ranked based on their contribution to affinity and reversed frequency). The frequency statistics are listed in Table .3.• Salt bridges [10] , which are electrostatic interactions between oppositely charged centers, are often considered among the strongest interactions in protein structures and other biomolecular complexes. They can significantly contribute to stability and binding affinity due to their strong electrostatic nature. To form salt bridges, two centers of opposite charges need to be below a distance of SALTBRIDGE_DIST_MAX.• Hydrogen bonds [35] occur between a hydrogen atom covalently bonded to a more electronegative atom (like oxygen or nitrogen) and another electronegative atom. Their strength is less than that of salt bridges but is significant in biological contexts.A hydrogen bond is established between a hydrogen bond donor and acceptor (OpenBabel [63] is used to detect hydrogen bond donor/acceptor). The distance between the donor and acceptor needs to be less than HBOND_DIST_MAX. The donor and acceptor angle needs to be larger than HBOND_DON_ANGLE_MIN and HBOND_ACC_ANGLE_MIN respectively. Since PocketFlow only considers heavy atoms (no hydrogen atoms), we consider the geometry of hydrogen bonds without protonation [35] (see Figure . 4 ). For simplicity, we do not differentiate donor/acceptor in the interaction geometry guidance.• π-π stacking [69] involve the stacking of aromatic rings (like those found in phenylalanine, tyrosine, or tryptophan) due to favorable van der Waals forces and sometimes electrostatic interactions. π-π stacking is crucial in the structure of nucleic acids and proteins, especially in the active sites of many enzymes, although they are generally weaker than hydrogen bonds and salt bridges.To form π-π stacking, we first need two aromatic rings (OpenBabel [63] is used to detect aromatic rings). The distance between the two ring centers needs to be below PISTACK_DIST_MAX. The angle between two normal vectors of ring planes needs to be below PISTACK_ANG_DEV. Additionally, each ring center is projected onto the opposite ring plane. The distance between the other ring center and the projected point (i.e., the offset) has to be less than PISTACK_OFFSET_MAX. • Hydrophobic Interactions [11] are caused by the tendency of hydrophobic side chains to avoid contact with water, leading them to aggregate. While these are not strong interactions on their own, they play a crucial role in the folding and stability of proteins by driving the burial of nonpolar groups away from the aqueous environment, thereby contributing significantly to the overall stability. To form hydrophobic interactions, the atom distance needs to be less than HYDROPH_DIST_MAX. In PocketFlow, we adopt a neural network architecture modified from the FrameDiff [86] . This architecture consists of Invariant Point Attention from AlphaFold2 [43] and transformer blocks [79] . In this section, we use superscripts to refer to the network layer and subscripts to indexes or variables. In the network, each residue/ligand atom is represented by one embedding h ∈ R Dx and a frame T ∈ SE(3). For the ligand atoms, the orientation matrix of frame is set as identity matrices. Overall, at the ℓ-th layer of the network, h ℓ = [h ℓ 1 , . . . , h ℓ N ] ∈ R N ×Dx are all the node embeddings where h ℓ i is the embedding for the i-th node and N = N p + N l is the total number of nodes; T ℓ = [T ℓ 1 , . . . , T ℓ N ] ∈ SE(3) N is the frames of every node at the ℓ-th layer; z ℓ ∈ R N ×N ×Dz are edge embeddings with z ℓ ij being the embedding of the edge between residues i and j. In the following paragraphs, we introduce the details of feature initialization, node/edge update, backbone update, and residue type/interaction type/sidechain torsion angle predictions.Feature initialization. Following [86] , node embeddings are initialized with residue indices and timestep while edge embeddings additionally get relative sequence distances. Initial embeddings at layer 0 for residues i, j are obtained with an MLP and sinusoidal embeddings ϕ(•) [79] over the features. Following [86] , we additionally include self-conditioning of predicted C α displacements. Let x be the C α coordinates predicted during self-conditioning. 50% of the time we set x = 0. The binned displacement of two C α is given as,EQUATIONwhere d 1 , . . . , d Nbins are linspace(0, 20) are equally spaced bins between 0 and 20 angstroms. In our experiments we set N bins = 22. The initial embeddings can be expressed asEQUATIONEQUATIONwhere D h , D z are node and edge embedding dimensions. For the initialization of C α coordinates, we use the interpolation and extrapolation strategy of FAIR [92] .Node update. The process of node update is shown below. Invariant Point Attention (IPA) is from [43] . No weight sharing is performed across layers. We use the vanilla Transformer from [79] . We use Multi-Layer Perceptrons (MLP) with 3 Linear layers, ReLU activation, and LayerNorm [6] after the final layer.EQUATION)EQUATIONEQUATIONEQUATIONEQUATIONEQUATIONEdge update. Each edge is updated with a MLP over the current edge and source and target node embeddings. In the first line, node embeddings are first projected down to half the dimension.EQUATIONEQUATIONEQUATIONBackbone update. Our frame updates follow the BackboneUpdate algorithm in AlphaFold2 [43] .We write the algorithm here with our notation,EQUATIONEQUATIONEQUATIONEQUATIONEQUATIONwhere b i , c i , d i ∈ R, x update i ∈ R 3 . Equ. 27 constructs a normalized quaternion which is then converted into a valid rotation matrix in Equ. 28 . Following [84, 81] , we use the planar geometry of the backbone to impute the oxygen atoms. Note that we only update the pocket and ligand nodes in PocketFlow while setting the scaffold nodes fixed.Residue/Interaction Type and Torsion angle Prediction. We predict the residue/interaction types and sidechain torsion angles based on node embeddings.EQUATIONEQUATIONEQUATIONwhere c ∈ R N ×20 , I ∈ R N,5 , and χ ∈ [0, 2π) 4N . In PocketFlow, the number of network blocks is set to 8, the number of transformer layers within each block is set to 4, and the number of hidden channels used in the IPA calculation is set to 16. The node embedding size D h and the edge embedding size D z are set as 128. We removed skip connections and psi-angle prediction. For model training, we use Adam [45] optimizer with learning rate 0.0001, β 1 = 0.9, β 2 = 0.999. We train on a Tesla A100 GPU for 20 epochs. In the sampling process, the total number of steps T is set as 50.Theorem 1. Denote the SE(3)-transformation asT g , PocketFLow p θ (R, G|P \ R) is SE(3) equivari- ant i.e., p θ (T g (R, G)|T g (P \ R)) = p θ (R, G|P \ R), where R denotes the designed pocket, G is the binding ligand, and P \ R is the protein scaffold.Proof. The main idea is that the SE(3)-invariant prior and SE(3)-equivariant neural network lead to an SE(3)-equivariant generative process of PocketFlow. By subtracting the CoM of the scaffold from the initialized structure, we obtain an SE(3)-invariant prior distribution similar to [86, 29] . Moreover, the neural network for structure update as shown in Appendix C is SE(3)-equivariant. Formally, the two conditions to guarantee an invariant likelihood p θ (R 1 , G 1 |P \ R) are as follows (we use subscripts to denote the time steps from t = 0 to t = 1):EQUATIONEquivariant Transition:EQUATIONWe can obtain the conclusion as follows:p θ (T g (R 1 , G 1 )|T g (P \ R)) = p(T g (R 0 , G 0 , P \ R)) T -1 s=0 p θ (T g (R (s+1)∆t , G (s+1)∆t )|T g (R s∆t , G s∆t , P \ R)) = p(R 0 , G 0 , P \ R) T -1 s=0 p θ (T g (R (s+1)∆t , G (s+1)∆t )|T g (R s∆t , G s∆t , P \ R)) = p(R 0 , G 0 , P \ R) T -1 s=0 p θ (R (s+1)∆t , G (s+1)∆t |R s∆t , G s∆t , P \ R) = p θ (R 1 , G 1 |P \ R),where T is the total number of steps. We apply the invariant prior and equivariant transition conditions in the derivation.Here, we present the Bayesian approach to guide the flow matching with the affinity predictor. The key insight comes from connecting flow matching to diffusion models to which affinity guidance can be applied. Sampling a data point from the prior distribution p 0 , we have the following ordinary differential equation (ODE) [72] that pushes it to data distribution:EQUATIONwhere ∇ log p t (C t ) is the score function, f (C t , t) and g(t) are the drift and diffusion coefficients respectively. We modify Equ. 36 to be conditioned on the affinity label (v b = 1) followed by an application of Bayes rule,EQUATIONwhere the first term is the unconditional vector field and the second term is the affinity guidance term.In practice, we do not directly predict the affinity label based on C t because the intermediate structure is noisy. We use the following transformation:EQUATIONwhere Ĉ1 (C t ) is the expected denoised protein-ligand complex structure based on C t . Details of the affinity predictor is introduced in the Appendix. E.1. We need to choose g(t) such that it matches the learned probability path. Previous works [20, 86] showed g(t) 2 = t 1-t in the Euclidean setting. For simplicity, we set g(t) as constant 1 and observe good performance in experiments.In PocketFlow, we leverage a binding affinity predictor p θ (v b | Ĉ1 (C t )) to guide the denoising process, where v b ∈ {0, 1} is the binary label of binding affinity and Ĉ1 (C t ) is the expected protein-ligand structure at t = 1. Following [66, 29] , we leverage a 3-layer EGNN [70] with the node initialized embeddings and residue/ligand atom coordinates from Appendix C. Specifically, we take the C α coordinates for the residues and ligand atom coordinates and construct k-NN graphs (k set as 9). Let h ℓ i and x ℓ i be denote the node representations and coordinates at the ℓ-th layer. The (ℓ + 1)-th layer is computed as follows:EQUATIONEQUATIONwhere d ℓ ij = ∥x ℓ i -x ℓ j ∥ represents the Euclidean distance between node i and node j at the ℓ-th layer, N denotes the k-NN neighbors, and e ij indicates the direction of message-passing, including from protein to protein, from protein to ligand, from ligand to protein, and from ligand to ligand. The functions f h and f x are graph attention networks. Finally, we append an average pooling, one linear layer, and softmax operation at the end to predict the binary label of affinity.To train the binding affinity predictor, we first annotate the data points in the corresponding training set: data points are annotated 1 if their affinity is higher than the average score of the dataset, otherwise 0. We train the predictor separately instead of joint training with flow matching because we find it can converge more quickly than the flow matching losses. We did not train the predictor on the intermediate structures as we find they are noisy and deteriorate the predictor and PocketFlow's overall performance. In experiments, we use the Adam optimizer and train for 10 epochs.Distance Guidance. For hydrogen bonds, the distances between donor and acceptor atoms need to be less than 4.1 Å and larger than 2 Å to reduce steric clashes [35] . The following inequality is a necessary condition for residues in Ĉ1 (C t ) with predicted interaction label Î1 as hydrogen bond:EQUATIONwhere l min and l max are distance constraints; Ahbond denote the k-th residue in the set of pocket residues with predicted hydrogen bonds. With a little abuse of notations, x (i) and x (j) denote the atom coordinates in the residue and ligand respectively. We use the following derivations to obtain the guidance term for the distance constraints:EQUATIONEQUATIONEQUATIONEQUATIONEQUATIONwhereξ 1 = 1/∇ Ct P (min i∈A (k) hbond ,j∈G ∥x (i) -x (j) ∥ 2 ≤ l max ) and ξ 2 = 1/P (-min i∈A (k) hbond ,j∈G ∥x (i) -x (j) ∥ 2 ≤ -l min ).Due to the discontinuity of the indicator function I(•) that is incompatible with the gradient, we apply ξ -max(0, ξ -y) as a surrogate of I(y < ξ) in the above equation. Although ξ 1 and ξ 2 are dependent on C t , we find setting them as constant still works well in experiments. With these approximations, we can derive guidance term for hydrogen bond distance constraints:EQUATIONwhered (k) = min i∈A (k) hbond ,j∈G x (i) -x (j)2 . Such distance guidance terms for hydrophobic interactions, salt bridges, and π -π stackings are similar. The difference is to replace A hbond with A hydro , A salt , and A π that denotes the residue sets with corresponding interactions. We modify the functions in plip2 for the ease of detecting interaction atom pair candidates. In practice, ∇ Ct takes gradients with each component in C t , including χ t , x t , O t , c t , and I t .Angle Guidance. Besides the distance constraint, the hydrogen bond needs to satisfy the acceptor/donor angle constraint [69] , e.g., the donor/acceptor angle needs to be larger than 100 • . hangle(•, •) calculates the acceptor/donor angle in Figure . 4 .EQUATIONEQUATIONEQUATIONwhere ξ 3 = 1/P (α min ≤ max i∈A (k) hbond ,j∈G hangle(x (i) , x (j) )). The final guidance term is:EQUATIONwhere ϕ (k) = max i∈A (k) hbond ,j∈G hangle(x (i) , x (j) ). The angle constraint is similar for the π -π stacking and the final guidance term is: Sidechain Ensemble. PocketFlow takes the co-design scheme, where the residue type/side chain structure of the pocket is not determined during sampling. Directly sampling from the residue type distribution makes the model not differentiable [38] . We propose to use the sidechain ensemble for the interaction geometry calculation, i.e., the weighted sum of geometric guidance with respect to residue types. For example, for Equ. 54, we have:EQUATIONEQUATIONwhere ĉ(i) 1 [n] denote the n-th residue type probability and ϕ (k) calculates the angle with the n-th type residue side chain. Here, we show additional results on efficiency analysis (Figure. Figure . 7 shows that the PocketFlow is much more efficient than stat-of-the-art diffusion-based models such as RFDiffusionAA. Considering the high quality of the generated pockets, the slight time overhead over models based on iterative refinement (e.g., dyMEAN and FAIR) is acceptable. We find that Affinity and Interaction Geometry Guidance do not add much overhead to the generation process. Therefore, these prior guidance are efficient tools for pocket optimization.In Figure . 8, we explore the impact of Affinity Guidance Strength (γ) on various generation metrics. As γ is scaled up, the Vina Score significantly improves and quickly stabilizes; AAR initially increases before gradually decreasing; scRMSD, on the other hand, increases with higher γ. These observations underscore the importance of selecting an appropriate γ to effectively balance the guidance and unconditional terms. While Affinity Guidance promotes the generation of high-affinity pockets, an excessively high γ can result in less valid pocket sequences or structures. In the default configuration, γ is set to 1.To evaluate the validity of the generated sidechain structure, we compute the Mean Absolute Error (MAE) of sidechain angles (degrees) following [90] in Table . 5. We mainly compare PocketFlow with RFDiffusionAA [48] +LigandMPNN [22] on the recovered residues. In the DEPACT [19] 3 is a template-matching method that follows a two-step strategy for pocket design. Firstly, it searches the protein-ligand complexes in the template database with similar ligand fragments and constructs a cluster model (a set of pocket residues). The template databases are constructed based on the corresponding training datasets for fair comparisons. Secondly, it grafts the cluster model into the protein pocket with PACMatch. It works by placing residues from the cluster model on protein scaffolds by matching the atoms of residues with atoms of the protein scaffold. The backbone coordinates of the pocket residues are also modified in the process. The qualities of the generated pockets are evaluated and ranked based on a statistical scoring function. We take the top 100 designed pockets for evaluation. The output of DEPACT+PACMatch is complete protein structures with redesigned pockets. In the paper, we only use DEPACT to represent the whole method of DEPACT+PACMatch for conciseness.RFDiffusionAA [48] 4 is the latest version of RFDiffusion which combines a residue-based representation of amino acids and atomic representations of all other groups to model protein-small molecules/metals/nucleic acids/covalent modification complexes. Starting from random distributions of amino acid residues surrounding target small molecules, RFDiffusionAA can directly generate the small molecule binding protein backbone. Furthermore, with LigandMPNN [22] , the latest version of ProteinMPNN [21] , we can assign residue types and predict sidechain conformations considering the protein-ligand interactions. Experiments in RFDiffusionAA [48] show that the generated protein by RFDiffusionAA has better binding affinity than those obtained by RFDiffusion with auxiliary potential. We use the provided checkpoints of RFDiffusionAA for all the experiments since the training code is unavailable.dyMEAN [47] 5 is an end-to-end full-atom model for E(3)-equivariant antibody design given the epitope and the incomplete sequence of the antibody. Its previous version, MEAN [46] , only considers the backbone atoms, while dyMEAN considers the complete atom structure and performs better on downstream tasks. Generally, dyMEAN co-designs antibody sequence and structure via a multi-round progressive full-shot refinement manner, which is more efficient than auto-regressive or diffusionbased approaches. An adaptive multi-channel equivariant encoder is used in dyMEAN, which can process protein residues of variable sizes when considering full atoms. To adapt dyMEAN to our pocket design task, we replace the antigen with the target ligand molecule to provide the context information for pocket generation. We set the hidden size as 128, the number of layers as 3, and the number of iterations for decoding as 3.FAIR [92] 6 is our previous method for full atom pocket sequence-structure co-design. FAIR operates in two steps, proceeding in a coarse-to-fine manner (backbone refinement to full atoms refinement, including side chains) for full-atom generation. In FAIR, residue types and atom coordinates are updated using a hierarchical graph transformer composed of a residue-level and atom-level encoder.The number of layers for the atom and residue-level encoder are 6 and 2, respectively. K a and K r are set as 24 and 8 respectively. The number of attention heads is set as 4; The hidden dimension d is set as 128.1. Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: In the abstract and introduction, we clearly state the contributions of our paper. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.https://github.com/pharmai/pliphttps://github.com/chenyaoxi/DEPACT_PACMatchhttps://github.com/baker-laboratory/rf_diffusion_all_atomhttps://github.com/THUNLP-MT/dyMEANhttps://github.com/zaixizhang/FAIR 


Decision: The authors present a novel protein-ligand interaction prior flow-matching model for protein pocket generation. The approach defines flow matching for backbone frameworks, side-chain torsion angles, and residue/interaction types. To enhance the structural validity and binding affinity of generated pockets, the authors propose incorporating affinity and geometric guidance during the sampling process. Experimental results show that PocketFlow is a generalized generative model capable of handling a wide range of ligand types, including small molecules, peptides, and RNA, achieving SOTA performance.

The reviewers universally found this work interesting and timely, particularly given the growing ability to protein design. Despite some minor concerns, this is a strong paper that I recommend for acceptance to NeurIPS.

Reviews: review: paper_summary: paper_summary: The paper proposed PocketFlow, a generative model for designing protein pockets that bind with ligands. It aims to overcome limitations in existing methods by incorporating protein-ligand interaction priors and utilizing flow matching. PocketFlow is designed to handle multiple ligand modalities and demonstrates superior performance on various benchmarks.
strength_weakness: strength_weakness: - The method only considers interactions between protein and ligand, potentially neglecting interactions between protein sidechains within the pocket region.
- For some residue types, there might be π instead of 2π symmetry in the sidechain structures, which the proposed method seems to simplify.
- The use of flow matching and multiple guidance mechanisms could result in higher computational costs compared to simpler models.
questions: questions: 1. Why were these four types of interactions (hydrogen bond, salt bridge, hydrophobic, π-π stacking) chosen? Are there other interactions existing in protein-ligand binding that should be considered?
2. Can the proposed method be extended to protein-ligand docking (fixing the pocket type and structure)?
3. What is the efficiency of calculating different interaction types?
4. Is there any metric to evaluate the correctness of the interactions generated?
5. The original IPA relies on the rotational equivariance of frame orientation to achieve model’s invariance. However, the proposed method additionally treats the ligand atom as a residue and uses an invariant identity matrix to represent its orientation. Will the proposed IPA still output invariant embeddings?
6. How are the guidance coefficients for different guidance mechanisms determined?
7. Is RFDiffusionAA trained on the same dataset as the proposed model?
limitations: limitations: N/A
paper_summary: paper_summary: The paper explores methods for generating protein pockets given a ligand using a flow matching generative approach. Unlike previous methods, the proposed approach integrates additional constraints into the flow matching learning process to guide the search for relevant pockets. Two types of constraints are considered: 

1. Affinity score prediction, which utilizes an oracle predictive model to guide the generation process.
2. Geometric constraints, which ensure that the distances between atoms involved in specific types of bonds remain below certain thresholds.

The authors examine the effectiveness of these constraints, incorporating both domain knowledge and transferred knowledge from an affinity prediction model, in the generation of protein pockets. They benchmark their approach against state-of-the-art methods that do not use such constraints. The results demonstrate that the new pockets generated have improved RMSD and Vina scores.
strength_weakness: strength_weakness: I have a concern about the inclusion of the affinity oracle predictor, which was trained on separate datasets. The authors did not specify which dataset was used to train their predictor or address the potential for information leakage between the training dataset and the test set used for pocket finding. Since the predictor provides the flow matching with a prior on which pocket might be relevant for a given ligand, any leakage would give a clear advantage. I looked for more detailed information about the affinity predictor in the appendix but found none regarding the training dataset.

I also question the novelty of the work in terms of its technical contribution to the machine learning community. While conditional flow matching itself is not new, the novel aspect of this work is the incorporation of additional domain constraints into the framework. Therefore, it is unlikely to be of significant interest to the machine learning research community from a methodological development perspective.

Additionally, since the authors retrained the baseline models with additional data, it is important that they report how hyperparameters were selected and tuned for these baselines.
questions: questions: 1. Could you please address the potential leakage between the dataset used for training the affinity predictor and the datasets used for assessing the pocket generation?
2. Could you please clarify the impacts of HPO of the baselines and the proposed method in this work?
limitations: limitations: The authors have addressed relevant limitations of the work in the paper.
paper_summary: paper_summary: This paper studies the task of generalized ligand-binding protein pocket generation. To tackle the challenges of existing works, the authors proposed PocketFlow, a generative model that incorporates protein-ligand interaction priors based on flow matching. PocketFlow explicitly learns the protein-ligand interactions during training and leverages multi-granularity guidance to generate high-quality pockets during sampling. Experiments show that PocketFlow is a generalized generative model across various ligand modalities, including small molecules, peptides, and RNA.
strength_weakness: strength_weakness: 1. What is the time cost of PocketFlow and baseline methods in generating protein pockets? Would the prior knowledge-based guidance bring an extra burden to pocket generation?  
2. In line 222, the authors mentioned sidechain ensemble for the interaction geometry calculation, which is too concise. The authors are recommended to elaborate more on this in the main pages. 
3. In Table 2, some metrics of PocketFlow are not the best and RFDiffusionAA has better results.
questions: questions: 1. Why didn’t the authors apply discrete flow matching methods for residue/interaction type prediction, e.g., [1]?

[1] Dirichlet flow matching with applications to dna sequence design, arXiv preprint arXiv:2402.05841, 2024
limitations: limitations: The authors discussed the limitations and broader impacts in Section 5.5.
paper_summary: paper_summary: In this paper, the authors proposed PocketFlow, a protein-ligand interaction prior-informed flow matching model for protein pocket generation. The flow matching for backbone frames, sidechain torsion angles, and residue/interaction types are appropriately defined. To enhance the structural validity and binding affinity of the generated pockets, the authors proposed to leverage affinity and geometry guidance in the sampling process. Extensive experiments show that PocketFlow provides a generalizable model for pocket generation covering various ligand modalities such as small molecules, nucleic acids, and peptides.
strength_weakness: strength_weakness: 1. The backbone model of PocketFlow is modified from existing work such as FrameDiff and is less novel.   
2. In lines 252-253, the authors said PocketFlow is only pretrained on protein-small molecule datasets, i.e., CrossDocked and Binding MOAD. Is it possible to train PocketFlow on the combination of protein-small molecule/peptide/RNA datasets for better performance? 
3. In experiments on protein-peptide and RNA, PocketFlow represents peptide/RNA ligands as molecules. Could the frame representation for the protein residues also applied to peptide/RNA ligands?
questions: questions: Please see the Weaknesses.
limitations: limitations: The limitations and broader impacts are well discussed in Section 5.5 of the main paper.

Comments: title: Thanks for the response!
Response to Author
Response to author
Further Response to Reviewer gNH7
comment: Thank you for clarification, I increase my score.
Yes, the average score (Vina score) is calculated on the training data. We annotate the data point as 1 if its calculated Vina score is lower than the average (higher affinity); we annotate the data point as 0 if its calculated Vina score is higher than the average (lower affinity). Therefore, all the calculations are based on the training data and there are no data leakage risks. We will make the statements in the paper clearer in the revised version. Thanks for the comments!

Bests,  
Authors
Thank you for your response to my concern, I read the paragraph in the appendix:
"To train the binding affinity predictor, we first annotate the data points in the corresponding training
set: data points are annotated 1 if their affinity is higher than the average score of the dataset, otherwise 0." 
Could you please clarify more on "higher than the average score of the dataset", the average score you mentioned in this sentence is calculated on the training data?
Hi,

Thanks for the response, my concerns have been successfully addressed! 

Regards,
Thanks for the authors response. After reading the rebuttal, my concerns are resolved and I decided to rise my score.
**Comment 6**: What is the efficiency of calculating different interaction types?

**Response 6**: Thanks for the question! In Figure 7 included in the Appendix, we compared the average generation time of PocketFlow and its variants such as PocketFlow without different guidance terms. We can observe that calculating the protein-ligand interaction for guidance will not bring much overhead (~28% of the total generation time).

To detect the different interaction types in the generated pockets, we leverage PLIP and posecheck. They are also efficient tools and it totally takes around 20 seconds to process 100 generated pockets.

**Comment 7**: Is there any metric to evaluate the correctness of the interactions generated?

**Response 7**: Thanks for the question! In PocketFlow, we used the protein-ligand interaction profiler (PLIP) [69] to detect and annotate the protein-ligand interactions for each residue by analyzing their binding structure (Table 3 and lines 304-314 of the paper). Generally, PLIP is based on physical/chemical rules and employs four steps including structure preparation, functional characterization, rule-based matching, and filtering to detect the generated interactions. In the PLIP paper [69], the authors compared the detected/ground truth interactions of 30 literature-validated examples and achieved good consistency. Therefore, we can regard the detected interactions by PLIP as correct. 

**Comment 8**: The original IPA relies on the rotational equivariance of frame orientation to achieve model’s invariance. However, the proposed method additionally treats the ligand atom as a residue and uses an invariant identity matrix to represent its orientation. Will the proposed IPA still output invariant embeddings?

**Response 8**: Thanks for the insightful question! We agree that IPA relies on the rotational equivariance of frame orientation to achieve the model’s invariance. In our case, there is no canonical orientation for the ligand atoms and we set them as an identity matrix for simplicity.
To achieve invariant embeddings, we can initialize the protein scaffold by aligning the protein principal axes with the coordinate axes. This can be achieved by subtracting the center of mass (COM), computing the inertia matrix, diagonalizing the inertia matrix, and aligning the protein to principal axes. In experiments below, we compared such an initialization strategy with the default setting (only subtracting COM). The experimental results are comparable and show no significant differences. Therefore, we use the default setting for simplicity. We will include more discussions in our revised paper. 

| Model         | CrossDocked           |                   |                  | Binding MOAD         |                   |                  |
|---------------|-----------------------|-------------------|------------------|----------------------|-------------------|------------------|
|               | AAR (↑)               | scRMSD (↓)        | Vina (↓)         | AAR (↑)              | scRMSD (↓)        | Vina (↓)         |
| PocketFlow    | 52.19±1.34%       | 0.67±0.04         | -8.236±0.16 | 54.30±1.70%      | 0.68±0.03         | -9.370±0.24  |
| w/ principal axes aligning | 52.12±1.29%          | 0.67±0.03        | -8.227±0.18      | 54.47±1.73%          | 0.69±0.03         | -9.362±0.28     |

**Comment 9**: How are the guidance coefficients for different guidance mechanisms determined?

**Response 9**: In the default setting, the guidance coefficients of PocketFlow including $\gamma, \xi_1, \xi_2,$ and $\xi_3$ are set as 1 and achieve good results. We also explore the influence of guidance coefficients in the appendix. For example, in Figure. 8 of the submitted paper, we explore the impact of Affinity Guidance Strength ($\gamma$) on various generation metrics. As $\gamma$ is scaled up, the Vina Score significantly improves and quickly stabilizes; AAR initially increases before gradually decreasing; scRMSD, on the other hand, increases with higher $\gamma$. These observations underscore the importance of selecting an appropriate $\gamma$ to effectively balance the guidance and unconditional terms. While Affinity Guidance promotes the generation of high-affinity pockets, an excessively high $\gamma$ can result in less valid pocket sequences or structures. In the default configuration, $\gamma$ is set to 1 for simplicity.

**Comment 10**: Is RFDiffusionAA trained on the same dataset as the proposed model?

**Response 10**: As indicated in lines 822-823, we used the provided checkpoints of RFDiffusionAA for all the experiments because the training code is not available (https://github.com/baker-laboratory/rf_diffusion_all_atom). The original training data of RFDiffusionAA contains 121,800 protein-small molecule structures, 112,546 protein-metal complexes, and 12,689 structures with covalently modified amino acids, which represent a broad set of protein-ligand complex structures.