Design of Ligand-Binding Proteins with Atomic Flow Matching

Authors: Junqi Liu, Shaoning Li, Chence Shi, Zhi Yang, Jian Tang
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
Keywords: protein generation, protein-ligand interaction, protein binder design

Abstract:
Designing novel proteins that bind to small molecules is a long-standing challenge in computational biology, with applications in developing catalysts, biosensors, and more. Current computational methods rely on the assumption that the binding pose of the target molecule is known, which is not always feasible, as conformations of novel targets are often unknown and tend to change upon binding. In this work, we formulate proteins and molecules as unified biotokens, and present AtomFlow, a novel deep generative model under the flow-matching framework for the design of ligand-binding proteins from the 2D target molecular graph alone. Operating on representative atoms of biotokens, AtomFlow captures the flexibility of ligands and generates ligand conformations and protein backbone structures iteratively. We consider the multi-scale nature of biotokens and demonstrate that AtomFlow can be effectively trained on a subset of structures from the Protein Data Bank, by matching flow vector field using an SE(3) equivariant structure prediction network. Experimental results show that our method can generate high-fidelity ligand-binding proteins and achieve performance comparable to the state-of-the-art model RFDiffusionAA, while not requiring bound ligand structures. As a general framework, AtomFlow holds the potential to be applied to various biomolecule generation tasks in the future.

Decision: reject
Reviewer scores: 5, 6, 5, 6

Reviews and discussion:
<thread>
creation date:
1730552951244

thread_type:
official_review

<thread_object>
creator: reviewer_iafd

<content>
summary:
In this work, the authors present ATOMFLOW, a deep generative model based on a flow-matching framework for designing ligand-binding proteins from 2D target molecular graphs. ATOMFLOW operates on representative atoms of biological tokens to capture ligand flexibility, iteratively generating ligand conformations and protein backbone structures. Overall, this paper introduces an innovative model that offers a novel approach to designing proteins that bind to specific molecules.

strengths:
-  The paper is well-structured and logically organized. Readers can easily follow the authors’ thought process, and the content is presented in a way that is easy to understand.
- Conceptualizing both proteins and molecules as biotokens with representative atoms is an interesting approach. Although similar concepts are mentioned in Umol and RoseTTAFold All-Atom, using this framework to design specific proteins without relying on the initial ligand structure demonstrates impressive performance and is commendable.
- Compared to RFDiffusionAA, ATOMFLOW shows competitive results in specific protein-molecule design tasks as well as in binding affinity prediction. The supplementary experiments in Table 8 of the appendix provide additional support for the current experimental results.

weaknesses:
- The intervals used in modeling the distance map for ligands and proteins are not uniformly divided. While the authors account for differing precision requirements between residues and atoms, given that the paper frames proteins and molecules as biotokens with representative atoms, could the distance partitioning criteria be standardized? Establishing a more universal distance characterization approach—without introducing additional biases—might provide a more consistent method for distance representation.
-  The authors mention that “the model is first trained on solely generating the protein structure for 40k steps.” In this phase, is ligand information completely omitted, or is it set as specific input information? Further details on the full training process would be beneficial.
- The paper indicates that, during training, the FAPE loss is divided into protein-protein interaction, protein-ligand interaction, and ligand-ligand interaction components. However, these three losses are not directly introduced or explained in detail. Could the authors elaborate on the definitions of these three loss components and how they are calculated?
- The paper appears to lack ablation studies that could support certain design choices in the model. For instance, what impact would removing the first phase of training (the 40k steps on protein structure generation only) have on model performance? Additionally, how were the initial training and fine-tuning parameters in the second training phase determined, and what criteria guided the transition steps?

questions:
- As I mentioned in the weakness, I hope the author can add some ablation experiments to demonstrate the effectiveness of some training strategies used in the paper.
- The author can provide a more detailed explanation of the composition of the loss in Section 4.4. I haven't seen relevant information in the appendix yet.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730583333184

thread_type:
official_review

<thread_object>
creator: reviewer_nty9

<content>
summary:
This paper presents AtomFlow, a flow matching model for designing a protein structure to bind a small molecule ligand. The model jointly denoises the structure of the protein and ligand and thus does not require knowledge of the ligand pose, unlike RFDiffusionAA. The architecture is based on AlphaFold and predicts denoised structures from a distance map input of the input structure. The method is evaluated on (1) the set of 4 ligands studied in RFDiff-AA, and (2) an expanded set of ligands curated by the authors. AtomFlow is shown to have comparable or better designability than RFAA as well as similar Vina score.

strengths:
* The work is solidly executed, with sensible architectural choices, strong initial evaluations, and clear and concise writing. 
* The task tackled is significant and the competitive results signify well-executed model engineering and training practices.
* The authors re-derive the quotient-space flow matching fromework from AlphaFlow with more solid theoretical justification.
* The paper is quite clearly written. The figures are well made, visually appealing, and informative.

weaknesses:
**Originality**
* The methodology can be described as a flow-matching version of RFDiff-AA and does not score high on originality / novelty from a ML perspective. Further, the flow model architecture and noising process are based on AlphaFlow, with different justification but no difference in practice as far as I can tell. To improve on this axis, while it's not clear that more methodological novelty is needed for its own sake, the authors could focus on novel evaluations or applications of the proposed method.

**Quality**
* The computational evaluations are well executed, but limited in scope. Most of the analysis focuses on only 4 ligands, raising concerns about sample size and statistical significance. 
* The diversity and novelty evaluations are nice, but only AtomFlow is evaluated, not RFDiff-AA or the other baselines.


**Significance**
* The overall significance of the contribution is unclear as it represents an incremental methodological advance over RFDiff-AA with more or less the same model capabilities. The authors argue that not needing to specify the ligand pose is a big plus, but no meaningful evidence or use case is provided for this distinction. After all, RFDiff-AA has been experimentally validated, whereas AtomFlow-generated poses have not. It is of course not expected for a ML submission to experimentally validate the proteins, but it should be made a bit clearer why the main point of difference with RFDiff-AA is important to tackle as a ML problem.

questions:
No specific questions.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730589162004

thread_type:
official_review

<thread_object>
creator: reviewer_z34b

<content>
summary:
The authors present the ATOMFLOW architecture for the design of protein folds conditioned by the ligand SMILES description. The tool actually can be useful for the bioinformatics community, but many more validation tests are required here. I also thank the authors for providing the code, we had fun time generating different folds and assessing them.

strengths:
It's a well written technical paper about a novel architecture that can be useful for the bioinformatics community. I am thankful to authors for providing the code. Please remove the infinite loop and the check for the clash score, it takes too much time. 

We have executed it on a couple of example, including recent L targets from CASP. I have carefully examined the generated output for L1000 ligand set. With a significant probability (about 30%), AtomFlow finds the native experimental fold (we have ~20 PDB structures with ligands different from those in CASP) and the binding site, and is also rather stable with variations of the sequence length. If I vary the design length slightly, AtomFlow seems to prefer to shorten loops even those inside the binding site, so it seems it has memorized the space of all protein folds in the PDB and simply selects the one it has seen during training. It actually can be a useful tool for fold search and classification.

Individual protein-ligand contacts vary in quality significantly. So I have doubts in any usefulness of further affinity prediction providing a very low quality of experimental affinity data (see below).

We have also tried to generate folds for a purine ligand, which is a part of nucleotide bases (expecting many different folds to be generated). Surprisingly, at a constant generation length, the results were extremely conservative, with zero novelty and extremely low diversity, preferring the fold that dominantly occurs in the PDB. So, I suggest the authors to run more cross-validation tests hiding some fold classes from the training and trying to reproduce them in the test. Otherwise it seems there is a significant bias towards the folds that are abandon in the PDB.

weaknesses:
There is little technical novelty in the architecture blocks, loss function of flow matching process (but the architecture is novel and useful).

Since you are using the FAPE loss, why do you need equations 3-5, references to Riemannian Flow Matching, and fancy words? It will be much cleaner to remove this part. 

Binding affinity experiments are far from complete and better to be removed. You need to split the data correctly and run multiple affinity benchmarks, for example, following CASF and DUDe protocols. It will also be useful to provide energies and folds for some well-studied proteins that have multiple structures with different ligands (they are currently coming in the PDB, you may look at the last CASP experiment). Please also provide Vina scores for native ligand poses. Vina is not the best proxy for the affinity, as it was design only to rank binding poses for the same protein-ligand complex.

Please also be aware of the quality of current affinity training sets -- e.g., https://pubs.acs.org/doi/full/10.1021/acs.jcim.4c00049.

"The scatter plot of scRMSD vs. pdbTM shown in Figure 6B reveals that ATOMFLOW has the ability to generate structures that are quite different from existing proteins with acceptable designability" - Well, all of your designs have scores > 0.5, a few are around 0.5. It means that the fold structure is the same as in the PDB, no new folds have been discovered. Please try to discover a fold with a TM score around 0.2.

questions:
LCFM and LCFM-FAPE indeed give zero at the same point. But why their gradients will be similar? 

Can you explain more what does it mean "since the aligning object x varies upon training."? It contradicts all the theory you have just presented, no?

Can you comment on proper cross-validation experiments by removing certain fold classes from training? The same is valid for affinity experiments. Please also consider novel data with one protein and multiple ligands as validation, see four datasets from CASP16, for example.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730637273360

thread_type:
official_review

<thread_object>
creator: reviewer_x7zf

<content>
summary:
This paper presents ATOMFLOW, a novel deep generative model under the flow-matching framework for the design of ligand-binding proteins from the 2D target molecular graph alone.  It co-design the protein binder structure and the target-molecule flexibility. Experimental shows the method's effectiveness.

strengths:
S1. This method is based on full-atom level protein and molecule generation,  with both the protein binder and the molecule pose is designed.
S2. Experiments show the effectiveness of the method.

weaknesses:
W.1 I think the major weakness lies in its novelty. First of all, using Flow-matching with full atom generation in peptide [1] or protein [2] other bio-molecules is not a new idea. Besides, the equivariant generation process [3,4,5] and FAPE loss in AlphaFold still follow previous works. Therefore,this work appears to be a fusion of several previous studies, applied in a new task, lacking innovation in terms of the methodology. I hope the author could clarify the difference between the techniques used in this paper and the origianl ones, and the  improvements upon previous approaches. This would give the authors an opportunity to highlight any methodological innovations that may not be immediately apparent. Therefore, I have doubts about whether the level of novelty in this paper meets the standards expected for a top-tier conference like ICLR.

W.2 In the experiment, RFDiffusion based method like RFDiffusionAA is compared, but there should be more works which can fulfill such tasks, such as [6] and [7]. I am wondering that can such baselines be all included into experimental comparison? I hope the author  could discuss any technical or practical limitations that may have prevented including other baselines. Besides, I think AlphaFold3 can also fulfill this task, so what is the advantage of yours over AlphaFold3? Or there are several distinct differences between AtomFlow and AlphaFold3 in design protein binder with molecule targets? I hope the authors can clarify if AlphaFold3 is directly applicable to this specific task of designing ligand-binding proteins from 2D molecular graphs, and if so, I hope the authors to explain the key differences or advantages of their approach compared to AlphaFold3 for this particular application.

[1] Kong, Full-Atom Peptide Design with Geometric Latent Diffusion, https://arxiv.org/abs/2402.13555

[2] Avishek Joey Bose, SE(3)-Stochastic Flow Matching for Protein Backbone Generation, https://arxiv.org/abs/2310.02391

[3] Xu, GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation, https://arxiv.org/abs/2203.02923

[4] Emiel, Equivariant Diffusion for Molecule Generation in 3D, https://arxiv.org/abs/2203.17003

[5] Lin, PPFlow: Target-aware Peptide Design with Torsional Flow Matching, https://arxiv.org/abs/2405.06642

[6] Zhang, Pocketgen: Generating Full-Atom Ligand-Binding Protein Pockets. bioRxiv, 2024.

[7] Hannes Stark, Bowen Jing, Regina Barzilay, and Tommi Jaakkola. Harmonic Self-Conditioned Flow Matching for joint Multi-Ligand Docking and Binding Site Design. In Forty-first International Conference on Machine Learning

questions:
My major question is about the novelty and insufficient experiments. I hope the author can help to address my concerns.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732027689088

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
A Heartfelt Thanks to Our Reviewers

comment:
We sincerely thank all reviewers for their insightful comments and valuable feedback on our paper. Below, we summarize the updates made in response to your suggestions:

### New Experiments and Metrics

Recognizing the absence of a well-defined in silico metric for the binder design problem, we initially followed the conventions of existing literature in our experiments. After engaging with the reviewers' thoughtful feedback, we introduced several new metrics to strengthen the connection between our machine learning methodology and the biological community.

During the review process, several AlphaFold 3-like models were released. Leveraging this advancement, we added a new in silico experiment in the appendix to evaluate binding affinity, using the minimum interchain pAE predicted by AlphaFold 3 as a proxy for affinity, inspired by the approach in AlphaProteo.

We also added metrics to assess the geometric distribution and chemical validity of AtomFlow’s generated results. Additionally, we included diversity and novelty results for RFDiffusion-AA in the appendix.

### Supporting Results

To explicitly demonstrate that pocket design models are not suitable for the full protein binder generation setting, we conducted and reported experiments on PocketFlow in the appendix. We also performed a speed comparison between AtomFlow and RFDiffusion-AA, showing that AtomFlow is approximately 5× faster. This justifies our initial decision not to provide baseline results in the supplementary extended experiments (we plan to include these baseline results in the final version). The speed test further highlights the added value of AtomFlow.

### Text Refinement

We refined the text in Section 1 to better explain our contribution. We added an explicit introduction of excluding pocket design models in Section 5. We also added more details of the loss and our training process in the appendix.

We addressed each reviewer's concern in our reply to the review. We sincerely appreciate your feedback and are always happy to clarify further or discuss ways to improve our work.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper presents a method, AtomFlow, to design protein structures that bind to (a small molecule) input ligands. Contrary to prior work, it does so without the assumption that the ligand’s bound pose is given by adopting a flow that jointly generates the structure of the protein and the binding ligand. The general method is a parallel to the diffusion-based RFDiffAA that is based on flow matching. It evaluates the method against RFDiffAA baselines and shows comparable designability results, without knowing the bound structure, both on the RFDiffAA benchmark as well as on a new set of ligands. 

The reviewers acknowledged the importance of the task, the suitability of the architecture and novel use of flow matching, the clarity of the writing and presentation, the meaningful results in the experiments, the practical usefulness of the method,

On the other hand, the reviewers were concerned about the lack of technical novelty in light of prior work such as AlphaFlow and RFDiffAA (for a technical paper), and –as an application paper– they were concerned about  proper and conclusive comparison with baselines, the size of the test set, the extent and quality of evaluation tasks and metrics.

The authors did a thorough rebuttal which addressed some of the concerns and provided additional results and insights, through which the paper clearly improved as indicated by the reviewers. Some concerns were fully resolved such as comparison with methods beyond RF-DIFF family and several clarifications on how the evaluation is currently done. 

The AC believes the current paper has several merits and majorly improved during the rebuttal period as indicated by the reviewers. However, during the discussion among the reviewers and after the rebuttal period, the reviewers felt that the paper can further improve along the directions mentioned in the original reviews and during the rebuttal. The AC agrees with this evaluation and believes it is important that, after the suggested improvements, the paper goes through another round of review. Therefore, the AC suggests rejection at this time. A resubmission considering all the valuable feedback from the reviewers should make the paper much stronger for the next top venue. The main points that can improve the paper are the suggested directions to extend the empirical evidence so that it becomes a stronger application paper. Alternatively, or in tandem, the paper can better highlight what technical contribution the paper has that goes beyond the application of the paper.