Kati8/ICLR_2023_2025 huggingface dataset:
authors: Xiaoran Jiao
authors: Weian Mao
authors: Wengong Jin
authors: Peiyuan Yang
authors: Chunhua Shen
reviews: 
Comments: The paper proposes a novel and interesting method that leverages Boltzmann Alignment to establish a more sophisticated connection between inverse folding tasks and binding free energy (ΔΔG) prediction. Despite relying on somewhat idealized approximations, the method leverages a thermodynamic cycle and the principles of the Boltzmann distribution to incorporate both bound and unbound states of the protein complex, offering a physically motivated alignment technique that addresses limitations of traditional pre-trained inverse folding models.  Through evaluations on the selected SKEMPI v2 dataset, the paper demonstrates that the method achieves significant improvements over existing approaches in both supervised and unsupervised settings.  Overall, the paper provides a new perspective for integrating domain knowledge into alignment techniques.
Strengths: 1) The paper is well-written.

2) The approach achieves state-of-the-art results compared to baselines.

3) The paper effectively incorporates domain-specific priors, specifically leverages Boltzmann principles to connect energy with probability, along with the assumption that unbound states can be approximated as independent probabilities for each chain. Though this assumption is idealized, it still is an interesting approach that introduces a motivated alignment.

4) The paper demonstrates practical applicability on binding energy prediction, protein-protein docking and antibody optimization tasks.
Weaknesses: 1) The method makes strong assumptions, particularly in treating the unbound state as independent probabilities for each chain, which oversimplifies the inter-chain interactions. Additionally, the predictions of inverse folding model’s do not fully align with the predicted probabilities (not the ground truth), introducing biases when single-chain probabilities are combined to approximate complex states. This approximation may result in significant deviations in some situations.

2) The method has only been validated on a limited dataset, which lacks sufficient evidence to demonstrate the generalizability of the approach and its underlying assumptions.

3) It seems that the paper lacks sufficient discussion on how improvements in the inverse folding model would translate to gains in this approach.  It’s also unclear which specific module in this sophisticated approach contributes most significantly to the information gain.
Recommendation: 8
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 3
reviews: 
Comments: The paper presents Boltzmann Alignment, a novel approach for predicting changes in protein-protein binding free energy due to mutations. It utilizes pre-trained inverse folding models and introduces physical inductive bias through the Boltzmann distribution.
Strengths: - The integration of Boltzmann distribution within the framework to enhance ∆∆G prediction is innovative and addresses a significant challenge in computational biology.
- The authors effectively showcase the versatility of their method in binding energy prediction, protein-protein docking, and antibody optimization, indicating its broad applicability.
- The paper is well-organized, and the results are presented clearly, making it easy for readers to follow the methodology and findings.
Weaknesses: I haven't identified any significant weaknesses.
Recommendation: 6
Reviewer confidence: 3
Soundness/correctness: 3
Clarity: 3
reviews: 
Comments: In this paper authors propose a novel, thermodynamics inspired, approach for predicting binding free energy change upon mutation (ddG) building, in particular, on the capabilities of protein inverse folding models. Validation of the method employs standard literature benchmark (SKEMPI v2 curating and integrating a vast number of experimental measurements of ddG across various types of proteins). Beyond that, authors demonstrate applicability of their method across several tasks (binding energy prediction, antibody optimisation and docking). The reported results, in general, significantly outperform a spectrum of modern methods presented in the relevant literature both in supervised and unsupervised context.
Strengths: The paper is very well written and, I believe, easy to follow both for machine-learning and also, to large extent, for biology-oriented audiences thus extending its potential impact. The limitations and weaknesses are appropriately discussed. It focuses on relevant and important problem in protein science - predicting the energetic effect of mutation on the free energy of protein-protein complex formation. The contribution is clearly original, correctly references and builds upon several already existing contributions in the field (in particular - inverse folding models, direct preference optimisation). Most importantly, benchmark reports significant improvement across several investigated metrics on commonly-accepted dataset (SKEMPI) and described ablations and controls (folded structures) strengthen the confidence in the reported result.
Weaknesses: - Method is structure based, it requires at least experimentally resolved or confidently predicted complex structure,
- The representation of protein is rather coarse, only backbone. Side-chain atoms and solvent molecules are omitted yet they are obviously important drivers of protein-protein interactions.
- No representation of dynamics (structures are represented as static snapshots), which can affect the method performance for mutations inducing significant conformational changes.
- The described method seems to excel at point mutations and there's a trend towards decreased performance in more complex multiple mutations.
Majority of those weaknesses are highlighted and discussed by the authors in the appropriate sections across the paper.
Recommendation: 10
Reviewer confidence: 3
Soundness/correctness: 3
Clarity: 4
reviews: 
Comments: This paper employs pre-trained inverse folding models based on the Boltzmann distribution and thermodynamic cycles to associate energy with log-likelihood from inverse folding models for predicting ΔΔG. It introduces both unsupervised and supervised inverse folding models, each achieving state-of-the-art (SoTA) accuracy on the SKEMPI v2 dataset. Additionally, the study demonstrates that the proposed thermodynamic cycle is effective and offers advantages over traditional SFT and DPO methods. Notably, the method does not strictly require crystal structures when a reasonably accurate predicted structure is available. Furthermore, the paper explores applications of this approach in binding affinity energy prediction, protein-protein docking, and antibody optimization.
Strengths: 1. It creatively used Boltzmann Alignment, which introduces physical inductive bias through Boltzmann distribution and thermodynamic cycle to link binding energy and log-likelihood in inverse folding models.

2. This work achieved SOTA and outperforms other SFT and DPO methods. The method can not only be used in binding energy prediction, but also be used in  protein-protein docking, and antibody optimization.
Weaknesses: 1. The code is not available,  it should be open-sourced upon publication.

2. Dividing the dataset into 3 folds by structure cannot strictly prevent data leakage, different protein complexes might have similar protein binding structures.
Recommendation: 6
Reviewer confidence: 3
Soundness/correctness: 3
Clarity: 3
reviews: 
Comments: This work introduces an innovative approach grounded in thermodynamic principles for estimating the change in binding free energy following mutations (ddG), leveraging recent advances in protein inverse folding models. The methodology is rigorously evaluated using SKEMPI v2, a comprehensive experimental database containing numerous ddG measurements across diverse protein systems. Furthermore, the authors demonstrate the versatility of their approach through multiple applications including binding affinity assessment, antibody enhancement, and molecular docking. The empirical findings consistently surpass existing state-of-the-art approaches documented in the literature, encompassing both supervised and unsupervised learning paradigms.
Strengths: The manuscript is exceptionally well-articulated and accessible to both computational and experimental audiences, thereby broadening its potential reach and influence. The paper thoughtfully addresses its own constraints and limitations. It tackles a critical challenge within structural biology—quantifying how mutations modulate the thermodynamic stability of protein-protein complexes. The proposed methodology demonstrates clear novelty while appropriately synthesizing prior work, particularly relating to inverse folding approaches and preference optimization techniques. Notably, the benchmarking demonstrates substantial performance gains across multiple evaluation metrics on the widely-recognized SKEMPI benchmark, with supporting ablation studies and control experiments that reinforce the credibility of the findings.
Weaknesses: - The approach relies on structural information, requiring either experimentally determined or reliably predicted three-dimensional complex geometries. - The protein representation is relatively simplified, considering only the polypeptide backbone while disregarding side-chain geometry and hydration shell effects, both of which substantially influence intermolecular recognition. - The method does not account for protein conformational dynamics, treating structures as static models, which may limit predictive accuracy for mutations triggering substantial structural rearrangements. - Performance appears particularly strong for single amino acid substitutions, with a noticeable performance degradation trend when handling combinatorial or epistatic mutations. It should be noted that the authors have acknowledged and thoughtfully examined these limitations within the manuscript.
Recommendation: 10
Reviewer confidence: 3
Soundness/correctness: 3
Clarity: 4