AtomSurf: Surface Representation for Learning on Protein Structures
Authors: Vincent Mallet
Yangyang Miao
Souhaib Attaiki
Bruno Correia
Maks Ovsjanikov
Venue: ICLR 2025 Poster (2025)
Keywords: deep learning
protein representation learning; surface methods; geometric deep learning
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We propose a state of the art surface encoder for proteins and show it performs deceptively compared to graph methods. The combination of a graph method and this encoder, however, yields state-of-the-art results.

Abstract: While there has been significant progress in evaluating and comparing different representations for learning on protein data, the role of surface-based learning approaches remains not well-understood. In particular, there is a lack of direct and fair benchmark comparison between the best available surface-based learning methods against alternative representations such as graphs. Moreover, the few existing surface-based approaches either use surface information in isolation or, at best, perform global pooling between surface and graph-based architectures. 
    
In this work, we fill this gap by first adapting a state-of-the-art surface encoder for protein learning tasks. We then perform a direct and fair comparison of the resulting method against alternative approaches within the Atom3D benchmark, highlighting the limitations of pure surface-based learning. Finally, we propose an integrated approach, which allows learned feature sharing between graphs and surface representations on the level of nodes and vertices \textit{across all layers}.
    
We demonstrate that the resulting architecture achieves state-of-the-art results on all tasks in the Atom3D benchmark, while adhering to the strict benchmark protocol, as well as more broadly on binding site identification and binding pocket classification. Furthermore, we use coarsened surfaces and optimize our approach for efficiency, making our tool competitive in training and inference time with existing techniques.

Code can be found online: https://github.com/Vincentx15/atomsurf

Published as a conference paper at ICLR 2025
ATOMSURF: SURFACE REPRESENTATION FOR LEARNING ON PROTEIN STRUCTURES
Vincent Mal.et Souhaib Attaiki*
Yangyang Miao Bruno Correia", Maks Ovsjanikov'
Equal Contribution
LIX, Ecole Polytechnique, IPP Paris, Paris, France
Mines Paris, PSL Research University, CBIO, Paris, France Institut Curie, PSL Research University, Paris, France INSERM, U900, Paris, France
Institute of Bioengineering, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland vincent ma malles let @minesparis psl. eu maks@ miks@lixx.e ptw..x.o.pttcciiiuee.........e fr
ABSTRACT
While there has been significant progress in evaluating and comparing different representations for learning on protein data, the role of surface-based learning approaches remains not well-understood. In particular, there is lack of direct and fair benchmark comparison between the best available surface-based learning methods against alternative representations such as graphs. Moreover, the few existing surface-based approaches either use surface information isolation or, best, perform global pooling between surface and graph- based architectures.
In this work, we fill this gap by first adapting a state-of-the-art surface encoder for protein learning tasks. We then perform direct and fair comparison of the resulting method against alternative approaches within the Atom3D benchmark, highlighting the limitations of pure surface-based learning. Finally, we propose integrated approach, which allows learned feature sharing between graphs and surface representations on the level nodes and vertices across all layers.
We demonstrate that the resulting architecture achieves state-of-the-ar results on all tasks in the Atom3D benchmark, while adhering the strict benchmark protocol, well as more broadly on binding site identification and binding pocket classification. Furthermore, we use coarsened surfaces and optimize our approach for efficiency, making our tool competitive in training and inference time with existing techniques. Code can be found online: [github. com/Vincentx15/ at fonsur
INTRODUCTION
Structural bioinformaties data is becoming available unprecedented pace. Advances in cryogenic Electron Microscopy (cryo-EM) in particular, have led to the production of evermore experimentally derived structures, well as larger systems and better resolutions (Fontana al. 2022). The development of AlphaFold (Jumper et al.) 2021) along with many subsequent works have made protein structures abundantly available, with over million high-quality predictions in the Protein Data Bank (PDB) (Berman 2000) and over 600 million in the ESM Metagenomic Atlas (ESMatlas) (Lin al.) 2022). There thus growing demand for machine learning techniques which can leverage this structural data to help advance the fields of structural bioinformaties and drug design.
Protein structures are complex objects characterized both by atomic coordinates as well as intricate bio-chemical interactions between them that depend on their geometry. To be used in a learning pipeline, an initial modeling step transforming protein structures into a well-defined mathematical object is necessary. Different mathematical representations encode different structural and biological priors. For instance, the point cloud representation disregards the connectivity induced by chemical interactions, but allows for the most generic geometric description of the data. Protein surfaces choose to trade fine- aeeigraided information of the interior of a protein for an accurate depiction of its outer surface. This representation is thought to be of particular interest to study active interaction

---

Published as a conference paper at ICLR 2025
sites that mostly depend on properties of the surface because of the screening effect. However, even for interactions dominated by surface terms, the knowledge of the interior can encode the stability of the surface. These different representations are illustrated in Supplementary Figure
The choice of representation is particularly prominent in the context of learning- based methods, as specialized architectures have been developed to process each type of data. The range of approaches is studied within the field of geometric deep learning (Bronstein et al.] (017). and specialized methods have been developed to process different data types from graphs (Bruna al.. [0033] Kipf & Welling 2016), to point clouds (Qi et al. 2017 Wang et al. 2019). surfaces (Masci et al., 2015 Monti et al.
2017 equivariant methods that respect group symmetry of the data (Cohen & Welling 2016a b).
equivariant message passing (Fuchs et al. 2020 Satorras et al. (2221) and more.
few pioneering works have applied geometric deep learning to structural biology data representa- tions, using 3DD convolutional networks (Jiménez et al., (017). equivariant convolutional networks (Weiler et al., 2018), sequence (Rao et al. (001], surfaces (Gainza et al.) 2020), graphs (Aumentado- Armstrong 2018) and equivariant discrete networks (Jing et al. 2021 Stärk et al., (022). They were followed by several others, traditionally classified based on the mathematical representation they use
see for instance [sert et al. (2023). In addition, some methods were developed ad-hock to handle
protein structure, where protein properties are baked into the network Zhang et al. 2022 Hermosilla
cet al. 2020 Fan et al. 2022).
In this context, the seminal work of At om3d (Townshend et al. 2020) aims for a fair compari- son across both different representations and learning approaches, within well-defined protocol. Specifically the benchmark includes a set of nine benchmark tasks for dineesninaiii molecular structures and establishes consistent set of input features and parameter count to be used across all tested methods. The authors also compare different representations by evaluating neural networks based on 3D grids, graphs, and equivariant networks on the proposed tasks.
Beyond using a single representation for proteins, the simultancous representation of a protein as several mathematical objects holds promise. Indeed, different representations encode different biolog- ical priors of the data and present different computational advantages. A well-studied combination the use of sequence information along with a graph representation of the structure. For instance, in Hermosilla et al.] (2020); Fan et al. (2022) the authors enrich the graph with additional edge types that encode the sequence. Another way to include sequence information in a graph, is to use sequence embeddings, especially ones derived from protein language models and hence benefiting from the large amounts of available sequence data (Wu et al. 2023 Zhang et al. (202)). Finally, some approaches include information derived from protein structures in the training of protein language models (Bepler & Berger 2019 Heinzinger et al. 2023 Su et al. 2023
MOTIVATION AND CONTRIBUTION
Despite this recent progress in comparing different representations for learning on protein data, relatively less focus has been given to surface-based representations, even though they have shown promising results in several applications (Gainza et al. 20200] Sverrisson et al., 2021] Wang et al.) 2023). Approaches based on the surface representation have typically followed the initial MaSIF paper validation (Gainza et al. 2020). and hence have never been directly compared to other representations in the context of single well-established benchmark. At the same time, powerful tarface-based encoders have recently been proposed in the geometry processing/computer graphics literature, such as DiffusionNer (Sharp al. 2022) significantly outperforming, in terms of both robustness and accuracy, the early Geodesic- Mondesic-CNN based techniques (Masci et al., 2015) Monti et al. 2017) which formed the basis of (Gainza et al.) 2020). Unfortunately, it is not currently known how the best currently available surface-base based encoders compare other dearning-based paradigms in the protein analysis tasks (c.g., on the At omd benchmark).
We fill this gap by first adapting the current cate-of-theear surface- based learning architecture to protein analysis tasks. We then perform the first fair and comprehensive comparison of a pure surface-based learning method, while adhering to the benchmark protocol, with fixed input features and parameter count. A key finding of this analysis is that surface-bass encoders are competitive, but do not provide state-of-the-art results.
2

---

Published as a conference paper at ICLR 2025
We then focus on exploring whether surface- basiee learning for protein analysis can provide comple- mentary information to that of other representations. Related efforts have been made in the recent literature; in Lee et al. (202)), the authors propose to use implicit representation of the surface as a pretraining objective. Sommath et al. (202)) proposed to first encode surface properties and use them initial embeddings for the graph nodes, while Pegoraro et al. (2024); Xu et al. (2024) averaged the predictions made by a surface-base and a graph-based model. Nevertheless, those efforts consider surface and other (e.g., graph-based) learning separately and only aggregate results in global manner (early or late fusion (Karpathy et al., 2014)). Instead, we show that by creating an integrated approach, in which the features are shared and passed between a surface and graph representation, even within the middle layers, allows to significantly boost performance and improve results. Crucially, by exploiting the natural spatial relations based on proximity that exist between graph nodes and surface vertices, we show that node-wise feature sharing creates synergy between the two representations. Furthermore, we demonstrate that by using embeddings from text encoders as input features, coupled with careful and efficient architecture design, it is possible to achieve unprecedented state-of- oate-of-thhearr art results on a wide range of tasks.
To summarize, our key contributions include:
An adapted design of the recent -fate-of-the- art DiffusionNer architecture, which addresses some of its limitations (including instabilities and scale independence) in the context of protein analysis tasks.
The first comprehensive comparison of surface-based learning against alternative representa- tions such as graphs or grids within an established benchmark.
A novel integrated approach, which is based on node-wise feature-sharing across all learned layers, between surface and graph-baset encoders that are learned jointly.
state-of-the-art results in wide variety of challenging scenarios and enhanced computa- tional throughput by using residue graphs and coarsened meshes.
The rest of the paper is organized as follows: in Section 3.1 we present the surface representation we use and the specialized networks used to process it. Section 3.2 highlights challenge for surface networks learning on various scales and presents solutions to mitigate these issues. In Section 3.3] we propose to synergistically integrate graph and surface information within unified architecture, harnessing the power of both representations. We provide the details regarding the chosen architecture in Section 3.4 and analyze its computational aspects in Section 3.5
METHODS
3.1 SURFACE REPRESENTATION LEARNING
Our first objective is to study the utility of the best existing surface encoders for learning on protein data. To generate the surface representation Sp of the protein P. we rely on MSMS and on mesh coarsening and cleaning steps, detailed in Appendix C.I At the basis for our surface-based learning method, we then employ the DiffusionNer approach (Sharp et al. 2022).. This method has been proven to be highly robust and effective across diverse set shape analysis tasks (e.g.. (Attaiki) et al. 2021, Cao & Bernard, 2022 Sun et al.] 22023] Li et al. 2022) among others). In particular, DifjusionNet avoids using local patch parametrizations, which lead to instabilities across different mesh structures and enables long range and multi-scale information propagation by using learned diffusion for information sharing. The mathematical foundation of DiffusionNNtt is the heat equation, which simulates the diffusion of heat, or, equivalently, the behavior of Brownian motion on a surface over time. For a surface S. let f. : S - R be the function that defines the heat distribution over S at time and As the saplace-Beltrame operator (Meyer et al. 2003 Vallet & Levy 2008) of the surface. The heat equation is the linear differential equation below (equation D.. solved by diagonalizing the Beltace-Beltrami, In practice, we store the A 128 smallest eigenvalues of As in a diagonal matrix R with the corresponding eigenvectors in E B and vertex area weights M E Raxx The spectrally truncated solution to the heat equation is given as f, De -e (TTTMM M fo.
Of Asf.
(Heat equation)
(1)
at
3

---

Published as a conference paper at ICLR 2025
In DiffusiinNet. this equation is used to perform information propagation surface using feature map as fo and learning the diffusion time in a task-specific manner. This mechanism relies on dense linear algebra operations, offering straightforward differentiation with respect to both f and t. The diffusion layers are combined with features based on f, their spatial gradients and standard, point-wise MIPss. This leads an architecture which can capture multi-scale geometric details of surface in task-specific manner.
3.2 ADAPTING TO PROTEIN SURFACES OF DIVERSE SCALES
As we demonstrate in Section beloow our first empirical observation is that applying the DiffusionNer architecture directly to protein datasets, without modifications, yields relatively poor performance. We attribute this primarily the fact that the initial DiffusionNer architecture targeted applications involving related near-isometric shapes (e.g., humans in different poses). A key technical issue is that most existing approaches using DiffusionNee's normalize all shapes to uniform surface area. This step ensures that shapes have the same global scale which stabilizes learning. Unfortunately, in the context of protein analysis, tasks such as binding preference determination depend critically on the relative sizes of proteins and ligands, making considerations on the scale of proteins essential, and global scale normalization would lose this precious information.
On the other hand, the efficacy of DiffusionNett' receptive field is contingent upon the diffusion times learned within each diffusion layer. Without scale normalization variations in the sizes input shapes can lead to discrepancies in the network's 's learned receptive field. This is elucidated by the following well-known proposition, whose proof is provided in the supplementary material for completeness:
Proposition 3.1. Let X be shape and Y aX its scaled version by a factor >> O. Denoting by F. x) the expected geodesic distance for Brownian motion starting from point x: after time t, it holds that: (t, = aEx (Z)
Importantly, this result suggests that the time parameter of diffusion must be adapted depending on the scale, whereas in DiffusionNer the learned time parameters are shape independent. We also remark that in scenarios involving non-isometric surfaces, such as proteins that might have different scales, learned diffusion, especially if is learned without biological considerations, can generalize poorly across highly diverse shapes, and lead to instabilities during training.
To address this issue, we enhance the original DiffusionNer framework in two ways. First, we enable support for batch (the original model was limited to batch sizes of one) and incorporate a Batch Normalization layer (loffe & Szegedy 2013) after each diffusion layer to stabilize learning. Second, facilitate the optimization process by incorporating biological priors relevant to spatial scales. Consequently, we determined that diffusion times around 10 resulted in receptive fields around 10 Á (see Supplementary Figure 2). which aligns with the spatial scale of binding sites. Inspired by the inherent multi-scale nature of protein structures, we opted to draw samples from normal distribution t ~ ~1,10 5), characterized by a relatively high variance. The absolute values of these samples were then utilized the initial values for our diffusion timescales. Both the large scale and the large variance are retained during training, as illustrated in Supplementary Figure(3). enabling efficient multi-scale and long-distance message passing. These enhancements DiffussinNNet implementation mitigate the instabilities in the training process (as seen in Supplementary Figure 2 and are available in the provided code repository and as a pip package.
3.3 HYBRID REPRESENTATION LEARNING
As mentioned earlier, beyond assessing the efficacy of surface-based learning compared to other representations for protein learning, we explore the benefits of integrating different representations in unified framework, harnessing their distinct strengths. Intuitively, surface representations can capture the intricate geometric details critical for tasks involving protein interactions, while graph representations detail the specific atomic interactions within a protein's interior that indirectly influence its surface dynamics and interaction capabilities. Furthermore, these representations facilitate complementary approaches to learning: local message passing through graphs and global information dissemination for surfaces via learned diffusion. Inspired by these considerations, we propose method that enables feature sharing between graph and surface-base based representations. As
4

---

Published as a conference paper at ICLR 2025
emphasized below, unlike previous related approaches (Somnath al. 2021 Pegoraro et al. 2024 we enable communication between the two representations across all learned layers of the network.
As basis for our hybrid representation. in addition to the surface Sp, construct graph repre- sentation Gp = (Vg. Eg), We use either a graph whose nodes are atoms, which aligns with the one used within the otom3d benchmark, enriched with ESM-650M (Rives et al. 2021) sequence embeddings used as node features.
defined at the residue level. The residue-leve graph is
To construct our hybrid approach, then build bipartite graph G EV, where V UV, represents graph nodes and surface vertices, respectively. For each vertex on the surface, we find its 16 nearest neighbors in the graph and add the corresponding bidirectional edges in the bipartite graph. We provide a more detailed description of the construction and features of the atomic, residue and bipartite graphs Appendix C.T
We now define block operations to encode protein using Sp, Up and G. Denote encoders on surfaces and graphs as sg and go. respectively, and the set of input features as X {I,, n E V)- The corresponding encoded features are 7 {h,, E V/ with h so so (2,) for nodes n E Vs Vg. Our general methodology incorporates message-passing neural
and h., = (r(x,) ) for nodes networks, denoted MPs. over the bipartite graph G. such that at a layer we get X MP (H'). By employing distinct sets, O2g and Ogs. the architecture handles messages traversing from the surface to the graph and vice versa. Those block operations can be stacked as shown in Figure [ We emphasize that our feature sharing occurs the local (node) level and is enabled by the proximity relations in 3D space. Moreover, our hybrid approach trains both representations jointly, while enabling information sharing across all network layers, which is crucial to its success.
90
MP 0gg
g0
MP 0sg
S0
MP g8
S0
MP 0
ge
Illustration of our approach integrating surface and graph information. We ensure joint learning across the two representations and enable information propagation across all layers of the network. Our information sharing is based on the spatial proximity relations between individual graph nodes and surface vertices (not shown here).
Figure 1:
3.4 PROPOSED ARCHITECTURES
Our framework incorporates surface encoding blocks, 8g. which consist of a diffusion operation followed by pointwise neural network with two hidden layers of specified width. The first network we propose, Surface Diff is only based only on those surface blocks. It is used to assess the relevance of the surface representation used isolation. Note that Surface Diff based on DiffusionNer but incorporates our modifications mentioned in Section 3.2 For all other methods that use hybrid approach, the widths for both encoding blocks so and go were consistently set to equal values.
We introduce following the om 3d benchmark protocol. Its graph encoder go consists of Graph Convolutional Networks (GCN) (Kipf Welling 2016), intertwined with Batch Normalization operations and its message-passee passing over the bipartite graph is Graph Attention Layer (Veličkovié et al. (017). Following the At om3d benchmark standards, At omsurf -bench has 200k learnable parameters and does not use surface input features, only considering the atom type onto its atomic-level graph representation.
model aimed comparing representations fair way by
In addition, introduce At f that makes of the aforementioned residue graph, along ProNet (Wang et al. (2022a) encoder. Despite acting on residue graphs, with the featurization of the geometric conformation of the atoms composing each residue, ProNet adds
5

---

Published as a conference paper at ICLR 2025
based on relative local coordinate systems that uniquely and equivariantly identify the conformation, allowing for completeness (as introduced in ComeNet (Wang et al., (222bb). Those features are then embed using spherical harmonics, and ProNet relies on the message passing introduced in GraphCony (Morris et al. 2019) to perform learning. In addition, we perform our message passing using the aforementioned bipartite graph features and GVP (Jing al., 202T) encoder.
Different bipartite message passing networks and organization are possible, encompassing as special cases several existing approaches. We provide the implementation details in Appendix We evaluate these configurations and conduct several ablation studies on our final model, in Section 4.5
3.5 COMPUTATIONAL ENIIANCEMENTS
Surface-methods are traditionally thought to be compute-expensive methods, motivating approaches side-step their intrinsic complexity (Sverrisson et al.) (021). Through a complexity analysis presented in Appendix D.I] we found the number of vertices to be critical in DiffusionNel runtime, which we addressed by coarsening our meshes. In this coarse surface regimen, the graph encoding with ProNet is the computational bottleneck, which is smaller than when used in isolation. Hence, we claim that our method is faster than this efficient graph encoder in the coarse surface regimen.
From the storage and memory perspectives however, the memory footprint of the surface-relate operations dominate the ones originating from the graphs, especially because of stochasticity. We did not find it to be limiting in terms of I/O, but rather in terms of batch size. We implemented a dynamic batching procedure, alleviating our memory issues. Further details are provided Appendix [D.2 Finding ways to reduce the memory footprint of surface networks remains important direction.
RESULTS
4.1 PERFORMANCE OF SURFACE REPRESENTATION
We start by validating our surface encoder on the RNA segmentation benchmark (Poulenard et al. 2019) for surface methods. The task is to segment 5s ribosomal RNA molecules into functional components. This dataset consists of 640 RNA surface meshes of about 15k vertices, and was already used to compare modern surface encoders. We assess the impact the proposed enhancements to DiffusionNer by showing the learning curves of the enhanced models the RNA segmentation task (see Figure 2 and Appendix [F.] for a similar analysis on PSR). Moreover, we compare their performance other recent surface encoders, DGCNN (Wang al. 2019) and DeltaCony (Wiersmal et al. 2022) and report results in Table [] Our experiments show that our adjustments significantly improve the DiffissioNNr stability issues, and allow it to converge to a model with a test accuracy of 84.1% 5% instead of of 80.9%. This accuracy is significantly higher than DGCNN (74.7% and DeltaConv (78%).
10'
DilfinsunNel
DiffisionNe (enbancol)
Method
Accuracy
DGCNN
74.7
10""
DeltaCony
78
DiffusionNet (original)
80.9
0.50
DiffusionNet (enhanced)
84.1
Steps
x10
Figure 2: Learning curve on the RNA segmentation using the original and our enhanced DiffusionNet models.
Table 1: Performance of Different Surface
Encoders on the RNA Segmentation Task.
Then, to compare the performance of our surface encoder to the use other representations, we turn to the Add benchmark, focusing on its three tasks exclusive proteins. The Protein Interaction Prediction (PIP) task aims to predict which part of a protein interacts with another, and holds about 120k examples. The Mutation Stability Prediction (MSP) task is to determine if a mutation enhances the stability of protein-protenn protein interaction (5k examples). Finally, the Protein Structure Ranking (PSR) task aims to assign quality score to predicted protein structures with around 10k proteins.

---

Published as a conference paper at ICLR 2025
A more detailed description of all our datasets is available in Appendix
B
We anticipate that the surface representation will excel in tasks involving interactions, like PIP, but may underperform in the PSR task, which is heavily influenced by subtle internal changes in the protein volume that might not affect the surface. The results are presented Table
Surprisingly, observe that the surface method Surface Diff, despite using state-of-the- surface encoder, sistently falls short in its performance, interaction
Table 2: Comparison of different representations, includ- ing surface performance. Dashes in the equivariant meth- ods' column indicate that these methods could not be used due to memory constraints.
even on the protein-proted task. Such an observation challenges as- sertions in previous purely surface-based methods, and highlights the importance of direct benchmarking general. De- spite promising modeling of protein in- terfaces, which are intrinsically surface objects, the network could not reach satisfactory test performance. We em- phasize that all networks are trained in a vanilla setting, in particular, unlike MaSIF, input features minimalistic. Surface networks may excel when supplemented with richer information. However, when all input features are held constant, surface networks do not emerge as top performers.
PIP Auroc
MSP
PSR
Auroc
R
Rg
3DCNN
0.844
0.574
0.431
0.789
ENN
0.574
Graph
0.669
0.609
0.411
0.75
Surface Diff
0.837
0.5
0.33
0.643
AtomSurf-bench
0.876
0.707
0.452
0.831
4.2 SYNERGY IN COMBINED REPRESENTATIOON
In this section, we assess the performance of our proposed hybrid methods, which have the particular- ity of combining surface and graph representations. First, we use the same experimental setup and compare our At AmSmur ch to the previous models exclusively grounded in one representation. Our results are reported in the last row of Table
Our method outperforms single modalities in all three tasks, while adhering to the At Atom3d bench- mark protocol. This result highlights the synergy between the two representations. Achieving this is noteworthy because, first, the input features remain minimal, and second, to maintain a consistent parameter count, both the graph and surface encoders are considerably condensed. An interesting result is that even in the case of PSR, where surfaces do not intuitively seem relevant, the mixed model outperforms its graph counterpart with a comfortable margin. One possible interpretation for this result is DiffusionNet's ability perform long-range message passing.
In addition, we compare to popular, state-of offethe-art models outside of the benchmark constraints. Namely, we compare against ProNet (Wang al.] 2022a) and GearNet (Zhang al. 2023), which are graph-based methods at the residue level, but encode the local geometry of residues as features; include their extension with features derived from protein language models (Zhang et al. 2023). Please note that differing parameters and training procedures do not necessarily follow the protocol of the original benchmark. For this reason, we also include the performance of our second proposed model, At attouuuf f. which uses all input features and more recent encoders. Out of fairness, we follow exactly the training protocol and problem formulation for each considered task, and use externally reported performance for other methods. The results are presented in T
and the atomic-level extension of GVP (Jing al. 2021). Finally,
Our first result is that even AtomSurf f-bnnhh is competitive with more highly engineered ap- proaches and outperforms them in two of the three evaluated tasks. even outperforms pretrained methods the PIP task. The PSR task, which is aimed at detecting non-canonical protein structures, appears to benefit methods that integrate explicit biological insights into protein geometry, such as the computation of side-chain angles.
Moreover, At fmSurf successfully results another performance boost on this benchmark, setting clear saate-of-the-eee performance the PIP and MSP tasks, and closing the gap PSR. Importantly, our method does not rely on pretraining and uses less than half the number parameters than any other method.
7

---

Published as a conference paper at ICLR 2025
Table 3:
Performance of our hybrid approach on benchmark tasks, compared ofate-of-the-art models. Best result in bold, second best underlined.
PLM
Params
PIP Auroc
MSP Auroc
PSR
R
Rg
ProNet
2M
87.1
63.2
84.9
GVP
11M
86.6
68.0
51.1
84.5
pretraining
11M
87.4
71.1
51.5
84.8
GVP-ESM
XX
11M
61.7
86.6
GearNet-ESM
42M
68.5
82.9
pretraining
42M
70.2
86.3
AtomSurf-bench
200k
87.6
70.7
45.2
83.1
AtouSu
600k
90.9
71.6
61.7
85.7
4.3 EVALUATION ON PROTEIN INTERACTION PREDICTIONS
In addition to the Atom3d benchmark tasks, we evaluate our approach on the task of protein binding sites prediction. Binding sites are regions of protein surfaces where partner (that can be another protein, an RNA molecule, a small molecule) interacts with the protein. fine characterization of those regions helps in understanding protein functions and plays significant role in drug design.
Masif-ligand We start by evaluating our proposed approach on the task of ligand-binding preference prediction for protein binding sites, introduced in (Gainza et al. 2020). It holds 1459 protein structures bound to one the seven most common ligands the PDB. Given binding site, the task amounts to predicting its corresponding co-factor. We benchmark our approach against MaSIF (Gainza et al.) 2020) and HMR (Wang et al. (023). with the latter being recognized as state-of-the-an on this task. Additionally, considering the notable performance of ProNet on the previous benchmark, include in this experiment. We use balanced accuracy (the average recall achieved for each class) as our performance metric consistent with prior studies.
As depicted in Table 4 our method surpasses the existing methods and sets a new state-of-the-art for this task. ProNet achieves disappointing AuROC of 0.75. Unlike HMR, which relies solely on surface representation, our results further validate the efficacy of synergistically combining surface and graph representations, showcasing that this integration leads superior performance greand-binding pocket classification.
Antibody epitope prediction Improved understanding and ability to predict the interaction between an antibody and its target, denoted as the antigen, has direct applications in antibody-baan based treatments, which represent a highly promising therapeutic avenue (Kaplon et al., 2023 Jamali et al. 2024). Pegoraro et al.] (2024) introduced dataset containing 235 antibody-ante antigen complexes. The task consists in predicting interacting residue on the antibody and antigen, from their two structures. The method proposed in their article, denoted GEP, is relevant comparison to ours, because it averages predictions made by surface model and by graph-based model. We present the results in Figure
AuROC
15
0.3
RooaBazk
MaSIF Gainza et al.
2020
0.74
04
Pronet
Wang al.
2022a
0.75
030
HMR
Wang et al.
2023
0.81
0.74
12
AtomSurt-bench
0.84
AtomSurf
0.88
0.54
Ab AuROO
Ag AvROE
AB MCE
Ag MCC
Table 4: Balanced accuracy of hybrid approach on the MaSIF-ligand task.
Figure 3: Performance of model the binding site detection task.
8

---

Published as a conference paper at ICLR 2025
As can be in results, AtomSurf has comparable performance GEP the antigen binding site prediction, but outperforms with a comfortable margin on the antibody side (+0.25 MCC points). An accurate prediction of the antibody residues involved in the antigen recognition is key for antibody specificity optimization.
Validation PINDER In addition to this relatively small dataset, validate approach
the recently proposed, large-scale PINDER dataset (Kovtun et al.) (024). We used the clustered version of this dataset that holds around 42k structures. Systems split rigorously, based interface similarity. Moreover, test systems are also available in their unbound form (Apo setting) and as predicted by AlphaFold2 (AF2 setting), representing more realistic use case. We consider a first task formulated as PIP (Pinder-Pair) of predicting interacting pairs of residues and another close Masif-Site (Gainza al. 2020), only taking one protein as input to predict interacting residues (Pinder-site). We compare to ProNet in Table and include a comparison of accuracies in Supplementary Table 3
Table 5: Auroc of our method compared ProNet on the PINDER dataset.
Task
Pinder Finder-Pair Apo
Pinder-Site Apo
Split
Holo
AF2
Holo
AF2
Pronet
80.1
78.2
73.5
74.3
70.7
60.6
AtomSurf
92.8
88.4
87.1
88.3
84.2
82
AtomSurf widely outperform the ProNet baseline all tasks, splits and metrics. The performance over the apo set is decreased compared to the holo set, as well as the performance on predicted structures, expected (Huang et al. 2024). However, network is able to retain remarkable accuracy across the board, validating its robustness.
4.4 ADDITIONAL RESULTS
Visualization of predictions We display the interaction probability predicted our model across two protein surfaces, a homodimer and a heterodimer, and plot the results in Figure g Despite minor prediction errors, such the misidentification of residues in the lower region of 3dbh, our results clearly show that the model effectively identifies binding sites on proteins.
Ground
Truth
Ours
Chain
3DBI E
Figure 4: A qualitative view of results: The top row shows the ground truth with interaction sites marked in red, while the bottom row displays our predictions. On the left, the interaction between chains A and B of the system with PDB ID 4ACM is illustrated. The two rightmost columns depict chains and F of the protein with PDB ID 3DBH from two rotated perspectives of 180".
Learned time analysis the Supplementary Fig, and 3 we provide visualizations showing both the relation between the effective receptive field and diffusion times, well the distribution of

---

Published as a conference paper at ICLR 2025
learned times the MSP task. These visualizations demonstrate that long-range communication enabled by learned diffusion is indeed useful, and can contribute to overall performance, thus shedding light on the relative strengths of surface-bate based learning.
Assessment of the reproducibility Supplementary shows the standard error of our estimate of the mean performance of AtomSurf, assessing its reproducibility and the significance of its difference. Across tasks, the performance gap is higher than the standard error.
4.5 ABLATION STUDY
Model ablations Finally, we examine the impact of different design choices on our tasks. In Appendix [ we introduce the sequential and bipart i scenario, which amount to variations in the connectivity of the blocks. Another major design choice the choice of the Message Passing (MP) component. We explore several options, including discarding the geometric notion of neighborhood, allowing for potentially dong-distance message passing (Att. setting), as well as varying the number of blocks. Our detailed results are presented in Appendix F.4
The sequent ial strategy displays underwhelming results (70.7 vs 60.9 on MSP for example), which could root from its incapacity to handle multi-scale, simultaneous message passing. Among the bipartitt taartitte settings, Att. consistently outperformed by the localized message-passing networks. In short, the best setting leverages several blocks of message passing, with enhanced results for the most interconnected networks. In addition, the geometry of the bipartite graph is important, and attentive mechanism generally benefits the optimal mixing.
ESM embeddings ablation Due to the strong performance of protein language models, we also perform an ablation by removing the ESM embeddings from our graph initial node features. Ablated model's 's performance decreases as expected, showing that ESM is truly an important feature. However, without ESM, our method retains state-of oftte-of-the-art the-art performance interaction tasks such as Masif-L (84 vs 81), PINDER (overall +10 AuROC points) and others the PINDER dataset that has stricter splits, the performance drop seems to be reduced. One possible explanation for this result would be that those sequence-derived features contribute to memorization, whereas learned structural properties lead to better generalization. We believe that more in-depth investigation of this phenomenon is an interesting direction for future work.
most tasks, notably on
(detailed results in Appendix [:5). Interestingly,
CONCLUSION & LIMITATIONS
In this paper, we analyzed the utility of the surface representation in machine learning protein structures. We first adapted the design of the recent Diffusionner in the context of protein analysis tasks, and compared it against other methods by adhering to the Atom3d benchmark protocol, revealing both the promise and the limitations of surface-only learning. We then introduced a novel integrated architecture that combines surface and graph representations and achieves state-of-the off-the-art art results across several benchmark tasks. Key to our approach is a node-wise information sharing mechanism, which allows for joint training of graph and surface representations, coupled with localized information propagation across all network layers. Our ablation analysis further highlights that simplistic approaches combining different representations lead to suboptimal performance. We also demonstrated the performance of our approach in identifying antibody-antie and ligand- binding preferences achieving state-of-the-art results.
Our work strongly supports the notion that leveraging multiple representations with unique strengths a promising strategy for advancing protein analysis. Moreover, integrating biological priors both within the learning frameworks and into information sharing across representations seems crucial for achieving high accuracy in challenging scenarios. A validation on real-life scenarios is our next step to fully establish this method. Despite our optimizations, one of the current limitations of our approach is its significant memory requirements, highlighting the need for computationally more efficient surface-based pipelines, in line with suggestions by Sverrisson et al. [202I). Beyond addressing these challenges, our work can help pave the way to more powerful integrated multi- modal solutions for additional tasks within structural bioinformatics. including generative modeling, but also investigating ways in which information sharing across both representations and tasks can lead to improvements in robustness and accuracy.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS & FUNDING
The authors would like to thank Emanuele Rodolà and Marco Pegoraro for discussions that contributed to initiate this project. They also would like thanks Erkan Turan for trying setup more tasks, as well as Victor Laigle for making the Supplementary Figures on batch size variability.
V.M., S.A. and M.O. supported by the ANR Chair AIGRETTE, the ERC Starting Grant No. 758800 (EXPROTEA) and the ERC Consolidator Grant No. 101087347 (VEGA). V.M. was addi- tionally supported by DatalA and Sanofi. Y.M and B.C. are supported by Swiss National Science Foundation grants 310030_ 197724. TMGC-3 213750 and 200020 214843. This work was per- formed using HPC resources from GENCI-IDRE (Grant 2023-AD010613350 and CITAS at EPFL.
BROADER IMPACT
This paper presents work to advance the field of machine learning on protein structure. There is no direct ethical or societal implication of this work, but potential indirect ones, none of which we feel must be specifically highlighted here.
REFERENCES
Souhaib Attaiki, Gautam Pai, and Maks Ovsjanikov. Dpfm: Deep partial functional maps. In 2021 International Conference on 3D Vision (3DV), pp. 175-185. IEEE, 2021.
Tristan Aumentado-Armstonn Latent molecular optimization for targeted therapeutic design. arXiv preprint UrXiv:1809 180.00033, 2018.
Tristan Bepler and Bonnie Berger. Learning protein sequence embeddings using information from structure. arXiv preprint 1902.00990 2019.
H. M. Berman. The protein data bank. Nucleic Acids Research, 28(1):235 242, January 2000. doi: 10.1093/nar/28. 1. 1.235. URL
1.235
Shaked Brody, Uri Alon, and Eran Yahav. How attentive are graph attention networks? arXiv preprint arXiv:2105. 14491,
14491, 2021.
Michael M. Bronstein and Iasonas Kokkinos. Scale-invariant heat kernel signatures for non-rigid shape recognition. In 2010 JEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, June 2010. doi: 10. 00.110999cp:220055599883. URL http:://tx/x doi...doo..oom doi org/
10. 10.1109/CVPF CVPR 5539938
Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18-42, 2017.
Joan Bruna, Wojeiech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and locally connected networks on graphs. arXiv preprint 1 132.6623,, 2013.
Dongliang Cao and Florian Bernard. Unsupervised deep multi-shape matching. In European Conference on Computer Vision, pp- 55-71. Springer, 2022.
Taco Cohen and Max Welling. Group equivariant convolutional networks. In International conference on machine learning, pp. 2999-2999. PMLR, 2016a.
Taco S Cohen and Max Welling. Steerable cnns. arXiv preprint arXiv: 2016b.
Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness. Advances in Neural Information Processing Systems, 35:16344-1635, 2022.
Hehe Fan, Zhangyang Wang. Yi Yang. and Mohan Kankanhalli. Continuous-discrete convolution for geometry-sequence modeling in proteins. The Eleventh International Conference on Learning Representations, 2022.
11

---

Published as a conference paper at ICLR 2025
Piero Fariselli, Osvaldo Olmea, Alfonso Valnncia, and Rita Casadio. Prediction of contact maps with neural networks and correlated mutations. Protein engineering, 14(11):835-88) 843, 2001.
Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop Representation Learning Graphs and Manifolds, 2019.
Pietro Fontana, Ying Dong, Xiong Pi. Alexander B Tong, Corey W Hecksel, Longfei Wang, Tian-Min Fu, Carlos Bustamante, and Hao Wu. Structure of cytoplasmic ring of nuclear pore complex by integrative cryo-em and alphafold. Science, 37666598):aamm9322, 2022.
Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling, Se (3)-transformers: 3d roto- translation equivariant attention networks. Advances in neural information processing systems, 33: 1970-1981, 2020.
Pablo Gainza, Freyr Sverrisson, Frederico Monti, Emanuele Rodola, D Boscaini, MM Bronstein, and BE Correia. Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning. Nature Methods, 17(2):184-192 2020.
Michael Garland and Paul S Heckbert. Surface simplification using quadric error metrics. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques, pp. 209-216, 1997.
Rana Hanocka, Amir Hertz, Noa Fish, Raja Giryes, Shachar Fleishman, and Daniel Cohen-Or. Meshenn: network with an edge. ACM Transactions on Graphics (ToG), 1-14 2019.
Michael Heinzinger, Konstantin Weissenow, Joaquin Gomez Sanchez, Adrian Henkel, Milot Mirdita, Martin Steinegger, and Burkhard Rost. Bilingual language model for protein sequence and structure. bioRxiv, pp. 2023-07, 2023.
Pedro Hermosilla, Marco Schäfer, Matëj Lang, Gloria Fackelmann, Pere Pau Vázquez, Barbora Kozliková, Michael Krone, Tobias Ritschel, and Timo Ropinski. Intrinsic-extrinsic convolution and pooling for learning on 3d protein structures. arXiv preprint arXiv:2007.06235 2020.
Yufei Huang, Siyuan Li, Lirong Wu, Jin Su, Haitao Lin, Odin Zhang, Zihan Liu, Zhangyang Gao, Jianghin Zheng, and Stan Li. Protein 3d graph structure learning for robust structure-based protein property prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 12662-12670, 2024.
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning. pp. 448-456. pmlr, 2015.
Clemens Isert, Kenneth Atz, and Gisbert Schneider. Structure-based drug design with geometric deep learning. Current Opinion in Structural Biology, 79:102548, 2023.
Kiarash Jamali, Lukas Käll, Rui Zhang, Alan Brown, Dari Kimanius, and Sjors HW Scheres. Automated model building and protein identification in cryo-em maps. Nature, pp. 1-2, 2024.
José Jiménez, Stefan Doemr, Gerard Martínez-Rosel Alexander S Rose, and Gianni De Fabritiis. Deepsite: bronein-binding site predictor using ad-convolutional neural networks. Bioinformatics, 33(19):3036-3042, 2017.
Bowen Jing, Stephan Eismann, Pratham N. Soni, and Ron O. Dror. Equivariant graph neural networks for 3d macromolecular structure, 2021. URL http:::///xxxx..rrp/p attp//bss/211 2006.006000 03843
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Židek, Anna Potapenko, al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583 -589 2021.
Hélène Kaplon, Silvia Crescioli, Alicia Chenoweth, Jyothsna Visweswaraiah, and Janice M Reichert. Antibodies to watch in 2023. In MAbs, volume 15, pp. 2153410. Taylor & Francis, 2023.
Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung. Rahul Sukthankar, and Li Fei-Fei. Large-scale video classification with convolutional neural networks. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 1725-1732, 2014.
12

---

Published as a conference paper at ICLR 2025
Thomas N Kipf and Max Welling. Semi-supervisee classification with graph convolutional networks. arXiv preprint 1609:166000220000 2016.
Daniel Kovtun, Mehmet Akdel, Alexander Goncearenco, Guoqing Zhou, Graham Holt, David Baugher, Dejun Lin, Yusuf Adeshina, Thomas Castiglione, Xiaoyun Wang. et al. Pinder: The protein interaction dataset and evaluation resource. bioRxiv, pp. 2024-07, 2024.
Andriy Kryshtafovych, Torsten Schwede, Maya Topf. Krzysztof Fidelis, and John Moult. Critical assessment of methods of protein structure prediction (casp)-round xiii. Proteins: Structure, Function, and Bioinformatics, 1012):101111000 2019.
Youhan Lee, Hasun Yu, Jaemyung Lee, and Jachoon Kim. Pre-training sequence, structure, and surface features for comprehensive protein representation learning. In The Twelfth International Conference on Learning Representations, 2023.
Lei Li, Nicolas Donati, and Maks Ovsjanikov. Learning multi-resolution functional maps with spectral attention for robust shape matching. Advances Neural Information Processing Systems, 35:29336-29328 29349, 2022.
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Salvatore Candido, and Alexander Rives. Evolutionary-scale prediction of atomic level protein structure with language model. bioRxiv. 2022. /RLtttss://aoi. bioRxiv 2022.07.20.500902.
2022. 2077200.503300 500902
Jonathan Masci, Davide Boscaini, Michael Bronstein, and Pierre Vandergheynst. Geodesic con- volutional neural networks on riemannian manifolds. In Proceedings of the IEEE international conference on computer vision workshops, pp. 37-45, 2015.
Mark Meyer, Mathicu Desbrun, Peter Schröder, and Alan H Barr. Discrete differential-geomett operators for triangulated 2- 2-manifolds. In Visualization and mathematics III, pp. 35-57. Springer, 2003.
Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bronstein. Geometric deep learning on graphs and manifolds using mixture model cnns. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 5115-5124, 2017.
Christopher Morris, Martin Ritzert, Matthias Fey, William I. Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. Weisfeiler and leman go neural: Higher-order graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 4602-4609, 2019.
Marco Pegoraro, Clémentine Dominé, Emanuele Rodolà, Petar Veličković, and Andreea Deac. Geometric epitope and paratope prediction. Bioinformatics, 40(7), 2024.
Adrien Poulenard, Marie-Julie Rakotosaona, Yann Ponty, and Maks Ovsjanikov. Effective rotation- invariant point enn with spherical harmonics kernels. 2019 International Conference on 3D Vision (3DV), pp. 47-56. IEEE, 2019.
Charles R Qi. Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652-660, 2017.
Roshan M Rao, Jason Liu, Robert Verkuil, Joshua Meier, John Canny, Picter Abbeel, Tom Sercu, and Alexander Rives. Msa transformer. In International Conference on Machine Learning, pp. 8844-88566 PMLR, 2021.
Alexander Rives, Joshua Meier, Tom Sercu. Siddharth Goyal, Zeming Lin, Jason Liu. Demi Guo, Myle Ott, Lawrence Zitnick, Jerry Ma, et al. Biological structure and function emerge from scaling unsupervised learning 250 million protein sequences. Proceedings of the National Academy of Sciences, 118(15)e2016239113 2021.
Michel Sanner, Arthur J Olson, and Tean-Claude Spehner. Reduced surface: an efficient way to compute molecular surfaces. Biopolymers, 38(3):00--30,, 1996.
13

---

Published as a conference paper at ICLR 2025
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. In International conference on machine learning, pp. 9323-93322 PMLR, 2021.
Nicholas Sharp. Soubaib Attaiki, Keenan Crane, and Maks Ovsjanikov. Diffusionnet: Discretization agnostic learning on surfaces. ACM Transactions on Graphics (TOG), 41(3):1-16, 2022.
Vignesh Ram Somnath, Charlotte Bunne, and Andreas Krause. Multi-scale representation learning on proteins. Advances in Neural Information Processing Systems, 34:25244-25255, 2021.
Hannes Stärk, Octavian Ganea, Lagnajit Pattanaik, Regina Barzilay, and Tommi Jaakkola. Equibind: Geometric deep learning for drug binding structure prediction. In International conference machine learning, pp. 00503-20521. PMLR, 2022.
Jin Su, Chenchen Han, Yuyang Zhou, Junjie Shan, Xibin Zhou, and Fajie Yuan. Saprot: Protein language modeling with structure-aware vocabulary. bioRxiv, pp. 2023-10, 2023.
Jian Sun, Maks Ovsjanikov, and Leonidas Guibas. A concise and provably informative multi-scale signature based heat diffusion. In Computer graphics forum, volume 28, pp. 1383-1392. Wiley Online Library, 2009.
Mingze Sun, Shiwei Mao, Puhua Jiang, Maks Ovsjanikov, and Ruqi Huang. Spatially and spectrally consistent deep functional maps. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14497-14507, 2023.
Freyr Sverrisson, Jean Feydy, Bruno E Corrcia, and Michael M Bronstein. Fast end-to-end learning protein surfaces. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15272-15281, 2021.
Raphael JL Townshend, Martin Vögele, Patricia Suriana, Alexander Derry, Alexander Powers, Yianni Laloudakis, Sidhika Balachandar, Bowen Jing, Brandon Anderson, Stephan Eismann, et al. Atom3d: Tasks molecules in three dimensions. arXiv preprint arXiv:2012.04033 2020.
B. Vallet and B. Levy. Spectral Geometry Processing with Manifold Harmonics. Computer Graphics Forum, 2008. ISSN 1467-8659. doi: 10.111 147744666779000000 2008.0008010111 122.x.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv: 10900 2017.
Limei Wang, Haoran Liu, Yi Liu, Jerry Kurtin, and Shuiwang Ji. Learning hierarchical protein representations via complete 3d graph networks, 2022a.
2207 12600
Limei Wang, Yi Liu, Yuchao Lin, Haoran Liu, and Shuiwang Ji. Comenet: Towards complete and efficient message passing for 3d molecular graphs. Advances in Neural Information Processing Systems, 35:650-664 2022b.
Yiqun Wang, Yuning Shen, Shi Chen, Lihao Wang, Fei Ye, and Hao Zhou. Learning harmonic molecular representations riemannian manifold. arXiv preprint arXiv:2303. (Tt.:233../520,0 2023.
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay Sarma, Michael M Bronstein, and Justin Solomon. Dynamic graph cnn for learning point clouds. ACM Transactions on Graphics (tog), 38(5): 1-12, 2019.
Maurice Weiler, Mario Geiger, Max Welling, Wouter Boomsma, and Taco S Cohen. 3d steerable cnns: Learning rotationally equivariant features volumetric data. Advances Neural Information Processing Systems, 31, 2018.
Ruben Wiersma, Ahmad Nasikun, Elmar Eisemann, and Klaus Hildebrandt. Deltacony: anisotropic operators for geometric deep learning on point clouds. ACM Transactions on Graphics (TOG), 41 (4):1-10, 2022.
Fang Wu. Lirong Wu, Dragomir Radev, Jinbo Xu, and Stan Z Li. Integration of pre-trained protein language models into geometric deep learning networks. Communications Biology, 6(1):876, 2023.
14

---

Published as a conference paper at ICLR 2025
Shiyu Xu, Lian Shen, Menglong Zhang, Changzhi Jiang, Xinyi Zhang, Yanni Xu, Juan Liu, and Xi- angrong Liu. Surface-bul basee multimodal protein- binding affinity prediction. Bioinformatics, pp. btac413, 2024.
Zuobai Zhang, Minghao Xu, Arian Jamasb, Vijil Chenthamarakshan, Aurelie Lozano, Payel Das, and Jian Tang. Protein representation learning by geometric structure pretraining. arXiv preprint arXiv:2203.06612. 2022.
Zuobai Zhang, Chuanrui Wang, Minghao Xu, Vijil Chenthamarakshan, Aurélie Lozano, Payel Das, and Jian Tang. A systematic study of joint representation learning on protein sequences and structures, 2023. URL (RRLhhtts:///rret
15

---

Published as a conference paper at ICLR 2025
APPENDIX
In this document, we compile all results and discussions that could not be included the main manuscript due page constraints.
More specifically, we first provide additional data illustrations in Appendix A
Then, we provide proof for 33o Appendix B Following this, we detail the specifics of our implementation
in Appendix C
Then, we provide an analysis of the computational aspects of our technique in Finally, we provide additional results Appendix FF)
Appendix D
A ADDITIONAL DATA ANALYSIS
In Supplementary Figure !
we illustrate the different representations that exist for protein structures.
SCIPASEO.SVSEN
Supplementary Figure 1: Diverse mathematical objects used to represent a protein structure, se- quences, molecular surfaces (blue), atom-level and residue-level point clouds (red) and graphs (green). Effective machine learning for protein structures hinges selecting the appropriate mathematical representation along by a compatible machine-learning technique.
A. ASSESSMENT OF THE SCALE TO USE
In Figure we plot the results of diffusing a Dirac initial distribution for several diffusion times, over the surface of the MDM2 protein, involved in apoptosis and cancer treatment. On the figure its groove, which corresponds to its binding site. Orange vertices correspond to the ones that have highest probabilities, such that their represents 90% of the probability mass. A diffusion time of ten seems to convey a relevant scale for this binding site.
t==5
t=
Supplementary Figure 2: Visualization of Dirac delta function d, at a point, diffused for several diffusion times.
In Figure plot the diffusion times learnt by network (on the MSP task). We that the network keeps a variety of scales, and has learnt to use large scales, effectively enabling long-distance message passing.
16

---

Published as a conference paper at ICLR 2025
Block
Block
Block 2
Block
Frequency
10
Diffusion Times
Supplementary Figure 3: Histogram of the diffusion times obtained after training.
B PROOF OF PROPOSITION 3.1
Proof. For simplicity we the computations in the discrete setting (on meshes). Everything remains the same in the smooth (surface) setting, however. Let M1 represent shape modeled as triangular mesh, with W1 and A₁ denoting its cotangent Laplacian and area matrices, respectively. Therefore, its eigenvalue decomposition satisfies:
Wi = X A1Øl
Suppose M2 another mesh that is scaled version of M1 by scaling factor a. Then, according to (Bronstein & Kokkinos 2010):
A2 a² A
2 = X- 1/2
2² 0² //a
The heat kernel can be computed as per (Sun et al. 2009
lx v) = Σ expl (xpp(-Attoo()))0.0))
Consequently, we have:
() = Σ exp( (xxp(-A())d(()))(11)
Σ expp(~>//a2t) Exxp(-N/j22od()))((1/
2 (z, v)/a²
If g(x, y) denotes the geodesic distance between two points the surface and y, then:
v) ag-(2: (x. v)-
17

---

Published as a conference paper at ICLR 2025
Denoting at, the expected geodesic distance of Brownian motion starting from after time t, we find:
E² (1,x)
Σ (2)
Σ Nā y) (z, v)a A (1)
Σ M( (1,
E(klp v)a,v)//)))))))))) (z, y)a² A'
Σ Kija (z, ())
(t/a2,,)
Interestingly. if we assume E(l,2) Vl (as in the Euclidean setting), then: (1/2/
avt/a² = -v= (t,
L
C IMPLEMENTATION DETAILS
C.I PROTEIN REPRESENTATION DETAILS
Surface representation To generate the surface representation Sp the protein P, our initial step involves computing the protein surface using MSMS (Sanner et al. 1996). The resulting meshes are usually quite large, but in the rare cases of small proteins, we incrementally increase sampling density until we achieve minimum number of 256 vertices. Then, we employ quadratic decimation (Garland & Heckbert 1997). to embed our surfaces into coarser and more compact meshes, again ensuring minimum vertex count. Coarsening the mesh has impact on computational efficiency and on performance, assessed Sections 3.5 and 4 Finally, we ensure mesh quality by removing non-manifold edges, duplicated or degenerate vertices and faces, well small disconnected components. We compute several surface features: the Gaussian and mean curvature, as well as the shape index each vertex. We also include the normal as well the heat kernel signature (Sun et al. 2009) at each point.
Graph representation addition the surface Sp, we construct graph representation Gp (Vg, Eg). To ensure consistency and fairness in comparison within the At.om3d benchmark, we follow their conventions: each node the graph corresponds to an atom within the protein and edges Eg are defined between pairs of atoms that are within a 4.5 angstrom radius cutoff. The only initial node features are one-hot encoding of atom types.
Independently, we construct another graph representation to be used outside of the benchmark setting. We choose to use residue graphs for computational efficiency: each node in this graph represents an amino-acid residue and edges are introduced between pairs of atoms that are within 12 Á radius cutoff. For each residue, we add one hot encoding of its type, its secondary structure annotation and its hydrophobicity. We also compute sequence embeddings using the ESM-650M (Rives al. 2021 pretrained model and include them node features.
Bipartite graph construction To construct our hybrid approach, we start by building a bipartite graph G = (V, E),, where V = V, UV, represents graph nodes and surface vertices, respectively. For each vertex on the surface, we find its 16 nearest neighbors in the graph and add the corresponding bidirectional edges in the bipartite graph. We also experimented with edges based neighborhood of 8 Á, corresponding the value widely used to define contact maps (Fariselli et al. 2001}. However, distance- cutoffs result in varying number of neighbors which we found to make learning less stable.
geometric
Outside of the benchmark setting, we add edge features to our bipartite graph. Given an edge eg,a 2x
tying graph node to surface vertex with normal n., use its direction Tg
18

---

Published as a conference paper at ICLR 2025
vector edge feature. The associated distance and angle, (M, (g..)) encoded using sixteen radial basis Gaussian functions, and used as scalar edge features in the bipartite graph. We follow a similar protocol for edges going from vertex to graph node.
C.2 MODEL ARCHITECTURE DETAILS
In this section, give more details architectures that explored during investigations. All models rely on a Py PyTorch Geometric (Fey & Lenssen 2019) implementation.
Our initial attempt involves a sequential alternation between surface and graph encoding, referred to as the sequent i al setting. This approach involves a two-step block: first, surface encoding performed and its features are projected onto the graph via message passing, denoted as hg MP (s))))s(XX) Denoting the intermediate graph node embeddings as 7120 {hū E Vgl, they are propagated within the bipartite graph to derive surface embeddings again, using Xout = ax MP as (go (H+)),, where E R is a residual connection. The architecture proposed by (Somnath et al. 202Th fits within this sequential framework, employing just one block and sum pooling for message passing.
However, the sequential approach does not simultaneously leverage the distinct scales (local and global) offered by graph and surface processing. To overcome this, the bipartite approach processes graph and surface features concurrently in separate encoders, and merge them with a message passing incorporating learnable parameters, based the equation: Nout aX (H) MP.g (H).. Those architectures are illustrated in Figure
Graph method
gl
98
g0
Surface method
80
80
80
go
MP 0sg
Sequential method
MP 0g
go
MP 0gg
go
MP 0.g
Bipartite Method (ours)
30
MP gz
80
MP 8
Supplementary Figure 4: Illustration of our approach integrating surface and graph information.
The architecture we used outlined in the main text. We employ our modified version of NiffusionNet (by adapting the original implementation provided by the authors]) for each surface encoder and utilize GCN for the graph networks (using the implementation provided by PyTTrch Geometrid))
The surface methods were trained using only the surface encoder, whereas the mixed methods
19

---

Published as a conference paper at ICLR 2025
incorporated an additional graph encoder and message- paagge---ggnngggg framework. When employing both encoders, we ensure the number of channels is equal for both. Thus, the variables are the number of "blocks" and the number of channels for each block. Our networks were trained in accordance with the parameter counts of other methods, strictly adhering to their optimization protocols, including the number of epochs, learning rate, and batch size.
For the surface methods, we consistently used 3 blocks, with 94, 90, and 96 channels for the PIP, MSP, and PSR tasks, respectively. For the bipartite methods, on the PIP task, we utilized 4 blocks with a width of 1 118; for MSP, 3 blocks with a width of 148; and for PSR, 4 blocks with a width of 160. On the binding site classification task, our architecture featured blocks, each with a width of 128. The repository used to conduct experiments and reproduce these results is enclosed and will be publicly released upon acceptance.
D COMPUTATIONAL ANALYSIS
D.1 COMPLEXITY ANALYSIS
In a DiffusionNNe, the first main operations consist in diffusion step using the equation introduced in the main text: f; bb-AA(("" M), for each feature map. For given hidden dimension h, the complexity of this operation is hence (hlVV,!") with w the matrix multiplication complexity. Then, the two subsequent operations are the pointwise gradient construction and mixing, as well as point-wise MLP of complexity: O V&l). Message passing layers complexity with regards to h and their number of edges N is O N). our case, 16) Val for the bipartite graph.
Given additional time constants in a (1)) kinn kg. kip representing the time needed for spectral and point-wise operations in DiffusionNet, graph encoding operations and message passing over the bipartite graph, the overall model complexity is A 1 + (A + 16 * kap 1 V. + h² E)).
The spectral operation component (iff)) does not depend strongly, but has a strong dependency on the size of the mesh, which is not well addressed by reducing the number of parameters. Empirically, for the latent dimensions considered, this spectral component was the time bottleneck and thus, coarsening the graphs resulted in significant speedups. We adjusted the number of vertices to make the first term of similar magnitude to the others (obtained for a 0. coarsening rate). In this regimen, when dividing the parameter count equally between the three learnt components, we observed that the ProNet execution was the limiting operation, and hence conclude that our method has a similar throughput graph- based method for a fixed parameter count. Moreover, observed that our method seemed to require less parameters overall, and note that the parameter count is split between components, effectively resulting in smaller ProNets than in the pure ProNet network of the benchmark, and hence a slightly faster runtime overall.
D.2 MEMORY AND BATCHING
From the storage and memory perspective however, the memory footprint of the surface-related operations dominate the ones originating from the graphs. We did not find it to be limiting in terms of I/O, but effectively forbids the use of large batch sizes, which can be limiting.
In addition to individual objects being large, the large variance in the object size (see Figure 5) also prevents us from using large batch sizes. Indeed, due to the random composition of the batch, some unlucky batches contain several particularly large proteins, causing spikes in memory usage and -eemory errors. We validate in Figure that using coarsened surfaces, we stay in the linear dependency on the number of vertices for the hidden sizes at stake.
This is illustrated in Figure 7 for which we tracked the memory consumption of our model through one epoch of learning. Observe the memory peaks that result in over 20% excess memory consumption, keeping in mind that this is only one run over one epoch. When training several models for hours, the probability that outlier results in a 50% increase forces user caution.
naive solution is to use a safety margin, aggravating our small batch size issue. We implemented dynamic batching procedure create batches that are balanced in size, alleviating the memory
20

---

Published as a conference paper at ICLR 2025
600
Supplementary Figure 5: Distribution of the num- ber of vertices in dataset.
Supplementary Figure 6: Memory consumption plotted function of the number of vertex, in log-log scale.
448
600
64B
Supplementary Figure 7: Memory consumption monitoring through an epoch, for different batch sizes.
issues originating from batch size variability. Finding ways to reduce the memory footprint of surface networks remains an important direction.
D.3 COMPUTING RESOURCES USED FOR THE PROJECT
Depending on the task, the training takes between a few hours and up to four days (for PIP which is largest data set) on standard setting; CPU workers and single GPU such NVIDIA V100. To achieve our results, and our ablation experiment, we performed about 100 model training (5 data sets, several settings per datasets and some failed experiments).
E VALIDATION DATASET DESCRIPTION
RNA segmentation This task aims to segment 5s ribosomal RNA molecules into functional components, as defined by aligned regions in Multi-Sequence Alignment (Poulenard et al.) (019). This dataset consists of 640 RNA surface meshes of about 15k vertices, split at random. It was introduced as a benchmark to compare surface encoders.
Protein Interaction Prediction (PIP): This task aims to predict which part of a protein interacts with which part of another. Framed as a classification task, pairs residues from two proteins are labeled as positives if they interact and negatives if they do not. The dataset comprises 87k, 31k,
21

---

Published as a conference paper at ICLR 2025
and 15k training, validation, and test examples, split based 30% sequence identity. Moreover, proteins in the test set are presented in their apo conformation.
Mutation Stability Prediction (MSP) The objective here is to determine if mutation enhances the stability of protein-protein interaction. Given a protein-protein interaction structure and its mutated version, this classification task labels the pair as positive example if it exhibits increased stability. This task includes 2864, 937, and 347 examples in each data split, and the splitting is performed based on a 30% sequence identity. For both PIP and MSP, the performance metric the Area under the Receiver Operating Characteristic curve (AuROC).
Protein Structure Ranking (PSR) : PSR is a regression task and aims to assign a quality score to predicted protein structures from the Critical Assessment of Methods of Protein Structure Prediction (CASP) (Kryshtafovych et al., 2019) competition. The PSR data train, validation, and test splits hold 25.4k, 2.8k, and 16k systems respectively. Splits correspond to time split, with more recent CASP competition belonging to the test split. The " Rg" " term represents the mean correlation across all systems and proposals. Meanwhile, "Ri" refers the average correlation for each system.
Masif-ligand: This task aims to predict ligand-binding preferences for protein binding sites, as introduced in (Gainza al. 2020). Binding sites are regions of protein surfaces where small molecules (ligands) interact with the protein. A fine characterization of those regions helps understanding protein functions and plays significant role in drug design and discovery. This dataset consists of protein binding sites that accommodate one of seven co-factors It comprises 1,634 training instances, 202 validation instances, and 418 test cases, split using sequence identity. Given a binding site, the task amounts to predicting its corresponding co-factor.
AbAg: This dataset focuses on binding site prediction in the context of immunology: predicting the interaction between an antibody and its target, denoted as the antigen. Improved understanding and ability to predict such interactions has direct applications in antibody-basee treatments. (Pegoraro et al. 2024) introduces dataset containing 235 antibody- antigen complexes, with 186 for training and 49 for testing, split based on a 70% similarity threshold. The task consists in predicting the residues involved in the binding site the antihody and antigen separately. The performance the antibody side is computed only on the Complementarity Determining Regions (CDRs). They propose different methods and we chose to compare against MIXmean-egmn PiNer that had the most balanced performance.
PINDER:
This is recent, large-scale dataset (Kovtun et al. 2024) that holds over 2M interaction in its full version. We used the clustered version of this dataset that holds around 42k structures and reduces redundancy. Pinder also introduces a strict splitting criterion based on interface similarity. that was shown to reduce leakage. In addition to their bound form (holo), systems in the test set are also available in their unbound form (apo) and as predicted by AlphaFold (predicted), representing a more realistic use case. The number of systems present in the test set is 1955 clusters, with 342 corresponding apo and 1747 predicted structures. This is two orders of magnitude higher than AbAg.
We introduce two tasks related to binding site prediction. Our first task is formulated as PIP (Pinder- Puir): given two interacting monomers taken in isolation, aims at classifying pairs of residues into interacting, and non- -oteinttr.c.ing. The other task is close Masif-Site (Gainza et al. 2020), only taking one protein as input to predict interacting residues (Pinder-site).
F SUPPLEMENTARY RESULTS
F.1 VALIDATION OF THE NEW MODEL ON PSR
We conduct similar analysis of the proposed enhancements to the ribosomal RNA on the PSR task. We depict the obtained learning curves in Figure & and report performance in Table [ Again, we observe that our suggested enhancements result more stable learning.
22

---

Published as a conference paper at ICLR 2025
Gloja 01 valisatioon
PSR
R4
Rg
DiffusionNet (original)
0.17
0.39
0.2
DiffusionNet (enhanced)
0.33
0.643
DefiuienNa
Ppochs
Supplementary Figure 8: Global correlation learning curve on the PSR task before and after applying our enhancements to Diffi-- sionNet.
Supplementary Table 1: Results of our enhanced model on the PSR task.
F.2 VARIATION OF OUR RESULTS OVER DIFFERENT SEEDS
In table 2 we computed the standard error of the mean computed over three replicates. The standard error small compared to the gap of performance with the next method, hinting for statistically significant difference (unfortunately. there are no replicates for other methods to perform actual statistical tests). As expected, the standard error is higher for smaller datasets (MSP, AbAg) than larger ones (PIP, PSR).
Supplementary Table 2: Standard error of our model evaluated on three replicates for each of the task included. As reference, we include the best performing competing model, without pretraining.
Task Metric
PIP
MSP AuROC
PSR
AuROC
Ri
Rg
Runner-up method
87.1
68.5
63.2
86.8
Ours Mean
90.9
71.5
61.7
85.7
Ours standard error
0.088
0.27
0.22
Task
Ab
Ag
Masif Ligand Balanced acc.
Metric
AuROC
MCC
AuROC
MCC
Runner-up method
80
28
72
18
8/
Ours Mean
88
53
67.3
14.3
88.2
Ours standard error
2.1
1.8
1.5
0.3
F.3 PERFORMANCE ON PINDER TASK
Table shows the performance of At AmSuf,, compared to ProNet on the clustered PINDER dataset. In this dataset, the systems are clustered to remove redundancy. Moreover, the splitting strategy is stricter, based similarity score of the interfaces. In addition the canonical test set, the test set systems are also available in their unbound form (apo setting) and as predicted by AlphaFold (predicted setting). This results in report the accuracy and AuROC on both tasks, on each of those splits. Finally, we also report its performance without ESM embeddings.
realistic estimate of the quality of the predictions. We
23

---

Published as a conference paper at ICLR 2025
Supplementary Table 3: Performance on the Pinder Task.
Task
Pinder-Pate Pair Apo
Split
Holo
AF2
Metric
Auroc
Acc
Auroc
Acc
Auroc
Acc
Pronet
80.1
72.5
78.2
71.5
73.5
67.9
AtomSurf
92.8
85.3
88.4
80.7
87.1
80.2
AtomSurf no ESM
93
85.4
87.6
80.2
87
79.9
Task
Pinder-Site Apo
Split
Holo
AF2
Metric
Auroc
Acc
Auroc
Acc
Auroc
Acc
Pronet
74.3
69.2
70.7
71
60.6
56.3
AtomSurf
88.3
82.3
84.2
80.8
82
76.6
AtomSurf ESM
87.6
79.2
82.4
77.1
80.9
78.8
F.4 MODEL ABLATION
Here, we present the detailed ablation study of our method AtooSurrf-eenh. We examine the impact of different design choices on our tasks. All of these results are obtained benchmark settings, both in terms of features (only atom-type input) and of surface encoders and graph encoders (GCNs).
First, we look at the overall architecture of our approach, following Appendix C where we introduce the sequential and bipartite ite scenarios, which amount to variations in the connectivity the blocks. In particular, the HoloProt method proposed in Sommath et al. (2021) falls into the sequential framework with just one encoding block and a McshCNN (Hanocka et al. 2019) for protein encoder. For fairness, replace MeshCNN with encoder and refer to this setup HoloProt.
Another major design choice the choice of the Message Passing (MP) component. We explore the of three possible pessagg-passing networks. Motivated by the success of DGCNN (Wang eet al. 2019), in our Att. setting, we discard the geometric notion of a neighborhood, allowing for potentially long-distance message passing. In this setting, all nodes from the graphs attend to all vertices from the surface. To deal with the incurred computational burden, we use the recent memory-efficien Flash Attention (Dao et al. (022). We also explore the use of more conventional Graph Convolutional Networks (GCN setting) Kipf & Welling 2016) and the use of Graph ATtention networks (Veličković et al. 2017 Brody et al. (2021) for our final GAT setting. Finally, we also try using three or four blocks in our networks, always adjusting the network width to keep the number of parameters constant. Our results are presented in Table
Supplementary Table Ablation study of our method: We compare various architectural designs, and saasagg-pssiing methods for our task.
MSP AuROC
PIP
PSR
Method
MP
AuROC
local R
global
sequential
0.609
0.855
0.319
0.71
HoloProt
0.537
0.824
0.383
0.715
Att. 0,689
0.791
0.402
0.792
bipartite
GCN
0.697
0.868
0.421
0.797
GAT
0.707
0.876
0.452
0.833
The sequentia strategy displays underwhelming results, which could root from its incapacity to handle multi-scale, simultancous message passing. Similarly, HoloProt does not display top performance, suggesting that their results could enhanced by using better-performing mixing strategies.
24

---

Published as a conference paper at ICLR 2025
Among the bipartite settings, Att. is consistently outperformed by the localized message- passing networks. The other scenarios give an overall close performance, with an edge for the GAT network. This is especially true PSR, where surface methods alone were failing. The mixed approach could simply use surface diffusion as an efficient long-distance communication, which could explain why attentive mechanism results in performance boost in this scenario.
In short, the best setting leverages several blocks of message passing, with enhanced results for the interconnected networks. In addition, the geometry of the bipartite graph is important, and an attentive mechanism generally benefits the optimal mixing.
F.5 ESM EMBEDDINGS ABLATION
We conducted an Ablation of our method by training models without the ESM embeddings as input features. We do not change hyperparameters of our method make up for this change, which strongly disrupts the training on smaller datasets, and especially on MSP which fails to train. We report the result Table 3 all tasks but Pinder, and in Table 3 for the PINDER task.
Supplementary Table Performance of our model when ablating the ESM node embeddings.
Task
PIP AuROC
MSP AuROC
PSR
Metric
R
Rg
Runner-up method
87.1
68.5
63.2
86.8
Ours Mean
90.9
71.5
61.7
85.7
Ours no ESM
90
54.5
47
82
Task
Ab
Ag
Masif Ligand Balanced acc.
Metric
AuROC
MCC
AuROC
MCC
Runner-up method
80
28
72
18
81
Ours Mean
88
53
67.3
14.3
88.2
Ours ESM
86
48
68
14.5
84
The performance decreases expected, showing that ESM is truly important feature. However, even without ESM, our method still shows significant results on most tasks, such as PIP, AbAg and MasifLigand. The overall difference was more noticeable MSP for which the ablated network failed to learn a useful signal without ESM. Considering that AtomSurf-bence (which does not use ESM embeddings) gets 70.9 AuROC on MSP, we believe high performance could be recovered by tuning the optimization. Based on this ablation, we conclude that ESM features are an important source of information, but that even without them, integrating surface and graph method gives a strong performance.
smaller datasets, especially
PINDER, we obtain small benefit from using ESM embeddings as node features, most notably on the apo set. However, this gap seems to be quite small on this dataset with stricter splits. One possible explanation for this result would be that those sequence-derived features contribute to memorization, whilst learnt structural properties could better generalize.
F.6 ASSESSMENT OF THE IMPACT OF THE THRESHOLD DISTANCE
To evaluate the sensitivity of our results to variations of the distance threshold used to construct the bipartite graph, we varied this parameter on the PSR task. We report our findings table
6 Our investigation indicates that changes the threshold have moderate effect the model's performance. However, is important to note that excessively large values may lead to of out-of-mmmory emrrs..
25

---

Published as a conference paper at ICLR 2025
Supplementary Table 6: Performance on the PSR task as function of the threshold used.
Threshold (A)
10
20
local R global R
0.445
0.434
0.434
OOM
0.812
0.833
0.815
OOM
26


Reviews:
[{"review_id": "4m7Y2dmP2A", "reviewer": "Reviewer_3ZDU", "summary": "The paper aims to improve our understanding of surface-based learning for protein representations. This is achieved by first adapting the surface encoder from the graphics literature for protein-learning tasks, which is then compared against other representation modalities (graphs, convolutions, equivariance etc). Following this the authors consider learning on bipartite graphs comprising structure and surface, allowing information to be shared at every layer of learning, compared to previous multi-scale approaches. \n\nExperiments on the Atom3D benchmark showcase the improved performance of the method on PPI (Protein-Protein Interaction), MSP (Mutation-Stability Prediction), and PSR (Protein-Structure Ranking) tasks, while using a much smaller model. A further experiment on binding site classification sees the method performance favorably compared to previous surface-learning based approaches.", "questions": "1. Could the author(s) include the uncertainty estimates, measured across 3 experiment runs. I presume running these would not be out-of-scope for the rebuttal timeline, given the small size of the model and the datasets. \n\n2. For the different benchmark tasks, it would be very beneficial to understand / detail how the training splits were constructed. A common practice is to construct splits based on sequence similarity, but these don't capture the structural nuances / differences which are more relevant for OOD generalization.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. There is very limited work on understanding representation learning with protein surfaces, even though it is universally agreed upon as being a suitable choice for many tasks. In that light, the paper is original in its attempt to dissect and improve representation learning with protein surfaces, and their integration with other modalities.\n\n2. The quality of this work is good, showing a careful study of protein surfaces, adapting existing encoders to capture biological mechanisms, and experimental performance with relevant baselines on a well-known (albeit old) benchmark. The text is also clearly written, and generally easy to follow. For experiments, both the experimental comparisons against other representation modalities, and the experiments on antibodies are interesting and promising. \n\n3. Ablation studies are also performed for the different experimental choices, which further helps improve our understanding of protein-surface learning.", "weaknesses": "**Experiments**\n\n1. Atom3D is an old benchmark, without known issues with regards to the strategies used for splitting datasets. There has been some recent efforts [1] in curating tasks and datasets and evaluating structure based protein learning methods. While I understand carrying out an evaluation with this is not feasible within the scope of the current rebuttal timeline, it would be very handy to have this in a future revision of the paper. \n\n[1] Evaluating Representation Learning on the Protein Structure Universe\n\n2. For the BindingSite Identification experiment, there is a missing baseline from dMaSIF which also achieves a 0.87 AUROC.\n\n3. For most of these tasks, the dataset sizes available are typically limited. This makes it paramount to include uncertainty estimates, measured across 3 experimental runs. \n\n**Presentation**\nThe experiments section should include the corresponding table for the ribosomes (rather than in the appendix), or reorder the sections such that it falls under Ablation studies. This would improve the readability and flow of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rKtRILvZ88", "reviewer": "Reviewer_f7eD", "summary": "This work proposes a hybrid approach combining surface and graph-based encoders, an improved DiffusionNet architecture tailored for protein surface analysis, a comparison of protein surface-based representations against protein as graph-based representations and enhanced computational efficiency with residue graphs and coarsened meshes.", "questions": "1. The authors modified the design of the DiffusionNet architecture, which is typically used with geometric representations like triangle meshes or point clouds, and applied it at each point (or vertex). Initially, the protein surface was computed using MSMS, which generates a surface by rolling a virtual sphere (representing a solvent molecule, usually water) over the protein's atoms. Were traditional geometric CV surface reconstruction methods, such as Poisson surface reconstruction, considered to enhance compatibility with the adapted approach (DiffusionNet)?\n2. \"We choose to use residue graphs for computational efficiency: each node in this graph represents an amino-acid residue and edges are introduced between pairs of atoms that are within a 12 $\\unicode{x212B}$ radius cutoff. For each residue, we add a one hot encoding of its type, its secondary structure annotation and its hydrophobicity. We also compute sequence embeddings using the ESM-650M (Rives et al., 2021) pretrained model and include them as node features.\" How were ESM embeddings used for amino acid residue atoms?\n3. The surface encoder is based on the DiffusionNet architecture, which is applied to vertices (either from a point cloud or mesh). However, a more intuitive approach for capturing surface representation could be to use a triangulated mesh-based encoder defined on edges of the surface, such as MeshCNN. While DiffusionNet outperformed MeshCNN in the original study, it would be interesting to include an analysis of MeshCNN's performance within the protein domain.\n4. The ablation study lacks a comparison between random representations and ESM residue representations.\n5. Table 2 presents the number of parameters and indicates if pretraining was performed. If Atomsurf uses ESM embeddings, why it was not indicated as pretrained?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The proposed method shows superior empirical performance on Protein Interaction Prediction (PIP) and Mutation Stability Prediction (MSP) tasks compared to existing methods.\n2. The paper is easy to follow.\n3. The code is provided.", "weaknesses": "I believe the main weakness of the paper is that, in terms of the proposed method, the paper does not provide a significant novel contribution. While the authors argue that direct comparisons between protein surface representations and other forms of representation have not yet been conducted, the idea of using protein surface representations is not new. Additionally, the results for the Protein Structure\nRanking (PSR) task in the Atom3D benchmark demonstrate comparable performance with other approaches that incorporate different representations, such as GearNet and ProNet.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1LrFrV8Uuw", "reviewer": "Reviewer_LVjj", "summary": "## Analysis of surface representations methods in protein representation learning\n\nThis paper studies the utility of a diffusion kernel based surface representation ideas, introduced in DiffusionNet (Sharp et al. 2020 preprint, 2022 published) for protein representation learning. The authors update the original DiffusionNet formulation to (1) allow for batching and (2) initialising the diffusion time parameters around a meaningful value in proteins (10 Angstrom). \n\nThis protein-adapted version of DiffusionNet is then studied (1) in isolation and (2) in combination with residue graph-based representation learning on a handful of protein representation learning tasks:\n\nEvaluated are the protein interaction prediction (PIP), mutational stability prediction (MSP) and protein structure ranking (PSR) tasks of the Atom3D benchmark, as well as an antibody-antigen interaction prediction and a protein-ligand binding site prediction task.\n\nThe contributions of this work are:\n- (1) *Modelling contributions*: The above mentioned (batching, initialisation) adaptations to DiffusionNet and, for the models that work on both surface and residue graph level representations, a layer-wise cross-communication between surface & graph representations via cross attention.\n- (2) *Insight contributions (ablations and benchmark comparisons)*: The authors ablate the use of DiffusionNet-style surface representation learning versus other surface based approaches, as well as combining surface and graph level approaches. They compare to adequate baseline representatives of each approach that are common in the literature. They find that in their hands & on the above mentioned benchmarks their DiffusionNet-style approach together with a residue graph representation based approach outperforms the other methods.", "questions": "C.f. weaknesses above.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Writing: The paper was very well written overall and I enjoyed the read. Statements were clearly made and easy to follow -- well done on the writing and thank you for spending time on it!\n\n2. Benchmarks select relevant baselines: The authors selected relevant baselines (e.g. GearNet for graph-level representation learning, HoloProt for graph-surface combinations, MaSIF for surface based representation learning) that I would have expected to see in such a work. \n\n3. Interesting case of cross-domain method transfer: The authors transfer the DiffusionNet method from the general surface representation learning literature to the protein representation learning literature. As someone from the protein community I was not previously familiar with the DiffusionNet work and found this a valuable bridge and interesting comparison. While the adaptations to the DiffusionNet approach for proteins were minor (essentially constrained to the update of sampling the t values at 10 Angstrom) this adaptation was well reasoned and makes sense from a domain-level perspective.\n\n4. Ablations: I found the ablations to be informative and well constructed and described.", "weaknesses": "1. Clarity in benchmark comparisons: When seeing the comparison to the benchmark methods, I was left with the question whether the authors re-ran training on the same datasets & splits for each method, or whether these numbers were quoted from a table? What are the levels of variation for each? (e.g. +/- resulting from 3-5 different runs). Were all approaches 'tuned' with equal attention to hyperparameters? I would have expected a short discussion of these points.\n\n2. Limitation of these benchmarks: This is less of a direct critique of this paper, and more of the underlying benchmarks that were used (that I believe should be updated). Protein structures tend to not be rigid, and as a consequence surfaces tend not to be rigid either. While at a coarse grained level there is likely less variation (unless there are global conformation shifts), at a 3-8A level there can be significant reorganisation of side chains - relevant for ligand binding and polar interactions. Interaction prediction should therefore be done on 'relaxed' surfaces that were not extracted from protein complexes (as some of the benchmarks do) and holdout sets should be constructed in particular also based on structural similarities of surfaces rather than just sequence homology. I would highly recommend the authors to look into the pinder & plinder datasets & benchmarks, but I do acknowledge that these were just published briefly before this work was submitted. A comparison on these in the rebuttals would be of much interest. Without comparisons like these I doubt the real-world applicability of such surface representations for most practical tasks in biology.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vNxuTMap6L", "reviewer": "Reviewer_dxE8", "summary": "The paper proposes a new model for protein structure representation learning and property prediction. The main idea is to take as input both the 3D point cloud graph as well as surface mesh, jointly process both modalities within the network with information sharing among them, and then use the joint representation to perform prediction tasks per residue or globally. Experimental results show improvements or match state of the art results on all benchmarks evaluated.", "questions": "- Are the tables missing more recent results on ATOM3D? Eg. Table 1 only shows the results from the proposed models as well as the original ATOM3D benchmark. But there have been many other papers which have outperformed the original benchmark results. Is the rationale for not including them that they do not strictly conform to the benchmark protocol?\n\n- I realise that the results on all the benchmarks show improvements and each table has the new model in bold -- this is good -- but how relevant are some of the ATOM3D and other tasks in the current state of the field? Eg. Tasks like PSR seem irrelevant in the post AlphaFold 2 era. For RNA segmentation: I personally don't think this is relevant for real-world problems, but I am  happy to be shown wrong here.\n\n---\n\nMinor comments:\n- Line 287: typo 'embed'", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 2, "strengths": "- The general ideas of share information across the various representations of a biomolecule is simple and appealing. Implementing a tighter coupling between the graph and surface track in the newly proposed networks is a sensible idea. The results support all these design decisions well. I think the community will find the ideas and results in the paper to be useful.\n\n- Well written and well presented overall.", "weaknesses": "- There has been a recent focus in the community on purely sequence-based approaches and protein language models (see the last ICLR for several works on structure-aware pLMs). I would have been interested to see empirical comparisons to at least ESM2 or perhaps more recent structure-aware pLMs. In principle, the sequence contains all the information that represents a protein and pLMs could implicitly be able to learn surface representations. I would at least have liked to see an ablation on the use of ESM2 embeddings in the model. \n    - (In other words: Could the performance gain be coming from the use of ESM2 embeddings as node features? Perhaps yes, as these pLM embeddings have been shown to already be very strong at learning structural representations and for structure-based tasks: https://openreview.net/forum?id=sTYuRVrdK3, ICLR 2023.)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)