Title: DSR: Dynamical Surface Representation as Implicit Neural Networks for Protein
Venue: NeurIPS 2023
Accepted: yes
Keywords: Protein molecular dynamics

Protein surface representation

Implicit neural representation

Signed distance function

Continuous time modeling
Abstract: We propose a novel neural network-based approach to modeling protein dynamics using an implicit representation of a protein’s surface in 3D and time. Our method utilizes the zero-level set of signed distance functions (SDFs) to represent protein surfaces, enabling temporally and spatially continuous representations of protein dynamics. Our experimental results demonstrate that our model accurately captures protein dynamic trajectories and can interpolate and extrapolate in 3D and time. Importantly, this is the first study to introduce this method and successfully model large-scale protein dynamics. This approach offers a promising alternative to current methods, overcoming the limitations of first-principles-based and deep learning methods, and provides a more scalable and efficient approach to modeling protein dynamics. Additionally, our surface representation approach simplifies calculations and allows identifying movement trends and amplitudes of protein domains, making it a useful tool for protein dynamics research. Codes are available at https://github.com/Sundw-818/DSR, and we have a project webpage that shows some video results, https://sundw-818.github.io/DSR/.

DSR: Dynamical Surface Representation as Implicit Neural Networks for Protein

Abstract

We propose a novel neural network-based approach to modeling protein dynamics using an implicit representation of a protein's surface in 3D and time. Our method utilizes the zero-level set of signed distance functions (SDFs) to represent protein surfaces, enabling temporally and spatially continuous representations of protein dynamics. Our experimental results demonstrate that our model accurately captures protein dynamic trajectories and can interpolate and extrapolate in 3D and time. Importantly, this is the first study to introduce this method and successfully model large-scale protein dynamics. This approach offers a promising alternative to current methods, overcoming the limitations of first-principles-based and deep learning methods, and provides a more scalable and efficient approach to modeling protein dynamics. Additionally, our surface representation approach simplifies calculations and allows identifying movement trends and amplitudes of protein domains, making it a useful tool for protein dynamics research. Codes are available at https://github.com/Sundw-818/DSR, and we have a project webpage that shows some video results, https://sundw-818.github.io/DSR/.

1 Introduction

Proteins, which are made up of chains of amino acids, are essential molecules in living organisms. Protein with specific three-dimensional structures are responsible for a wide range of biological processes such as enzymatic reactions, transport of molecules, and structural support. However, proteins are not static objects in vivo, but constantly in motion, with individual amino acids vibrating, rotating, and shifting in response to the change of environment. The biological functions of proteins are fulfilled by their dynamical behaviors, and researchers study the dynamics and functions of proteins by molecular dynamics(MD) simulations[1, 2].

First-principles molecular dynamics use the forces calculated from the electronic states by solving the Schrödinger equation (namely, density functional theory) to compute the coordinate trajectories[3], with programs such as VASP[4] and GPAW[5]. However, the computational expense of MD typically increases exponentially in relation to the number of electronic degrees of freedom. In order to reduce the computational cost, empirical force field (classical force field) models with simplified functional forms are employed in MD simulation, such as AMBER[6, 7], CHARMM[8, 9], and GROMOS[1012], which are designed to capture the bonded and non-bonded interactions between atoms. Besides, machine learning techniques can be employed to fit force fields or potential energy surfaces[13-16]. These force field-based methods are much faster than ab initio methods without exhaustive QM

calculations. However, the integration time step in MD cannot be too large to resolve the fastest motion in the system, for example the vibration of hydrogen-involved bonds whose timescales are as short as a few femtoseconds. This restricts MD from having more speed advantages. To address this problem, coarse-grained methods are being developed to perform molecular dynamics simulations faster and on larger systems by eliminating some unimportant degrees of freedom[17-20]. Fast as it is, coarse-grained simulations are only able to reproduce molecular properties at low resolution, such as Rg 2 , sacrificing the all-atom details. Given this perspective, existing molecular dynamics approaches suffer from either unaffordable computational costs or insufficient precision. Moreover, inherent limitations such as the considerable random fluctuation of simulated trajectories and the consequential low signal-to-noise ratio pose significant challenges in detecting crucial protein functional motion. Even for the current most advanced Anton 3, under the parallel processing of 512 nodes, for a large system of 1 million atoms, only 100 microseconds of MD simulation can be performed each day[21].

These aforementioned methods give thorough understanding of the natures of proteins by modeling protein molecules as discrete particles. But when studying protein-protein interaction or protein-ligand binding, knowledge about surfaces of proteins is what researchers need. To fulfill such demands, a continuous, high-level representation of molecular shape known as the molecular surface is proposed, as shown in Figure 1.Based on the the concept of "key-lock pairs" emphasizing complementarity of molecular surface[22], the molecular surface models a protein as a continuous shape with geometric and chemical features. One repre-

sentative method, molecular surface interaction fingerprinting (MaSIF), is a geometric deep learning method that captures fingerprints important for specific biomolecular interactions, enabling accurate and efficient analysis of protein-ligand complexes and potential applications in drug discovery and molecular design[23, 24]. Furthermore, some works investigated the application of end-to-end learning techniques for the analysis of protein surfaces and the identification of potential functional sites, such as drug-target binding sites[25].

Previously, surface representation was mainly used for studying inter-molecular interactions or assisting molecular docking, and it had not been applied to analysing molecular dynamics simulation data. Inspired by the above works, we propose Dynamical Surface Representation (abbreviated as DSR), a method for modeling protein dynamics by representing protein surfaces with implicit neural networks. Implicit neural representations[26, 27] are a class of deep learning models that can represent complex geometric shapes using a continuous function without requiring an explicit surface representation. It's well-suited for modeling the dynamic shapes of proteins which have complex and varied conformations. Currently, deep learning applications in molecular dynamics (MD) mainly target small molecules, while our method can be applied to very large proteins.

In this paper, we explore the use of implicit neural representations for modeling the dynamic shapes of proteins. To be specific, protein surfaces are represented as a zero-level set of signed distance function (SDF) in DSR. The value of SDF is defined as the minimum distance from a point in Euclidean space to the surface of the object, where the positive value indicates that the point is outside the object, otherwise, it is inside. In addition, we introduce a time variable in DSR model to simulate the change of the protein surface over time, which is the dynamic simulation of the protein. In summary, we use the idea of implicit neural representation and use the neural network as the implicit neural representation of points on the 3D and time region, so as to achieve the purpose of modeling protein dynamic changes in continuous space and time domain.

In summary, our paper's main contribution is two-fold:

We build an implicit neural network that learns SDF from raw point cloud data in 3D + time domain, which is a both temporally and spatially continuous protein dynamic representation.

We have achieved dynamic modeling of large proteins by using surface representations, which overcomes the limitations of previous atomic models or coarse-grained models that were restricted to small molecules or proteins.

2 DSR

In this section, we present DSR, a dynamical surface representation as implicit neural networks for protein. We first introduce the definition of SDF, and then describe how we use SDF to model proteins. The architecture of the whole method is shown in Figure 2.

2.1 SDF Representation

Let Ω = [ -1 , 1] 3 be a spatial domain, τ = [ -1 , 1] a temporal domain, and M t be a 3D manifold embedded in Ω at time t ∈ τ . For any point x = ( x, y, z ) ∈ Ω at time t , the f M t : Ω → R is define as

where d( x, M t ) := inf y ∈M t d( x, y ) , d( x, y ) = ∥ x -y ∥ 2 .

The zero-level set, which is the surface of the protein at time t , is presented by the points where f M t ( · ) = 0 . That is, the points on the protein surface at time t can be represented as set { x | f M t ( x ) = 0 , x ∈ Ω } .

2.2 Modeling Protein Dynamical Surface with SDFs

The protein surface representation is in the form of a 3D shape. In order to utilize the implicit function form of SDF, a typical approach is to partition the space into a grid, and then compute the SDF value for each grid point. However, for irregular objects, the accurate SDF values cannot be computed and thus approximation algorithms, such as the 8SSEDT algorithm, are usually employed. This method is associated with two limitations: firstly, protein surfaces are often intricate and rugged, and estimation errors can result in larger biases after modeling; secondly, the SDF calculation for dynamic models is linearly dependent on time and cubically dependent on resolution, and pre-computed SDFs will incur high computational costs. Hence, an alternative strategy is used herein, whereby SDF is learned directly from the raw point clouds, rather than utilizing pre-computed SDF for supervised learning. This completely circumvents the need for pre-computing SDF, significantly reduces computational costs, and enhances computational efficiency. In the following, we will elaborate on how to learn SDF directly from the raw point clouds.

For a given input point cloud X = { x i } i ∈ I ⊂ R 3 at time t ,with point normal vector data (optional), N = { n i } i ∈ I ⊂ R 3 , our objective is to find the optimal parameters θ of an MLP f ( x , t ; θ ) , where f : R 4+ d z → R , that can accurately estimate a signed distance function to a surface M t defined by the point cloud X and normals N at time t .

The form of our loss function is as follows:

where λ > 0 is a parameter, ∥ · ∥ = ∥ · ∥ 2 is the Euclidean 2-norm, and

promotes the vanishing of f on X , and in the presence of normal data (i.e., τ = 1 ) , ∇ x f to the given normals N .

The first term in the summation part of Eq. (3) indicates that the SDF value of the surface points should be as small as possible. The second term represents the loss between the point normal vector and the ground truth.

The second term in Eq. (2) called the Eikonal term promotes the gradients ∇ x f to possess a 2-norm of unity, i.e. ∥∇ x f ∥ = 1 , which is the gradient property of SDFs. Note that this property is crucial to the loss design, allowing us to avoid using pre-computed SDF values for supervised learning. While its sufficiency has not been fully established in previous literature, we provide a more comprehensive proof in the Appendix B.

The third term z in Eq. (2) is a latent code that we introduce into the model to specify different protein types, and its parameters are learnable. We claim that with sufficient data, we can explore the latent space for greater generalization.

The global minimum of the loss in Eq. (2) will be the solution of the Eikonal partial differential equation, i.e.,

which will also be a signed distance function, where f approaches 0 on X , with gradients N . The calculation of the expectation is based on some probability distribution x ∼ D in R 3 , which will be introduced in Section 3.2.

Throughout the optimization process, we did not use the pre-computed SDF value to supervise the training, which is an end-to-end learning architecture from raw point clouds data to SDF value. In the next section, we will introduce the experiment setup in detail, including the datasets, implementation details, and model evaluation metrics.

manifoldSDF

manifold loss

manifoldpoints

normal loss

DSR

Gradient

time

Model

Module

gradientloss

spacepoints

spaceSDF

predictedSDF

differentlevelsets

gridpoints

DSR

Model

time

3 DSR on large-scale proteins simulation data

3.1 Dataset.

500ns Trajectory Data. This data set[28] currently contains two 100ns atomistic molecular dynamics trajectories of Abeta (40 residues), one wild type and one E22G, following the protocol of the 500ns trajectories published in [29]. The simulation was performed at 310K in a generalized Born implicit solvent. In this work, we used the first 2000 frames of wild type trajectories.

MDAnalysisData. MDAnalysisData collects a set of data resources pertaining to computational biophysics, primarily focusing on molecular dynamics (MD) simulations and the structural and dynamic attributes of biomolecules. We used five of these datasets,

AdK equilibrium dataset[30] of 4187 frames,

AdK conformation transition[31] sampled with two methods:

Dynamic importance sampling molecular dynamics (DIMS MD) of 102 frames[32],

Framework Rigidity Optimized Dynamics Algorithm (FRODA) of 142 frames[33],

A short MD trajectory of I-FABP (intestinal fatty acid binding protein) of 500 frames[34],

A trajectory of the NhaA membrane protein of 5000 frames[35],

Two MD trajectories of YiiP membrane protein, the short one has 900 frames, and the long one has 9000 frames[36]. We opted for the longer one here.

For these 6 trajectories data, we use a fixed size 2000 to crop the frames due to the computational resources, and we keep the full length for those shorter than the cropping size.

GPCRmd. The GPCRmd (http://gpcrmd.org/)[37] is an online platform that incorporates webbased visualization capabilities and shares data. The GPCRmd database includes at least one representative structure from each of the four structurally characterized GPCR classes. The GPCRmd platform holds more than 600 GPCR MD simulations from GPCRmd community and individual contributions. Each system was simulated for 500 ns in three replicates (total time 1.5 µs ). We selected one system per GPCR family, and a total of 4 trajectories were used to train the model, of which trajectory ids are 10792_trj_81, 10840_trj_87, 10912_trj_95 and 15711_trj_791 respectively.

DRYAD_MD. This dataset contains the trajectory data of 23 proteins simulated by Jumper JM[38]. It is worth noting that the protein trajectories within this dataset exhibit significant oscillations, posing a challenge for model training and resulting in a high failure rate. For this reason, we screened out 14 proteins with mean RMSD 2 less than 1.0 for our experiments.

3.2 Implementation Details.

Problem setup. In this paper, we use PyMol[39] to obtain the protein surface representation, the points and normal vectors, to train the model. The goal is to obtain a protein surface dynamic model with respect to continuous space and time.

Architecture. For representing shapes we used level sets of MLP f ( x, t ; θ ) = R 4 × R d z → R , called DSR Model, with 8 layers, each containing 512 hidden units, and a single skip connection from the input to the middle layer. We set the loss parameters to λ = 0 . 1 , τ = 1 , α = 1 e -3 . And the model architecture is as Figure 2. The activation function between fully connected layers is softplus activation: x ↦→ 1 β ln ( 1 + e βx ) , where β = 100 . The initial latent code vector z of size 192, were sampled from N (0 , 1 . 0 2 ) . The Gradient Module was implemented by PyTorch Autograd to calculate ∇ x f ( x ) .

Distribution D . For all experiments, we utilize the distribution D as defined in the IGR[40] for the expectation in Eq. (2). This distribution is determined by taking the average of a uniform distribution and a sum of Gaussians that are centered at X with a standard deviation equal to the distance to the k -th nearest neighbor (where we set k = 50 ).

Level set extraction. We extract the zero (or any other) level set of a trained model f ( x, t ; θ ) using the Marching Cubes algorithm[41] implemented in the python scikit-image package, which can use any large-size grid to achieve any high resolution.

3.3 Evaluation Metrics.

We use three evaluation metrics commonly used in 3D modeling to evaluate the similarity between two 3D shapes from three aspects: volume, distance, and normal vectors, which are Volumetric Intersection over Union (IoU) , Chamfer distance and Normal Consistency (NC) . These three metrics are all normalized to a range of 0-1, providing a comprehensive evaluation of the model's performance from different perspectives.

IoU. IoU compares the reconstructed volume with the ground truth shape (higher is better). For two arbitrary shapes A,B ⊆ S ∈ R n is attained by:

Chamfer distance. Chamfer distance is a standard metrics to evaluate the distance between two point sets X 1 , X 2 ⊂ R n (lower is better).

where

NC NC evaluate estimated surface normals (higher is better). Normal consistency between two normalized unit vectors n i and n j is defined as the dot product between the two vectors. For evaluating the surface normals, given the object surface points and normal vectors: X pred = { ( x i , - → n i ) } , and the ground truth surface points and normal vectors: X gt = { ( y j , - → m j ) } , the surface normal consistency between X pred and X gt , denoted as Γ , is defined as:

where

4 Results and Analysis

We design experiments from three aspects to verify the ability to learn SDF from the raw point clouds, the ability to reconstruct the protein dynamical surface, and the generalization of the model in terms of temporal interpolation and extrapolation. To our knowledge, we are the first to model long-term dynamics of large proteins through surface representation, with no other methods currently available for comparison. Therefore, we conduct the following analysis to verify the learning ability of the DSR model and the effectiveness of protein surface representation. Additionally, to demonstrate that our model is not doing something trivial, we compare our method with linear interpolation on interpolation tasks.

4.1 The Ability to Learn SDF

We show the reconstruction results of training on Abeta protein in Figure 3, which demonstrates the ability of our model to learn the signed distance field. The first row in the figure is the reconstruction results of our model, and the second row is the ground truth. It can be seen that our model has the ability to reconstruct the dynamic shape of the protein at different moments, and can restore a clearer and more delicate shape than the ground truth. Furthermore, we show some videos of this protein on the webpage and discuss the effect of normal vectors on learning process in the Appendix D.

0=#

t=5

t=10

t=15

t=25

t=30

t=35

Note that SDF represents the shortest distance (signed) from the surface of the object, which means that different level sets form surfaces at certain distances from the manifold surface. As shown

in Figure 4 (a), the visualization of different level sets of SDF values predicted by our model is consistent with the meaning expressed by SDF. In addition, our model is continuous with respect to spatial and can use to reconstruct and predict for arbitrary space resolutions. Figure 4(b) shows the reconstruction of a protein at different resolutions at a certain time. We also calculated three evaluation metrics, namely IoU, Chamfer distance and NC, which have values of 0.9361±0.0265, 0.0009±0.0008 and 0.9967±0.0011 respectively.

(a)

(b)

Resolution

[64, 64, 64]

[128, 128,128]

[256, 256, 256]

[512, 512, 512]

4.2 Reconstruction across Time

In this experiment, we validate the ability of our trained model to reconstruct the protein shapes in training frames (see Table 1). For data AdK equilibrium, NhaA, YiiP, and DRYAD_MD, we used the first 1500 frames of each trajectory and diluted them by ten times. That is, selecting 150 frames for each trajectory for training. Similarly, every second frame was selected within the first 80 frames for DIMS, every third frame was chosen within the first 120 frames for FRODA, and every fifth frame was selected within the first 400 frames for I-FABP.

As we can see from the Table 1, the model can reconstruct well in many cases. For example, DIMS, the IoU value is as high as 0.9316, which means that the shape reconstructed by the model is almost completely consistent with the ground truth on the entire timeline.

We analyze the effect of RMSD on model training quality, indicating that data containing high levels of noise and significant vibration pose challenges to model training. As illustrated in the Figure 5, the model shows superior performance on smoother and less noisy trajectory data.

4.3 Generalization Ability

Our model has generative capabilities to predict missing or future trajectories of a protein. Here we obtain different samples in two aspects, interpolation and extrapolation of time.

4.3.1 Interpolation

Our model enable the prediction of protein shape at any given time due to its temporal continuity. This is very useful when we only know the structures of two moments but not what the intermediate process is. During training sample selection, we extract one frame at intervals of several frames to evaluate the model's interpolation capability in the temporal space. To demonstrate that our model

DIMS

FRODA

I-FABP

YiiP

NahA

adk_equi

protG

bba

cspa

ntl9

T0765

ww

T0816

T0855

T0773

gpW

bbl

T0769

T0771

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

IoU

IoU

RMSD

RMSD (

Å

)

DIMS

FRODA

I-FABP

YiiP

NahA

adk_equi

protG

bba

cspa

ntl9

T0765

ww

T0816

T0855

T0773

gpW

bbl

T0769

T0771

10 -4

10 -3

10 -2

10 -1

0.0

0.2

0.4

0.6

0.8

1.0

Chamfer distance

Chamfer distance

RMSD

RMSD (

Å

)

DIMS

FRODA

I-FABP

YiiP

NahA

adk_equi

protG

bba

cspa

ntl9

T0765

ww

T0816

T0855

T0773

gpW

bbl

T0769

T0771

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

NC

NC

RMSD

RMSD (

Å

)

learns the protein's inherent dynamic variations and is not just performing trivial tasks, we compared it with linear interpolation, as shown in Figure 6, even though there were no previous works for direct comparison. The evaluation results of our model and linear interpolation are shown in Appendix Table 4, 5 respectively, and the performance of interpolation has reached a level comparable to that of reconstruction.

DIMS

I-FABP

YiiP

NahA

FRODA

adk_equi

protG

bba

ntl9

cspa

T0765

ww

T0855

T0816

T0773

gpW

bbl

T0769

T0771

0.0

0.2

0.4

0.6

0.8

1.0

IoU

Linear

Our

DIMS

I-FABP

YiiP

NahA

FRODA

adk_equi

protG

bba

ntl9

cspa

T0765

ww

T0855

T0816

T0773

gpW

bbl

T0769

T0771

10 -4

10 -3

10 -2

10 -1

Chamfer distance

Linear

ours

DIMS

I-FABP

YiiP

NahA

FRODA

adk_equi

protG

bba

ntl9

cspa

T0765

ww

T0855

T0816

T0773

gpW

bbl

T0769

T0771

0.0

0.2

0.4

0.6

0.8

1.0

NC

Linear

ours

4.3.2 Future Prediction

Our model can not only interpolate missing frames in the middle, but also predict future dynamics by extrapolating over time. To validate this, we use models trained on the MDAnalysisData and DRYAD_MD datasets to predict future frames. Table 2 shows the prediction results of the model for the next 10 frames and the next 11 ∼ 20 frames on some proteins. Due to space limitations of the main text, the full results are presented in the Appendix E.2.

4.3.3 A Case Study on GPCRmd

Furthermore, we have explored the model's simultaneous simulation of multiple protein trajectories. Here we use the four protein trajectories data in GPCRmd dataset. The model trains different latent vectors to represent different proteins. In addition, in selecting training frames, we have employed another strategy, wherein the interval between the frames utilized for training is gradually increased with the sequence.

As shown in Figure 7, our model can learn the shapes of multiple proteins simultaneously, and it can recover the fine details of the protein surfaces well despite their ruggedness. An interesting finding is that the trajectories obtained from molecular dynamics simulations often have a lot of random jittering or noise, while the trajectories interpolated by our model can well fit the main functional

reconstruction

.06

(a)

(b)

(c)

(d)

motion of the protein without noise. We put the detailed results and discussion in the Appendix E.3, video demonstration on the webpage.

5 Discussions

Molecular surface Representation The molecular surface representation is commonly adopted for tasks involving molecular interfaces[42], where non-covalent interactions (e.g., hydrophobic interactions) play a decisive role[43]. Non-Euclidean convolutional neural networks[44] and point cloud-based learning models[45] have been applied to encode the molecular surface for downstream applications, e.g., protein binding site prediction[46]. However, it has not been applied to molecular dynamic simulation, especially protein. In fact, this work explored the more essential application scenarios of molecular surface representation, that is, we believe that chemical molecules are more suitable to be regarded as electron clouds that can be represented by 3D shapes.

Deep learning molecular dynamics Deep learning is being used more in molecular dynamics simulations, which has shown promising results. There have been several approaches, such as BehlerParrinello network[47], DTNN[48], and SchNet[49], that focus on predicting molecular properties and potential energy. However, these methods are limited to small molecules and may not work for proteins, which have many atoms. To address this issue, several coarse-grained models have been proposed, such as CGnet[17], DeePCG[50], ARCG[51], and a CG auto-encoder[52]. Recent works, flow-matching[53] and DFF[54], have used normalizing flows and diffusion models to model coarse-grained force fields for dynamic simulation of small proteins. Another approach, DiffMD[55], can predict simulation trajectories without energy or forces. However, these methods are currently limited to simulating small proteins due to computational constraints and are not yet practical for real-world applications, while the method in this paper provides a new idea for long-term modeling of large proteins.

Implicit neural representations SIREN[56] and NeRF[26] have led to the popularity of implicit neural representations (INR) for tasks like new view synthesis and geometry reconstruction. INR involves creating a continuous function that can map coordinates to high-frequency information. NeRF's highly expressive nature has allowed it to be effectively employed in numerous fields, such as image processing[57] (e.g., compression, denoising, super-resolution, inpainting), video processing[58], medical imaging[59], etc. Signed distance function (SDF) is a type of INR that characterizes an object's shape well. DeepSDF[60] uses INR to model SDF, and IGR[40] simplifies the modeling of SDF by using its gradient property. Drawing inspiration from the aforementioned studies, our work learns a unified implicit representation of proteins that will facilitate future largescale modeling of protein dynamics like NeRF's contribution in vision.

6 Conclusions and Future work

We introduced a method for learning implicit neural representations of protein surface dynamics in continuous spatial and temporal space. Experimental results show that our method is very effective in reconstructing proteins with smooth dynamic changes, and has a certain ability to interpolate in time.

In addition, we found that our model tends to model major conformation changes of proteins rather than those noise-like random vibrations. The case study of GPCRmd shows that the model has the potential to learn the dynamics of a class of similar proteins, such as a protein family.

The main limitation of this method is that it is sensitive to noise in protein dynamic trajectories, as discussed in Section 4.2. In addition, the generalization of the model needs to be further improved. The first is the temporal generalization, especially future prediction. So far, there is no effective method that can better predict the future in various motion prediction tasks. We believe that modeling motion or velocity fields can help future predictions for a longer time.

We anticipate DSR as a powerful tool for building general-purpose protein surface models, and hope that our work helps shed some light on a more efficient and generalizable protein representation.

Acknowledgments

This work was supported by Mathematical Intelligence Application Laboratory, MIALAB, Institute for Mathematical Sciences, Renmin University of China, Beijing Academy of Artificial Intelligence and Public Computing Cloud of Renmin University of China.

References

Yao Li, Tong Wang, Juanrong Zhang, Bin Shao, Haipeng Gong, Yusong Wang, Xinheng He, Siyuan Liu, and Tie-Yan Liu. Exploring the regulatory function of the n-terminal domain of sars-cov-2 spike protein through molecular dynamics simulation. Advanced theory and simulations , 4(10):2100152, 2021.

Maxwell I Zimmerman, Justin R Porter, Michael D Ward, Sukrit Singh, Neha Vithani, Artur Meller, Upasana L Mallimadugula, Catherine E Kuhn, Jonathan H Borowsky, Rafal P Wiewiora, et al. Citizen scientists create an exascale computer to combat covid-19. BioRxiv , pages 2020-06, 2020.

Richard Car and Mark Parrinello. Unified approach for molecular dynamics and densityfunctional theory. Physical review letters , 55(22):2471, 1985.

Kresse Georg. Vasp group, theoretical physics departments, vienna. Retrieved February , 21, 2011.

Jussi Enkovaara, Nichols A Romero, Sameer Shende, and Jens J Mortensen. Gpaw-massively parallel electronic structure calculations with python-based software. Procedia Computer Science , 4:17-25, 2011.

Wendy D Cornell, Piotr Cieplak, Christopher I Bayly, Ian R Gould, Kenneth M Merz, David M Ferguson, David C Spellmeyer, Thomas Fox, James W Caldwell, and Peter A Kollman. A second generation force field for the simulation of proteins, nucleic acids, and organic molecules. Journal of the American Chemical Society , 117(19):5179-5197, 1995.

Lijiang Yang, Chun-hu Tan, Meng-Juei Hsieh, Junmei Wang, Yong Duan, Piotr Cieplak, James Caldwell, Peter A Kollman, and Ray Luo. New-generation amber united-atom force field. The journal of physical chemistry B , 110(26):13166-13176, 2006.

Sandeep Patel and Charles L Brooks III. Charmm fluctuating charge force field for proteins: I parameterization and application to bulk organic liquid simulations. Journal of computational chemistry , 25(1):1-16, 2004.

Sandeep Patel, Alexander D Mackerell Jr, and Charles L Brooks III. Charmm fluctuating charge force field for proteins: Ii protein/solvent properties from molecular dynamics simulations using a nonadditive electrostatic model. Journal of computational chemistry , 25(12):1504-1514, 2004.

Wilfred F van Gunsteren and Herman JC Berendsen. Groningen molecular simulation (gromos) library manual. Biomos, Groningen , 24(682704):13, 1987.

Lukas D Schuler, Xavier Daura, and Wilfred F Van Gunsteren. An improved gromos96 force field for aliphatic hydrocarbons in the condensed phase. Journal of computational chemistry , 22(11):1205-1218, 2001.

Markus Christen, Philippe H Hünenberger, Dirk Bakowies, Riccardo Baron, Roland Bürgi, Daan P Geerke, Tim N Heinz, Mika A Kastenholz, Vincent Kräutler, Chris Oostenbrink, et al. The gromos software for biomolecular simulation: Gromos05. Journal of computational chemistry , 26(16):1719-1751, 2005.

Stefan Chmiela, Alexandre Tkatchenko, Huziel E Sauceda, Igor Poltavsky, Kristof T Schütt, and Klaus-Robert Müller. Machine learning of accurate energy-conserving molecular force fields. Science advances , 3(5):e1603015, 2017.

Stefan Chmiela, Huziel E Sauceda, Klaus-Robert Müller, and Alexandre Tkatchenko. Towards exact molecular dynamics simulations with machine-learned force fields. Nature communications , 9(1):3887, 2018.

Huziel E Sauceda, Stefan Chmiela, Igor Poltavsky, Klaus-Robert Müller, and Alexandre Tkatchenko. Molecular force fields with gradient-domain machine learning: Construction and application to dynamics of small molecules with coupled cluster forces. The Journal of chemical physics , 150(11):114102, 2019.

Stefan Chmiela, Huziel E Sauceda, Igor Poltavsky, Klaus-Robert Müller, and Alexandre Tkatchenko. sgdml: Constructing accurate and data efficient molecular force fields using machine learning. Computer Physics Communications , 240:38-45, 2019.

Jiang Wang, Simon Olsson, Christoph Wehmeyer, Adrià Pérez, Nicholas E Charron, Gianni De Fabritiis, Frank Noé, and Cecilia Clementi. Machine learning of coarse-grained molecular dynamics force fields. ACS central science , 5(5):755-767, 2019.

Feliks Nüske, Lorenzo Boninsegna, and Cecilia Clementi. Coarse-graining molecular systems by spectral matching. The Journal of chemical physics , 151(4):044116, 2019.

Jiang Wang, Stefan Chmiela, Klaus-Robert Müller, Frank Noé, and Cecilia Clementi. Ensemble learning of coarse-grained molecular dynamics force fields with a kernel approach. The Journal of chemical physics , 152(19):194106, 2020.

Xiang Fu, Tian Xie, Nathan J Rebello, Bradley D Olsen, and Tommi Jaakkola. Simulate time-integrated coarse-grained molecular dynamics with geometric machine learning. arXiv preprint arXiv:2204.10348 , 2022.

David E Shaw, Peter J Adams, Asaph Azaria, Joseph A Bank, Brannon Batson, Alistair Bell, Michael Bergdorf, Jhanvi Bhatt, J Adam Butts, Timothy Correia, et al. Anton 3: twenty microseconds of molecular dynamics simulation before lunch. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis , pages 1-11, 2021.

Ye Li, Xianren Zhang, and Dapeng Cao. The role of shape complementarity in the proteinprotein interactions. Scientific reports , 3(1):3271, 2013.

Pablo Gainza, Freyr Sverrisson, Frederico Monti, Emanuele Rodola, D Boscaini, MM Bronstein, and BE Correia. Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning. Nature Methods , 17(2):184-192, 2020.

Pablo Gainza, Sarah Wehrle, Alexandra Van Hall-Beauvais, Anthony Marchand, Andreas Scheck, Zander Harteveld, Stephen Buckley, Dongchun Ni, Shuguang Tan, Freyr Sverrisson, et al. De novo design of site-specific protein interactions with learned surface fingerprints. bioRxiv , pages 2022-06, 2022.

Freyr Sverrisson, Jean Feydy, Bruno E Correia, and Michael M Bronstein. Fast end-to-end learning on protein surfaces. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15272-15281, 2021.

Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM , 65(1):99-106, 2021.

Chengan He, Jun Saito, James Zachary, Holly Rushmeier, and Yi Zhou. Nemf: Neural motion fields for kinematic animation. arXiv preprint arXiv:2206.03287 , 2022.

Carter Butts. 500ns Trajectory Data for 'Comparative Exploratory Analysis of Intrinsically Disordered Protein Dynamics Using Machine Learning and Network Analytic Methods,' (Grazioli.et.al. 2019), 2022. URL https://doi.org/10.7910/DVN/ERYOZS .

Gianmarc Grazioli, Rachel W Martin, and Carter T Butts. Comparative exploratory analysis of intrinsically disordered protein dynamics using machine learning and network analytic methods. Frontiers in molecular biosciences , 6:42, 2019.

Sean Seyler and Oliver Beckstein. Molecular dynamics trajectory for benchmarking mdanalysis, 6 2017. URL: https://figshare. com/articles/Molecular_dynamics_ trajec-tory_for_benchmarking_MDAnalysis/5108170, doi , 10:m9.

Sean L Seyler, Avishek Kumar, Michael F Thorpe, and Oliver Beckstein. Path similarity analysis: a method for quantifying macromolecular pathways. PLoS computational biology , 11 (10):e1004568, 2015.

Juan R Perilla, Oliver Beckstein, Elizabeth J Denning, and Thomas B Woolf. Computing ensembles of transitions from stable states: Dynamic importance sampling. Journal of computational chemistry , 32(2):196-209, 2011.

Daniel W Farrell, Kirill Speranskiy, and MF Thorpe. Generating stereochemically acceptable protein pathways. Proteins: Structure, Function, and Bioinformatics , 78(14):2908-2921, 2010.

Oliver Beckstein. Molecular dynamics trajectory of i-fabp for testing and benchmarking solvent dynamics analysis, 2018.

Ian M. Kenney, Shujie Fan, and Oliver Beckstein. Molecular dynamics trajectory of membrane protein nhaa, 2018. URL https://doi.org/10.6084/m9.figshare.7185203.v2 .

Shujie Fan and Oliver Beckstein. Molecular dynamics trajectories of membrane protein yiip, 2019. URL https://doi.org/10.6084/m9.figshare.8202149.v1 .

Ismael Rodríguez-Espigares, Mariona Torrens-Fontanals, Johanna KS Tiemann, David ArandaGarcía, Juan Manuel Ramírez-Anguita, Tomasz Maciej Stepniewski, Nathalie Worp, Alejandro Varela-Rial, Adrián Morales-Pastor, Brian Medel-Lacruz, et al. Gpcrmd uncovers the dynamics of the 3d-gpcrome. Nature Methods , 17(8):777-787, 2020.

John M Jumper, Nabil F Faruk, Karl F Freed, and Tobin R Sosnick. Data from: Trajectorybased training enables protein simulations with accurate folding and boltzmann ensembles in cpu-hours, 2018. URL https://doi.org/10.5061/dryad.h9f8sb7 .

Schrödinger, LLC. The PyMOL molecular graphics system, version 1.8. November 2015.

Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman. Implicit geometric regularization for learning shapes. arXiv preprint arXiv:2002.10099 , 2020.

William E Lorensen and Harvey E Cline. Marching cubes: A high resolution 3d surface construction algorithm. ACM siggraph computer graphics , 21(4):163-169, 1987.

Dina Duhovny, Ruth Nussinov, and Haim J Wolfson. Efficient unbound docking of rigid molecules. In Algorithms in Bioinformatics: Second International Workshop, WABI 2002 Rome, Italy, September 17-21, 2002 Proceedings 2 , pages 185-200. Springer, 2002.

Kim A Sharp. Electrostatic interactions in macromolecules. Current Opinion in Structural Biology , 4(2):234-239, 1994.

Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bronstein. Geometric deep learning on graphs and manifolds using mixture model cnns. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 5115-5124, 2017.

Freyr Sverrisson, Jean Feydy, Joshua Southern, Michael M Bronstein, and Bruno E Correia. Physics-informed deep neural network for rigid-body protein docking. In MLDD workshop of ICLR 2022 , 2022.

Stelios K Mylonas, Apostolos Axenopoulos, and Petros Daras. Deepsurf: a surface-based deep learning approach for the prediction of ligand binding sites on proteins. Bioinformatics , 37(12): 1681-1690, 2021.

Jörg Behler and Michele Parrinello. Generalized neural-network representation of highdimensional potential-energy surfaces. Physical review letters , 98(14):146401, 2007.

Kristof T Schütt, Farhad Arbabzadah, Stefan Chmiela, Klaus R Müller, and Alexandre Tkatchenko. Quantum-chemical insights from deep tensor neural networks. Nature communications , 8(1):13890, 2017.

Kristof T Schütt, Huziel E Sauceda, P-J Kindermans, Alexandre Tkatchenko, and K-R Müller. Schnet-a deep learning architecture for molecules and materials. The Journal of Chemical Physics , 148(24):241722, 2018.

Linfeng Zhang, Jiequn Han, Han Wang, Roberto Car, and Weinan E. Deepcg: Constructing coarse-grained models via deep neural networks. The Journal of chemical physics , 149(3): 034101, 2018.

Aleksander EP Durumeric and Gregory A Voth. Adversarial-residual-coarse-graining: Applying machine learning theory to systematic molecular coarse-graining. The Journal of chemical physics , 151(12):124110, 2019.

Wujie Wang and Rafael Gómez-Bombarelli. Coarse-graining auto-encoders for molecular dynamics. npj Computational Materials , 5(1):125, 2019.

Jonas Kohler, Yaoyi Chen, Andreas Kramer, Cecilia Clementi, and Frank Noé. Flow-matching: Efficient coarse-graining of molecular dynamics without forces. Journal of Chemical Theory and Computation , 19(3):942-952, 2023.

Marloes Arts, Victor Garcia Satorras, Chin-Wei Huang, Daniel Zuegner, Marco Federici, Cecilia Clementi, Frank Noé, Robert Pinsler, and Rianne van den Berg. Two for one: Diffusion models and force fields for coarse-grained molecular dynamics. arXiv preprint arXiv:2302.00600 , 2023.

Fang Wu13 and Stan Z Li. Diffmd: A geometric diffusion model for molecular dynamics simulations. 2023.

Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Implicit neural representations with periodic activation functions. Advances in Neural Information Processing Systems , 33:7462-7473, 2020.

Mikolaj Czerkawski, Javier Cardona, Robert Atkinson, Craig Michie, Ivan Andonovic, Carmine Clemente, and Christos Tachtatzis. Neural knitworks: Patched neural implicit representation networks. arXiv preprint arXiv:2109.14406 , 2021.

Yoni Kasten, Dolev Ofri, Oliver Wang, and Tali Dekel. Layered neural atlases for consistent video editing. ACM Transactions on Graphics (TOG) , 40(6):1-12, 2021.

Liyue Shen, John Pauly, and Lei Xing. Nerp: implicit neural representation learning with prior embedding for sparsely sampled image reconstruction. IEEE Transactions on Neural Networks and Learning Systems , 2022.

Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 165-174, 2019.

Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work proposes to model the surface of proteins using signed distance field (SDF) expressed by implicit neural representations. Previously, protein surfaces were studied to characterize protein interactions and ligand binding. In this work, surface representation learning is for analyzing MD simulation data. The network takes in 3D coordinates, as well as time, and returns SDF and is optimized in the typical fashion by minimizing the distance of 3D points on the surface, matching normals, plus Eikonal term regularization. The experiments on curated datasets show that network is able to provide dynamic reconstructions, while the performance degrades on structures that exhibit rapid movements. The networks is able to interpolate between moments and predict future dynamics.\n\nSTRENGTHS:\n1. The work is very-well written. In the introduction section, the relation with previous work and their issues are clarified and the task addressed in the work is well-motivated. Datasets used for evaluation are clearly described improving reproducibility. \n2. Additional visualizations provide in the supplement and main text helps understanding results on modeling dynamics with implicit neural representations. For example, both quantitative and qualitative results shows that with increase of RMSD the reconstructions do not match well with the ground-truth.\n3. The proposed method is evaluated in multiple contexts. For example, its ability to interpolate or extrapolate in time is assessed. Also, a case study shows that one can learn a latent space encode dynamics of similar proteins.\n\nWEAKNESSES:\n1. According to the provided visualizations (e.g. Figure 3), the proposed method could add some non-existent tiny areas close to the structure but disconnected. I believe these artifacts are due to implicitly representing surfaces.\n2. In L211, it is mentioned that the reconstructions are clearer than GT. This is a weird statement as one expect that GT be the most faithful representation of the surface. It is not clear if small rapid movements are due to noise or part of the true dynamics; If the latter, then the inductive bias in the network to output a smooth function is not really a merit of the work. On the other hand, if the GT is noisy, then it is not clear how to quantitatively evaluate the reconstructions; thus such results become less valid and hard to rely on.\n3. It is claimed that, unlike previous work, the proposed method is able to resolve dynamics of large proteins. However, there is no quantitative comparison with previous work to validate this.\n4. It is not clearly stated why implicit representation for surfaces is used, rather than other representations (such as algebraic ones). There is a lack of experimental validation on this, too.\n\nQUESTIONS:\n1. I suggest that you provide more context on why those rapid movements in the structure is noise and not part of real dynamics. I think that it is not fully noise but a blend of noise and true dynamics which is completely eliminated by smoothness inductive bias in the network.\n2. In the interpolation experiments, how do you extract one frame out of several frames? How much is the RMSD between two states that are used as endpoints of interpolation?\n\nLIMITATIONS:\nPlease address the weaknesses and questions above.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to learn an implicit neural network to model protein surface and dynamics. A protein surface is represented as a signed distance function (SDF), which can be modeled by implicit neural networks based on MLP. The training objective follows standard practice: the difference between true and predicted SDF values at each point (both interior and exterior) and an Eikonal term at the surface points to ensure the learned function is a valid SDF. DSR is evaluated on multiple protein dynamic simulation data such as GPCRmd. The model is able to reconstruct the surface with different levels of accuracy, depending on the types of proteins. DSR is also able to predict the future frames given partial protein trajectory.\n\nSTRENGTHS:\n* This paper presents an interesting direction to protein surface. Modeling protein dynamics at the surface level is a novel contribution to the field.\n* Using implicit neural networks to model protein surface is a novel application of existing surface modeling methods.\n* This paper presents a comprehensive analysis over a collection of 20 protein dynamic datasets, which can be a valuable resource to the protein dynamic modeling field.\n\nWEAKNESSES:\nThere are two limitation of the method.\n1. Currently, DSR cannot generalize to a new protein. In the current set of results, DSR is trained and tested on the same protein, i.e., it is trained on a partial trajectory of one protein and used to predict the future of that protein. In practice, we usually want to simulate a protein that we haven't simulated before.\n2. DSR only models the surface of the protein. Although the shape of a protein contains useful information, it is not all. We also need to know the chemical composition over the protein surface, i.e., which part of the surface is more hydrophobic/charged/etc. Predicting the surface itself is not sufficient for downstream analysis. For example, MaSIF (Nature Method 2020) used protein surface to predict protein-protein interaction. They find that their model performs poorly when using only the geometric shape as input. When they add chemical features of the protein surface patches (hydrophobicity, charge, etc.), the performance improved substantially. Therefore, it is necessary to predict not only the geometric shape but also the chemical composition of the protein surface.\n\nQUESTIONS:\nIn section 4.3.3, when you model multiple proteins, do you provide protein type as an input to DSR model? If not, how do you train the model so that DSR knows there are different protein types?\n\nLIMITATIONS:\nThe paper has included a short discussion on limitations and societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
The author puts forwards the DSR model, which is an implicit neural network, uses this model to accurately capture the protein dynamic trajectories and interpolate and extrapolate in 3D and time. The DSR is a scalable and efficient method to modeling protein dynamics, overcomes the limitations of first-principles-based and deep learning methods. As the most important contribution, it is successful for the large protein dynamic study.

STRENGTHS:
1. The method learns SDF well and achieves dynamic modeling for large proteins for the first time. This is an excellent strength.
2. The surface representation approach is useful in protein dynamic field. It simplifies calculations and allows identifying movement trends and amplitudes of protein domains.

WEAKNESSES:
1. In the reconstruction task, there are little comparison between the DSR and other methods.
2.

QUESTIONS:
1. In the background, there are some traditional method which is costly and low speed. Is there some reason or explanation that this method is fast and cheap, other than the application of neural network?
2. In the reconstruction task, there are three evaluation metrics, in what range do the values of these metrics indicate that the model is performing well?

LIMITATIONS:
1. The DSR model is sensitive to noise in protein dynamic trajectories, which is a challenge while faced with data with little smoothness or great noise.
2. As the analysis for the GPCRmd, the DSR model has the potential to learn the dynamics of a class of similar proteins, such as a protein family.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-11

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a framework for predicting protein dynamics through learning the time-dependent SDF. The authors showcase that the learned SDF can reconstruct/interpolate protein surfaces and predict the dynamics, thus bypassing the computationally expensive molecular dynamics simulation.\n\nSTRENGTHS:\n* The paper validates the model by conducting extensive numerical experiments on various protein systems.\n* Learning efficient surrogate models for protein dynamics prediction is important due to the computational cost associated with simulating protein dynamics using MD. The proposed method can potentially benefit both computational biology and drug discovery efforts.\n\nWEAKNESSES:\nThe presentation of the current manuscript can make it difficult to grasp its technical contribution. From the perspective of developing new deep learning models, the paper has combined many existing techniques. For example, parametrizing SDF with neural networks have extensively studied in many neural field works like SIREN/DeepSDF, and the Eikonal term has also been explored in prior work IGR (Gropp, Amos, et al. ). The experiments also do not show how it addresses or improves upon issues in existing deep learning models. For example, what is the advantage of the proposed method over a point-cloud GNN that learns to predict atom-wise displacement? \n\nOn the other hand, if the authors' primary objective is to introduce a new way to predict the protein dynamics for the simulation community, it is also not very clear to what extent the model can be used to replace classical MD. To use the model for prediction, whenever there is a new protein system, one needs to first train the model on its existing trajectory (needs to run MD to get it) and then extrapolate from it. The model cannot do any prediction from scratch on new configuration of molecular system without re-training. \n\nIn addition, the model is neither energy conserved nor equivariant, this might not be a serious issue in fitting and reconstructing a single protein trajectory but can cause instability for the long-term prediction.\n\nQUESTIONS:\n* The units for RMSD and CD in the result section are missing (I assume it is Angstrom).\n*  The latent code \"z\" in the equation 2 is not defined and discussed in the method section.\n\nLIMITATIONS:\nThe authors have discussed the limitation of their work in the conclusion section.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-26"}

comments: SUMMARY:
The paper proposes to learn an implicit neural representation to model protein surface across continuous spatial and temporal space.

STRENGTHS:
This is a great application of implicit neural networks for modeling protein surfaces, and the results seem promising.

WEAKNESSES:
The evaluation of the method can be improved:
- Even though there is no prior work as a baseline, simple methods should still be used to confirm the learned model is not doing something trivial. For instance, what happens if the author just linearly interpolate the 3D coordinates for the experiment in 4.3?
- The reconstructions metrics do not really reflect the usability of the model predicted trajectory. Can we see the correlation for docking on ground truth and generated surface? What about the distance between ground truth and the generated surface in terms of their MaSIF distance?

QUESTIONS:
I think the paper can be greatly improved if the author can address the weaknesses section and I am happy to consider raising my score then.

In addition, the author should consider spending more text to describe the evaluation setup, specifically around how train/test data is split and what has been done to prevent data leakage.

LIMITATIONS:
The author has sufficiently and fairly discussed the limitation of their methods.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-26