SlowGuess/NeurIPS_2023_OCR huggingface dataset:
As we can see from the Table 1, the model can reconstruct well in many cases. For example, DIMS, the IoU value is as high as 0.9316, which means that the shape reconstructed by the model is almost completely consistent with the ground truth on the entire timeline.
Table 1: Evaluation on the surface reconstruction across time.
IoU↑	Chamfer_dist↓	NC↑	IoU↑	Chamfer_dist↓	NC↑
DIMS	0.9316±0.0070	0.0003±0.0001	0.9671±0.0050	I-FABP	0.8907±0.0071	0.0005±0.0001	0.9382±0.0072
YiiP	0.8425±0.0161	0.0003±0.0001	0.8737±0.0164	FRODA	0.8318±0.0286	0.0004±0.0001	0.9098±0.0175
NahA	0.8265±0.0202	0.0005±0.0001	0.8372±0.0245	adk_equi	0.7526±0.0396	0.0020±0.0009	0.7914±0.0308
protG	0.6580±0.0695	0.0036±0.0017	0.8287±0.0595	ntl9	0.6342±0.1244	0.0066±0.0051	0.8165±0.0911
bba	0.6163±0.1481	0.0057±0.0073	0.8158±0.1044	cspa	0.6136±0.0682	0.0074±0.0036	0.8047±0.0527
T0765	0.5997±0.0923	0.0039±0.0022	0.7991±0.0757	ww	0.5920±0.1056	0.0088±0.0055	0.8196±0.0691
T0855	0.5585±0.0898	0.0117±0.0095	0.7804±0.0647	T0773	0.5289±0.0833	0.0123±0.0091	0.7404±0.0632
T0816	0.5284±0.1366	0.0083±0.0071	0.7368±0.0979	gpW	0.4836±0.1253	0.0093±0.0072	0.7176±0.0905
bbl	0.4603±0.1422	0.0094±0.0067	0.6968±0.0896	hyp	0.4578±0.1154	0.0099±0.0066	0.7104±0.0873
T0769	0.4535±0.0871	0.0191±0.0101	0.7102±0.0738	T0771	0.4219±0.0711	0.0111±0.0073	0.6537±0.0491
We analyze the effect of RMSD on model training quality, indicating that data containing high levels of noise and significant vibration pose challenges to model training. As illustrated in the Figure 5, the model shows superior performance on smoother and less noisy trajectory data.
4.3 Generalization Ability
Our model has generative capabilities to predict missing or future trajectories of a protein. Here we obtain different samples in two aspects, interpolation and extrapolation of time.
4.3.1 Interpolation
Our model enable the prediction of protein shape at any given time due to its temporal continuity. This is very useful when we only know the structures of two moments but not what the intermediate process is. During training sample selection, we extract one frame at intervals of several frames to evaluate the model's interpolation capability in the temporal space. To demonstrate that our model
Figure 5: The correlation between RMSD and three metrics was evaluated across various protein trajectories, revealing a significant relationship between the metrics and RMSD, highlighting that smaller RMSD and system noise levels correspond to improved model performance and easier modeling of protein surface dynamics.
learns the protein's inherent dynamic variations and is not just performing trivial tasks, we compared it with linear interpolation, as shown in Figure 6, even though there were no previous works for direct comparison. The evaluation results of our model and linear interpolation are shown in Appendix Table 4, 5 respectively, and the performance of interpolation has reached a level comparable to that of reconstruction.
Figure 6: The three evaluation metrics for the interpolation task are shown in the plot, with the red line representing our method and the blue line representing linear interpolation.
4.3.2 Future Prediction
Our model can not only interpolate missing frames in the middle, but also predict future dynamics by extrapolating over time. To validate this, we use models trained on the MDAnalysisData and DRYAD_MD datasets to predict future frames. Table 2 shows the prediction results of the model for the next 10 frames and the next  $11\sim 20$  frames on some proteins. Due to space limitations of the main text, the full results are presented in the Appendix E.2.
Table 2: Evaluation on the future 10 frames and  ${11} \sim  {20}$  frames
Future 10 frames	Future 11~20
IoU↑	Chamfer_dist↓	NC↑	IoU↑	Chamfer_dist↓	NC↑
DIMS	0.8467±0.0423	0.0010±0.0005	0.8666±0.0504	0.6835±0.0551	0.0055±0.0020	0.6932±0.0474
I-FABP	0.8878±0.0142	0.0006±0.0001	0.9271±0.0159	0.8444±0.0153	0.0012±0.0002	0.8797±0.0164
FRODA	0.7660±0.0502	0.0012±0.0006	0.8267±0.0490	0.6043±0.0474	0.0044±0.0012	0.6783±0.0384
4.3.3 A Case Study on GPCRmd
Furthermore, we have explored the model's simultaneous simulation of multiple protein trajectories. Here we use the four protein trajectories data in GPCRmd dataset. The model trains different latent vectors to represent different proteins. In addition, in selecting training frames, we have employed another strategy, wherein the interval between the frames utilized for training is gradually increased with the sequence.
As shown in Figure 7, our model can learn the shapes of multiple proteins simultaneously, and it can recover the fine details of the protein surfaces well despite their ruggedness. An interesting finding is that the trajectories obtained from molecular dynamics simulations often have a lot of random jittering or noise, while the trajectories interpolated by our model can well fit the main functional
Figure 7: Comparison of ground truth and reconstruction for GPCRmd data at  $t = 2000$ . From (a) to (d) are 10792_trj_81, 10912_trj_95, 15711_trj_791 and 16105_trj_848 respectively, ground truth and reconstruction (top row) after rotating 90 degrees (bottom row) clockwise along the horizontal axis.
motion of the protein without noise. We put the detailed results and discussion in the Appendix E.3, video demonstration on the webpage.
5 Discussions
Molecular surface Representation The molecular surface representation is commonly adopted for tasks involving molecular interfaces[42], where non-covalent interactions (e.g., hydrophobic interactions) play a decisive role[43]. Non-Euclidean convolutional neural networks[44] and point cloud-based learning models[45] have been applied to encode the molecular surface for downstream applications, e.g., protein binding site prediction[46]. However, it has not been applied to molecular dynamic simulation, especially protein. In fact, this work explored the more essential application scenarios of molecular surface representation, that is, we believe that chemical molecules are more suitable to be regarded as electron clouds that can be represented by 3D shapes.
Deep learning molecular dynamics Deep learning is being used more in molecular dynamics simulations, which has shown promising results. There have been several approaches, such as Behler-Parrinello network[47], DTNN[48], and SchNet[49], that focus on predicting molecular properties and potential energy. However, these methods are limited to small molecules and may not work for proteins, which have many atoms. To address this issue, several coarse-grained models have been proposed, such as CGnet[17], DeePCG[50], ARCG[51], and a CG auto-encoder[52]. Recent works, flow-matching[53] and DFF[54], have used normalizing flows and diffusion models to model coarse-grained force fields for dynamic simulation of small proteins. Another approach, DiffMD[55], can predict simulation trajectories without energy or forces. However, these methods are currently limited to simulating small proteins due to computational constraints and are not yet practical for real-world applications, while the method in this paper provides a new idea for long-term modeling of large proteins.
Implicit neural representations SIREN[56] and NeRF[26] have led to the popularity of implicit neural representations (INR) for tasks like new view synthesis and geometry reconstruction. INR involves creating a continuous function that can map coordinates to high-frequency information. NeRF's highly expressive nature has allowed it to be effectively employed in numerous fields, such as image processing[57] (e.g., compression, denoising, super-resolution, inpainting), video processing[58], medical imaging[59], etc. Signed distance function (SDF) is a type of INR that characterizes an object's shape well. DeepSDF[60] uses INR to model SDF, and IGR[40] simplifies the modeling of SDF by using its gradient property. Drawing inspiration from the aforementioned studies, our work learns a unified implicit representation of proteins that will facilitate future large-scale modeling of protein dynamics like NeRF's contribution in vision.
6 Conclusions and Future work
We introduced a method for learning implicit neural representations of protein surface dynamics in continuous spatial and temporal space. Experimental results show that our method is very effective in reconstructing proteins with smooth dynamic changes, and has a certain ability to interpolate in time.
In addition, we found that our model tends to model major conformation changes of proteins rather than those noise-like random vibrations. The case study of GPCRmd shows that the model has the potential to learn the dynamics of a class of similar proteins, such as a protein family.
The main limitation of this method is that it is sensitive to noise in protein dynamic trajectories, as discussed in Section 4.2. In addition, the generalization of the model needs to be further improved. The first is the temporal generalization, especially future prediction. So far, there is no effective method that can better predict the future in various motion prediction tasks. We believe that modeling motion or velocity fields can help future predictions for a longer time.
We anticipate DSR as a powerful tool for building general-purpose protein surface models, and hope that our work helps shed some light on a more efficient and generalizable protein representation.
This work was supported by Mathematical Intelligence Application Laboratory, MIALAB, Institute for Mathematical Sciences, Renmin University of China, Beijing Academy of Artificial Intelligence and Public Computing Cloud of Renmin University of China.
[1] Yao Li, Tong Wang, Juanrong Zhang, Bin Shao, Haipeng Gong, Yusong Wang, Xinheng He, Siyuan Liu, and Tie-Yan Liu. Exploring the regulatory function of the n-terminal domain of sars-cov-2 spike protein through molecular dynamics simulation. Advanced theory and simulations, 4(10):2100152, 2021.
[2] Maxwell I Zimmerman, Justin R Porter, Michael D Ward, Sukrit Singh, Neha Vithani, Artur Meller, Upasana L Mallimadugula, Catherine E Kuhn, Jonathan H Borowsky, Rafal P Wiewiora, et al. Citizen scientists create an exascale computer to combat Covid-19. BioRxiv, pages 2020–06, 2020.
[3] Richard Car and Mark Parrinello. Unified approach for molecular dynamics and density-functional theory. Physical review letters, 55(22):2471, 1985.
[4] Kresse Georg. Vasp group, theoretical physics departments, vienna. Retrieved February, 21, 2011.
[5] Jussi Enkovaara, Nichols A Romero, Sameer Shende, and Jens J Mortensen. Gpaw-massively parallel electronic structure calculations with python-based software. Procedia Computer Science, 4:17-25, 2011.
[6] Wendy D Cornell, Piotr Cieplak, Christopher I Bayly, Ian R Gould, Kenneth M Merz, David M Ferguson, David C Spellmeyer, Thomas Fox, James W Caldwell, and Peter A Kollman. A second generation force field for the simulation of proteins, nucleic acids, and organic molecules. Journal of the American Chemical Society, 117(19):5179-5197, 1995.
[7] Lijiang Yang, Chun-hu Tan, Meng-Juei Hsieh, Junmei Wang, Yong Duan, Piotr Cieplak, James Caldwell, Peter A Kollman, and Ray Luo. New-generation amber united-atom force field. The journal of physical chemistry B, 110(26):13166–13176, 2006.
[8] Sandeep Patel and Charles L Brooks III. Charmm fluctuating charge force field for proteins: I parameterization and application to bulk organic liquid simulations. Journal of computational chemistry, 25(1):1-16, 2004.
[9] Sandeep Patel, Alexander D Mackerell Jr, and Charles L Brooks III. Charm fluctuating charge force field for proteins: Ii protein/solvent properties from molecular dynamics simulations using a nonadditive electrostatic model. Journal of computational chemistry, 25(12):1504-1514, 2004.
[10] Wilfred F van Gunsteren and Herman JC Berendsen. Groningen molecular simulation (gromos) library manual. Biomos, Groningen, 24(682704):13, 1987.
[60] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 165-174, 2019.
[61] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
[62] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.