SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We conduct comprehensive evaluations on different datasets to demonstrate the effectiveness of the QuinNet model. First, we benchmark our model on the public MD17 [2] and revised MD17 [44] small molecular datasets, where the impact of five-body interactions is relatively weak. Despite this, we show that the QuinNet model shows comparable accuracy with the state-of-the-art models in these datasets. Moreover, QuinNet has also been benchmarked on the QM9 dataset [45, 46], the details of which can be found in the Supplementary Materials. Furthermore, we evaluate the performance of QuinNet on the larger molecular systems of MD22 [47] and Chignolin datasets [48]. These complex systems are known to exhibit a more pronounced influence of five-body interactions. Our evaluation demonstrates that QuinNet accurately models these interactions and achieves higher accuracy in energy and force prediction compared to other models.
Figure 4: The distribution of interatomic distances  $h(r)$  for (a) aspirin, (b) ethanol, (c) malonaldehyde, (d) naphthalene, (e) salicylic acid, (f) toluene, and (g) uracil in MD17 dataset. The insets display the ball-and-stick representations of these molecules.
5.1 MD17 Dataset and MD Simulation
The MD17 dataset [2] contains a diverse range of sizes, spanning from 150k to nearly 1M conformational geometries. Each trajectory was computed at a temperature of  $500\mathrm{K}$  with a resolution of 0.5 fs. The total energy and force labels for each dataset were computed using the PBE+vdW-TS electronic structure method [49, 50]. In Table 1, we present the mean absolute errors (MAEs) of QuinNet for 7 molecules in the MD17 dataset, along with comparisons to nine other models. QuinNet outperforms other tested models in nearly  $57.14\%$  of tasks, especially the performance on forces. Additionally, we perform MD simulations using trained QuinNets as force fields and plot the distribution of interatomic distances  $h(r)$  for these 7 molecules in Fig. 4. Further details regarding additional settings can be found in the Supplementary Materials.
Table 1: Comparison of the MAEs between several benchmarked models and QuinNet trained on MD17 dataset using 950 training samples and 50 validation samples (energies in kcal/mol and forces in kcal/(mol·Å)). The lowest values are highlighted in bold.
SchNet [11, 12]	DimeNet [13]	PaiNN [15]	SpookyNet [51]	ET [16]	GemNet [17]	NequIP (l=3) [7]	SO3KRATES [52]	ViSNet [20]	QuinNet
Aspirin	Energy	0.37	0.204	0.167	0.151	0.123	-	0.131	0.139	0.116	0.119
Force	1.35	0.499	0.338	0.258	0.253	0.217	0.184	0.236	0.155	0.145
Ethanol	Energy	0.08	0.064	0.064	0.052	0.052	-	0.051	0.052	0.051	0.050
Force	0.39	0.230	0.224	0.094	0.109	0.085	0.071	0.096	0.060	0.060
Malonaldehyde	Energy	0.13	0.104	0.091	0.079	0.077	-	0.076	0.077	0.075	0.078
Force	0.66	0.383	0.319	0.167	0.169	0.155	0.129	0.147	0.100	0.097
Naphthalene	Energy	0.16	0.122	0.116	0.116	0.085	-	0.113	0.115	0.085	0.101
Force	0.58	0.215	0.077	0.089	0.061	0.051	0.039	0.074	0.039	0.039
Salicylic acid	Energy	0.20	0.134	0.116	0.114	0.093	-	0.106	0.016	0.092	0.101
Force	0.85	0.374	0.195	0.180	0.129	0.125	0.090	0.145	0.084	0.080
Toluene	Energy	0.12	0.102	0.095	0.094	0.074	-	0.092	0.095	0.074	0.080
Force	0.57	0.216	0.094	0.087	0.067	0.060	0.046	0.073	0.039	0.039
Uracil	Energy	0.14	0.115	0.106	0.105	0.095	-	0.104	0.103	0.095	0.096
Force	0.56	0.301	0.139	0.119	0.095	0.097	0.076	0.111	0.062	0.062
5.2 Revised MD17 Dataset
Christensen and von Lilienfeld discovered that the energy of the original MD17 dataset was noisy [44]. Therefore, 100,000 structures of each molecule were selected from the original dataset to create the revised MD17 dataset, and the energies and forces were recalculated at the PBE/def2-SVP level of theory using a very tight SCF convergence and a very dense DFT integration grid [49, 53, 54]. As shown in Table 2, QuinNet matches the performance of popular test models on the revised MD17 dataset. Supplementary Materials provide additional information on the relevant settings.
5.3 MD22 Dataset
The MD22 dataset [47] is a recently introduced collection of MD trajectories, which encompasses four major classes of biomolecules and supramolecules, ranging from a small peptide consisting
Table 2: Comparison of the MAEs between several benchmarked models and QuinNet trained on revised MD17 dataset using 950 training samples and 50 validation samples (energies in kcal/mol and forces in kcal/(mol·Å)). The lowest values are shown in bold.
UNiT E [55]	GemNet (T/Q) [17]	NequIP (l=3) [7]	MACE [37]	Allegro [36]	BOTNet	ViSNet [20]	QuinNet
Aspirin	Energy	0.055	-	0.0530	0.0507	0.0530	0.0530	0.0445	0.0486
Force	0.175	0.2191	0.1891	0.1522	0.1684	0.1960	0.1520	0.1429
Azobenzene	Energy	0.025	-	0.0161	0.0277	0.0277	0.0161	0.0156	0.0394
Force	0.097	-	0.0669	0.0692	0.0600	0.0761	0.0585	0.0513
Benzene	Energy	0.002	-	0.0009	0.0092	0.0069	0.0007	0.0007	0.0096
Force	0.017	0.0115	0.0069	0.0069	0.0046	0.0069	0.0056	0.0047
Ethanol	Energy	0.014	-	0.0092	0.0032	0.0092	0.0092	0.0078	0.0096
Force	0.085	0.083	0.0646	0.0484	0.0484	0.0738	0.0522	0.0516
Malonaldehyde	Energy	0.025	-	0.0184	0.0185	0.0138	0.0185	0.0132	0.0168
Force	0.152	0.1522	0.01176	0.0946	0.0830	0.1338	0.0893	0.0875
Naphthalene	Energy	0.011	-	0.0046	0.1153	0.0046	0.0046	0.0057	0.0174
Force	0.060	0.0438	0.0300	0.0369	0.0208	0.0415	0.0291	0.0242
Paracetamol	Energy	0.044	-	0.0323	0.0300	0.0346	0.0300	0.0258	0.0362
Force	0.164	-	0.1361	0.1107	0.1130	0.1338	0.1029	0.0979
Salicylic acid	Energy	0.017	-	0.0161	0.0208	0.0208	0.0185	0.0161	0.033
Force	0.088	0.1222	0.0922	0.0715	0.0669	0.0992	0.0795	0.0771
Toluene	Energy	0.010	-	0.0069	0.0115	0.0092	0.0069	0.0059	0.0139
Force	0.058	0.0507	0.0369	0.0350	0.0415	0.0438	0.0264	0.0244
Uracil	Energy	0.013	-	0.0092	0.0115	0.0138	0.0092	0.0069	0.0149
Force	0.088	0.0876	0.0669	0.0484	0.0415	0.0738	0.0495	0.0487
Table 3: Comparison of the MAEs between several benchmarked models and the lowest values are marked in bold (energies in kcal/mol per atom and forces in kcal/(mol·Å)). Furthermore, the data splitting of the training set and validation set for training QuinNet on MD22 dataset is shown.
# Train/Val	sGDML [47]	ViSNet-LSRM [56]	ViSNet [20, 56]	MACE (3Å) [57]	MACE (6Å) [57]	MACE (5Å) [57]	QuinNet
Ac-Ala3-NHMe	5500/500	Energy	0.0093	0.0016	0.0019	0.0140	0.0080	0.0015	0.0020
Force	0.79	0.0942	0.0972	0.1753	0.3920	0.0876	0.0681
DHA (docosahexaenoic acid)	7500/500	Energy	0.023	0.0016	0.0027	0.0103	0.0092	0.0024	0.0021
Force	0.75	0.0598	0.0668	0.1430	0.5419	0.0646	0.0515
Stachyose	7500/500	Energy	0.046	0.0012	0.0015	0.0058	0.0082	0.0014	0.0026
Force	0.68	0.0767	0.0869	0.1568	0.6226	0.0876	0.0543
AT-AT	2500/500	Energy	0.012	0.0013	0.0028	0.0208	0.0036	0.0018	0.0024
Force	0.69	0.0781	0.1070	0.3067	0.3436	0.0992	0.0687
AT-AT-CG-CG	1500/500	Energy	0.012	0.0010	0.0017	0.0139	0.0038	0.0013	0.0032
Force	0.70	0.1064	0.1563	0.3759	0.4635	0.1153	0.1273
Buckyball catcher	550/50	Energy	0.0079	0.0029	0.0030	0.0110	0.0039	0.0033	0.0038
Force	0.68	0.1026	0.1335	0.3021	0.5120	0.0853	0.1091
Double-walled nanotube	750/50	Energy	0.0108	0.0049	0.0028	0.0048	0.0053	0.0045	0.0049
Force	0.52	0.3391	0.3959	0.4128	0.9132	0.2767	0.2473
of 42 atoms to a double-walled nanotube containing 370 atoms. The trajectories were sampled at temperatures between 400 and  $500\mathrm{K}$  at a resolution of 1 fs, and the corresponding potential energy and atomic forces were calculated using the PBE+MBD [49, 50] level of theory. Table 3 presents the MAEs of the QuinNet model for seven systems within the MD22 dataset, along with a comparison to six benchmark models. Our results show that QuinNet outperforms the vast majority of benchmark models in predicting atomic forces, and since the MD22 dataset claims that long-range interactions are crucial, this could be the reason why the ViSNet-LSRM model that incorporates long-range interactions and the models with chosen cutoff radius have more accurate energy predictions. Further details on the relevant settings refer to the Supplementary Materials.
5.4 Ablation study on Chignolin Dataset
The Chignolin, consisting of 166 atoms, serves as the simplest artificial protein for folding and unfolding studies. The dataset includes a diverse set of folding and unfolding states of Chignolin, comprising a total of 9,543 representative conformations [48]. As Wang et al.[25] noted that incorporating five-body interactions could enhance the performance of models on Chignolin, we performed an ablation study to evaluate the impact of various many-body interactions. As shown in Fig. 5, the explicit inclusion of five-body interactions up to part II led to a significant decrease in MAEs. It is worth noting that the energy errors exhibit some fluctuations when introducing five-body interactions up to part I. Considering that the five-body interactions up to part I to some extent describe the four-body improper terms, as demonstrated in the Supplementary Materials, we speculate that these two different representations of the same interaction may cause antagonism. To test this, we
Figure 5: The ablation study on Chignolin dataset.
removed the four-body (improper) terms and retained only the five-body interactions up to part I for training on the Chignolin dataset. The resulting energy MAE is  $1.2906 \pm 0.0267\mathrm{kcal / mol}$ , and the force MAE is  $0.2974 \pm 0.002417\mathrm{kcal / (mol\cdot A)}$ . These values are lower than the case where both four-body (improper) and five-body interactions up to part I coexist, thus confirming our hypothesis. These results demonstrate the importance of incorporating five-body interactions for accurately modeling the folding and unfolding conformations of Chignolin. Comparisons of the MAEs between different models and the detailed values of the ablation study refer to the Supplementary Materials.
In this work, we propose the QuinNet architecture, which efficiently incorporates many-body interactions up to whole five-body in graph neural networks for molecular dynamics simulations. Our experiments on several public datasets, including MD17, revised MD17, MD22, and Chignolin, demonstrate that QuinNet achieves high accuracy without significantly increasing computational complexity. Notably, our ablation study on Chignolin highlights the significance of five-body interactions in accurately modeling complex bio-molecular systems.
[1] Albert P Bartók, Mike C Payne, Risi Kondor, and Gábor Csányi. Gaussian approximation potentials: The accuracy of quantum mechanics, without the electrons. Phys. Rev. Lett., 104(13):136403, 2010.
[2] Stefan Chmiela, Alexandre Tkatchenko, Huziel E Sauceda, Igor Poltavsky, Kristof T Schütt, and Klaus-Robert Müller. Machine learning of accurate energy-conserving molecular force fields. Sci. Adv., 3(5):e1603015, 2017.
[3] Stefan Chmiela, Huziel E Sauceda, Klaus-Robert Müller, and Alexandre Tkatchenko. Towards exact molecular dynamics simulations with machine-learned force fields. Nat. Commun., 9(1):3887, 2018.
[4] Linfeng Zhang, Jiequn Han, Han Wang, Roberto Car, and Weinan E. Deep potential molecular dynamics: a scalable model with the accuracy of quantum mechanics. Phys. Rev. Lett., 120(14):143001, 2018.
[5] Jonathan P Mailoa, Mordechai Kornbluth, Simon Batzner, Georgy Samsonidze, Stephen T Lam, Jonathan Vandermause, Chris Ablitt, Nicola Molinari, and Boris Kozinsky. A fast neural network approach for direct covariant forces prediction in complex multi-element extended systems. Nat. Mach. Intell., 1(10):471-479, 2019.
[6] Yu Xie, Jonathan Vandermause, Lixin Sun, Andrea Cepellotti, and Boris Kozinsky. Bayesian force fields from active learning for simulation of inter-dimensional transformation of stanene. Npj Comput. Mater., 7(1):40, 2021.