SlowGuess/NeurIPS_2023_OCR huggingface dataset:
QM9 [100] is a dataset consisting of 134K molecules, each with up to 9 heavy atoms. It includes 12 tasks that are related to the quantum properties. For example, U0 and U298 are the internal energies at temperatures of 0K and 298.15K, respectively. On the QM9 dataset, we can easily get the 1D descriptors (Fingerprints/FPs [106], SMILES [126], SELFIES [70]), 2D topology, and 3D conformation. This enables us to build models on each of them respectively: (1) We benchmark 7 models on 1D descriptors, including multi-layer perception (MLP), random forest (RF), XGBoost (SGB), convolution neural networks (CNN), and BERT [18]. (2) We benchmark 10 2D GNN models on the molecular topology, including GCN [23, 66], ENN-S2S [38], GraphSAGE [43], GAT [119], GIN [130], D-MPNN [132], PNA [13], Graphormer [135], AWARE [17], GraphGPS [101]. (3) We benchmark 11 3D geometric models on the molecular conformation, including SchNet [109], DimeNet++, [68], SE(3)-Trans [35], EGNN [108], PaiNN [110], GemNet-T [67], SphereNet [89], SEGNN [4], Allegro [95], NequIP [3], Equformer [73]. The evaluation metric is the mean absolute error.
The results of these 28 models are in Table 1, and two important insights are observed: (1) There is no one universally best geometric model, yet DimeNet++, PaiNN, GemNet, and Equiformer perform well in most tasks. However, PaiNN takes less than 20 GPU hours, and the other three models take up to 5 GPU days per task. (2) The geometric conformation is important for quantum property prediction. The performance of 3D models is better than all the 1D and 2D models by orders of magnitudes.
Table 2: Results on 6 energy (kcal mol) and force (kcal mol-Å) prediction tasks in MD17 and rMD17 (w/o normalization), and the metric is the mean absolute error (MAE). The data split and complete results are in Appendices B and I.
Model	Energy/Force	MD17	rMD17
Aspirin ↓	Ethanol ↓	Malonaldehyde ↓	Naphthalene ↓	Salicylic ↓	Toluene ↓	Aspirin ↓	Ethanol ↓	Malonaldehyde ↓	Naphthalene ↓	Salicylic ↓	Toluene ↓
SchNet	Energy	0.475	0.109	0.300	0.167	0.212	0.149	0.534	1.757	0.260	0.124	2.618	0.119
Force	1.203	0.386	0.794	0.587	0.826	0.568	1.243	0.449	0.862	0.587	0.878	0.574
DimeNet++	Energy	4.168	1.238	1.385	1.846	2.445	1.484	2.438	1.456	2.317	1.648	1.555	1.210
Force	7.212	0.753	1.842	8.515	1.752	1.037	2.009	1.213	7.029	0.629	0.934	0.921
EGNN	Energy	17.892	0.436	0.896	12.177	6.964	4.051	17.35	0.402	0.534	12.164	7.794	15.021
Force	3.042	0.924	1.566	1.136	1.177	1.202	3.825	0.989	1.334	1.183	1.571	1.165
PaiNN	Energy	27.626	0.063	0.102	0.622	0.371	0.165	30.156	1.17	0.070	5.297	5.219	0.045
Force	0.572	0.230	0.338	0.132	0.288	0.141	0.573	0.316	0.377	0.161	0.321	0.231
GemNet-T	Energy	0.684	4.598	4.966	0.482	0.128	0.098	5.389	1.615	9.496	0.031	21.411	959.745
Force	0.558	0.219	0.433	0.212	0.326	0.174	0.555	0.233	0.337	0.154	0.371	0.400
SphereNet	Energy	0.244	1.603	1.559	0.167	0.188	0.113	0.304	0.072	0.138	0.093	0.771	20.479
Force	0.546	0.168	0.667	0.315	0.479	0.194	0.622	0.217	0.500	0.279	2.088	0.254
SEGNN	Energy	17.774	0.151	0.247	0.655	2.173	0.624	15.721	0.13	0.182	1.11	1.494	0.814
Force	9.003	0.893	1.249	0.895	2.220	1.138	8.549	0.846	1.185	0.926	2.056	1.241
NequiP	Energy	8.333	0.971	2.293	1.032	2.952	1.303	9.618	0.936	2.313	2.089	3.302	1.306
Force	23.769	5.832	12.099	5.247	14.048	6.8	22.904	6.027	12.372	5.529	15.693	7.094
Allegro	Energy	1.138	0.258	1.33	0.824	1.114	0.441	1.366	1.002	0.417	1.756	1.035	0.437
Force	3.405	1.412	4.191	3.743	4.934	1.968	3.186	2.799	2.125	3.815	4.781	2.048
Equiformer	Energy	0.308	0.096	0.183	0.097	0.189	0.209	0.375	0.064	0.085	0.069	0.143	0.104
Force	0.286	0.142	0.230	0.068	0.200	0.080	0.305	0.162	0.240	0.070	0.218	0.077
Figure 5: Ablation study on the effect of data normalization. Here are visualizations on performance differences on 6 tasks and 2 datasets, with MAE.force pred w/o normalization) - MAE force pred w/ normalization).
4.2 Small Molecules: MD17 and rMD17
MD17 [8] is a dataset of molecular dynamics simulation. It has 8 tasks corresponding to eight organic molecules, and each task includes the molecule positions along the PES (see Fig. 3(b)). The goal is to predict each atom's energy and interatomic forces for each molecule's position. We follow the literature [68, 89, 109, 110] of using 8 subtasks, 1K for training and 1K for validation, while the test set (from 48K to 991K) is much larger. However, the MD17 dataset contains non-negligible numerical noises [9], and it is corrected by the revised MD17 (rMD17) dataset [10]. 100K structures were randomly chosen for each task/molecule in MD17, and the single-point force and energy calculations were performed for each structure using the PBE/def2-SVP level of theory. The calculations were conducted with tight SCF convergence and a dense DFT integration grid, significantly minimizing the computational noises.
The results on MD17 and rMD17 are in Table 2. We select 12 tasks for illustration, and more comprehensive results can be found in Appendix I. We can observe that, in general, PaiNN, GemNet and Equiformer perform well on MD17 and rMD17 tasks. We also report ablation study on data normalization. NequIP [3] and Allegro [95] introduce a normalization trick: multiplying the predicted energy with the mean of ground-truth force (reproduced results in Appendix J). We plot the performance gap, MAE(w/o normalization) - MAE(w/ normalization), in Fig. 5, and observe most of the gaps are positive, meaning that adding data normalization can lead to generally better performance.
4.3 Small Molecules: COLL
The COLL dataset [36] comprises energy and force data for 140K random snapshots obtained from molecular dynamics simulations of molecular collisions. These simulations were conducted using the semiempirical GFN2-xTB method. To obtain the data, DFT calculations were performed utilizing the revPBE functional and def2-TZVP basis set, which also incorporated D3 dispersion corrections. The task is to predict the energy and force for each atom in the molecule, and we consider 10 advanced geometric models for benchmarking. The results are in Table 3, and GemNet, SphereNet, and Equiformer reach more optimal performance.
Table 3: Results on energy and force prediction in COLL. 120k for training, 10k for val, 9.48k for test. The metric is the mean absolute error (MAE).
Model	Energy (eV) ↓	Force (eV/Å) ↓
SchNet	0.178	0.130
DimeNet++	0.036	0.049
EGNN	1.808	0.234
PaiNN	0.030	0.052
GemNet-T	0.017	0.028
SphereNet	0.032	0.047
SEGNN	7.085	0.642
NequIP	0.120	0.113
Allegro	0.161	0.130
Equiformer	0.036	0.030
Table 4: Results on 2 binding affinity prediction tasks. We select three evaluation metrics for LBA: the root mean squared error (RMSD), the Pearson correlation  $(R_{p})$  and the Spearman correlation  $(R_{S})$ . LEP is a binary classification task, and we use the area under the curve for receiver operating characteristics (ROC) and precision-recall (PR) for evaluation. We run cross-validation with 5 seeds, and the mean and std are reported.
Model	LBA	LEP
RMSD ↓	RP↑	RC↑	ROC ↑	PR ↑
SchNet	1.521 ± 0.02	0.474 ± 0.01	0.452 ± 0.01	0.450 ± 0.03	0.379 ± 0.03
DimeNet++	1.672 ± 0.09	0.550 ± 0.01	0.556 ± 0.01	0.590 ± 0.06	0.496 ± 0.05
EGNN	1.494 ± 0.04	0.503 ± 0.04	0.483 ± 0.05	0.657 ± 0.05	0.559 ± 0.05
PaiNN	1.434 ± 0.02	0.583 ± 0.02	0.580 ± 0.02	0.585 ± 0.02	0.432 ± 0.03
GemNet-T	-	-	-	0.659 ± 0.05	0.506 ± 0.05
SphereNet	1.581 ± 0.02	0.538 ± 0.01	0.529 ± 0.01	0.523 ± 0.04	0.432 ± 0.05
SEGNN	1.416 ± 0.03	0.566 ± 0.02	0.550 ± 0.02	0.574 ± 0.03	0.485 ± 0.03
NequIP	1.606 ± 0.02	0.537 ± 0.01	0.520 ± 0.01	0.538 ± 0.12	0.481 ± 0.07
Allegro	1.567 ± 0.02	0.547 ± 0.00	0.534 ± 0.00	0.627 ± 0.04	0.525 ± 0.03
Equiformer	1.392 ± 0.03	0.598 ± 0.02	0.578 ± 0.02	0.618 ± 0.06	0.510 ± 0.05
Table 5: Results on 10 protein tasks from six datasets: ECSingle, ECMultiple, Fold (Fold, Sup., Fam.), GO (MF, BP, CC), MSP, and PSR. The evaluation metrics are Accuracy (ACC, %),  $F_{max}$  (definition in Appendix B), ACC,  $F_{max}$ , receiver operating characteristics (ROC), and Spearman's  $\rho$ , respectively.
ECSingle	ECMultiple	Fold	GO	MSP	PSR
Fold	Sup.	Fam.	MF	BP	CC
ACC ↑	Fmax↑	ACC ↑	ACC ↑	ACC ↑	Fmax↑	Fmax↑	Fmax↑	ROC ↑	Global ρ ↑	Mean ρ ↑
IEConv	-	-	45.0	69.7	98.9	-	-	-	-	-	-
GVP-GNN	65.5	0.712	34.8	52.7	95.0	0.476	0.312	0.389	0.574	0.744	0.302
GearNet	78.8	0.799	29.1	43.1	95.9	0.477	0.283	0.373	-	-	-
ProNet	86.4	0.823	52.7	70.3	99.3	0.559	0.367	0.414	0.634	\u818	0.462
CDConv	\u86.9	\u862	\u60.0	\u79.9	\u99.5	\u649	\u435	\u450	\u717	0.817	\u500
4.4 Small Molecules & Proteins Binding: LBA & LEP
The binding affinity measures the strength of the binding interaction between a small molecule (ligand) to the target protein. In Geom3D, we consider modeling both the ligands and proteins with their 3D structures. During binding, a cavity in a protein can potentially possess suitable properties for binding a small molecule, and it is called a pocket [113]. Due to the large volume of protein, Geom3D follows existing works [118] by only taking the binding pocket instead of the whole protein structure. Specifically, Geom3D models up to 600 atoms for each ligand and protein pair. For the benchmarking, we consider two binding affinity tasks. (1) The first task is ligand binding affinity (LBA) [123]. It is gathered from [124], and the task is to predict the binding affinity strength between a ligand and a protein pocket. (2) The second task is ligand efficacy prediction (LEP) [34]. The input is a ligand and both the active and inactive conformers of a protein, and the goal is to classify whether or not the ligand can activate the protein's function. The results on two binding tasks are in Table 4, and we can observe that PaiNN, SEGNN, and Equiformer are generally outstanding on the two tasks.
4.5 Proteins: ECSingle, ECMultiple, Fold, GO, MSP, and PSR
ECSingle is a classification task [45] that classifies 37K proteins into 384 four-level Enzyme Commission (EC) types. This task aims to recognize the fundamental role of proteins as bio-catalysts or enzymes, which are essential in facilitating biological reactions. The EC numbering system [63] serves as a comprehensive numerical classification scheme, systematically organizing the varied functionalities of enzymes and providing a structured approach to understanding their biological roles.
ECMultiple is a multi-label classification task proposed by Gligorijevic et al. [39], where 19K proteins are associated with 538 distinct EC categories, including both three-level and four-level types and a single protein can be concurrently labeled with several three-level or four-level EC numbers.
Fold is a task classifying 16K proteins into 1,195 fold patterns [47, 74]. It is an important biological task in predicting the 3D structures from 1D amino acid sequences. We further consider three testsets (Fold, Superfamily, and Family) based on the sequence and structure similarity [94].
GO (Gene Ontology) is a dataset [39] with 36K proteins for GO term classification, where the GO term provides a consistent description of gene product attributes across species and databases [12]. Concretely, each protein contains up to three types of GO terms, corresponding to three types of classification tasks: (1) Molecular Function (MF) has 489 classes; (2) Biological Process (BP) has 1,943
Table 6: Results on the 8 tasks from MatBench and 1 task from QMOF (with optimal DA). The data split and task unit are in Appendix B, and the metric is the mean absolute error (MAE).
Model	MatBench	QMOF
Per. Eform ↓ 18,928	Dielectric ↓ 4,764	log10G ↓ 10,987	log10K ↓ 10,987	Exfo ↓ 636	Phonons ↓ 1,265	Band Gap ↓ 106,113	Eform ↓ 132,752	Band Gap ↓ 20,425
SchNet	0.040	0.334	0.081	0.060	65.201	42.586	0.327	0.026	0.236
DimeNet++	0.037	0.357	0.081	0.058	68.685	38.339	0.208	0.025	0.234
EGNN	0.038	0.331	0.087	0.064	78.015	74.846	0.211	0.026	0.256
PaiNN	0.038	0.317	0.080	0.053	67.752	44.602	0.022	0.190	0.207
GemNet-T	0.042	0.325	0.088	0.061	68.425	48.986	0.186	0.026	0.207
SphereNet	0.043	0.388	0.087	0.061	72.987	36.300	0.217	0.029	0.251
SEGNN	0.046	0.360	0.087	0.059	65.052	43.638	0.330	0.047	0.330
Equiformer	0.046	0.280	0.087	0.057	62.977	37.381	0.202	0.027	0.234
classes; and (3) Cellular Component (CC) has 320 classes. Notice that each protein can be associated with multiple GO terms in each GO term type, thus all three tasks are multi-label classifications.
MSP & PSR are two protein tasks from a collection of benchmark datasets for machine learning in structural biology [118]. MSP (Mutation Stability Prediction) aims to predict whether the stability of a protein increases after mutation. The dataset is a mutation dataset containing 4K proteins. It is constructed by incorporating single-point mutations given in the SKEMPI database [56]. PSR (Protein Structure Ranking) is a regression task based on the Critical Assessment of Structure Prediction (CASP) [71]. In CASP, a protein structure is predicted and a quality score, the global distance test (GDT_TS), is calculated between the predicted structure and experimentally determined structure. This task aims to predict this score for 44K proteins.
The results of 5 models are in Table 5. CDConv [29] outperforms other models by a large margin on almost all 10 tasks, while ProNet [122] performs second well in general, and reaches the best result on the PSR task with global  $\rho$  metric. Notice that certain entries in the table are temporarily left blank due to memory constraints encountered. More detailed dataset specifications are in Appendix B.
4.6 Crystalline Materials: MatBench and QMOF
MatBench [21] is explicitly created to evaluate the performance of machine learning models in predicting properties of inorganic bulk materials covering mechanical, electronic, and thermodynamic material properties [21]. Here we consider 8 regression tasks with crystal structures, including predicting the formation energy (Perovskites,  $E_{\mathrm{form}}$ ), exfoliation energies ( $E_{\mathrm{exfo}}$ ), band gap, shear and bulk modulus ( $log_{10}G$  and  $log_{10}K$ ), etc. Please check Appendix B for more details.
Quantum MOF (QMOF) [107] is a dataset of over 20K metal-organic frameworks (MOFs) and coordination polymers derived from DFT. The task is to predict the band gap, the energy gap between the valence band and the conduction band. The results of 8 geometric models on 8 MatBench tasks and 1 QMOF task are in Table 6, and we can observe that the performance of all the models is very close, while DimeNet++, PaiNN, GemNet-T, and Equiformer are slightly better.
We also conduct ablation study on periodic data augmentation on crystal materials. We note that there are two data augmentation (DA) methods: gathered and expanded. Gathered DA means that we shift the original unit cell along three dimensions, and the translated unit cells will have the same node indices as the original unit cell, i.e., a multi-edge graph. How
Figure 6: Ablation study on the performance gap with data augmentation (DA): MAE(expanded DA) - MAE(gathered DA).
ever, expanded DA will assume the translated unit cells have different node indices from the original unit cell. (A visual demonstration is in Appendix A). We conduct an ablation study on the effect of these two DAs, and we plot MAE(expanded DA) - MAE(gathered DA) on six tasks in Fig. 6. It reveals that for most of the models (except EGNN), using gathered DA can lead to consistently better performance, and thus it is preferred. For more qualitative analysis, please check Appendix J.
Table 7: QM9 downstream results after pretraining, and the backbone model is SchNet. We take 110K for training, 10K for validation, and 11K for testing. The evaluation metric is the mean absolute error (MAE).
Pretraining	α↓	∇E↓	EHOMO↓	ELUMO↓	μ↓	Cv↓	G↓	H↓	R2↓	U↓	U0↓	ZPVE↓
-(random init)	0.060	44.13	27.64	22.55	0.028	0.031	14.19	14.05	0.133	13.93	13.27	1.749
Supervised	0.062	40.31	25.57	21.69	0.030	0.030	14.36	14.68	0.308	15.21	16.13	1.638
Type Prediction	0.073	45.38	28.76	24.83	0.036	0.032	16.66	16.28	0.275	15.56	14.66	2.094
Distance Prediction	0.065	45.87	27.61	23.34	0.031	0.033	14.83	15.81	0.248	15.07	15.01	1.837
Angle Prediction	0.066	48.45	29.02	24.40	0.034	0.031	14.13	13.77	0.214	13.50	13.47	1.861
3D InfoGraph	0.062	45.96	29.29	24.60	0.028	0.030	13.93	13.97	0.133	13.55	13.47	1.644
GeoSSL-RR	0.060	43.71	27.71	22.84	0.028	0.031	14.54	13.70	0.122	13.81	13.75	1.694
GeoSSL-InfoNCE	0.061	44.38	27.67	22.85	0.027	0.030	13.38	13.36	0.116	13.05	13.00	1.643
GeoSSL-EBM-NCE	0.057	43.75	27.05	22.75	0.028	0.030	12.87	12.65	0.123	13.44	12.64	1.652
3D InfoMax	0.057	42.09	25.90	21.60	0.028	0.030	13.73	13.62	0.141	13.81	13.30	1.670
GraphMVP	0.056	41.99	25.75	21.58	0.027	0.029	13.43	13.31	0.136	13.03	13.07	1.609
GeoSSL-DDM-1L	0.058	42.64	26.32	21.87	0.028	0.030	12.61	12.81	0.173	12.45	12.12	1.696
GeoSSL-DDM	0.056	42.29	25.61	21.88	0.027	0.029	11.54	11.14	0.168	11.06	10.96	1.660
MoleculeSDE (VE)	0.056	41.84	25.79	21.63	0.027	0.029	11.47	10.71	0.233	11.04	10.95	1.474
MoleculeSDE (VP)	0.054	41.77	25.74	21.41	0.026	0.028	13.07	12.05	0.151	12.54	12.04	1.587
4.7 Geometric Pretraining on Small Molecules
We run 14 pretraining algorithms, including one supervised pretraining: the pretraining dataset (e.g., PCQM4Mv2 [51]) possess the energy or energy gap label for each conformation, which can be naturally adopted for pretraining. The benchmark results of using SchNet as the backbone model pretrained on PCQM4Mv2 and fine-tuning on QM9 tasks are in Table 7. We observe that MoleculeSDE and GeoSSL-DDM utilizing the geometric denoising diffusion models outperform other pretraining methods in most cases. On the other hand, supervised pretraining (pretrained on energy gap  $\nabla \mathcal{E}$ ) reaches outstanding performance on  $\nabla \mathcal{E}$  downstream task, yet the generalization to other tasks is modest. Please check Appendix I for more pretraining results with different backbone models.
Geom3D provides a unified view on the SE(3)-equivariant models, together with the implementations. Indeed these can serve as the building blocks to various tasks, such as geometric pretraining (as displayed in Sec. 4.7) and the conformation generation (ClofNet [20], MoleculeSDE [79]), paving the way for building more foundational models and solving more challenging tasks.
Limitations on models and tasks. Geom3D includes 10 topological models, 16 geometric models, 14 geometric pretraining methods, and 52 diverse tasks. We would also like to acknowledge there exist many more tasks (e.g., Atom3D [118], Molecule3D [131], OC20 [7]) and more geometric models (e.g., OrbNet [99], MACE [2], Uni-Mol [144], and LieTransformer [52]). The continual updating may necessitate the collective efforts of our entire community, exemplifying our collaborative endeavors.
Foundation model as future exploration. Recently, there have been certain explorations on building the foundation models for molecule discovery, especially by incorporating textual data on the molecule's functionalities [25, 26, 83, 87, 88, 115, 139, 143]. However, existing works mainly focus on the 1D sequence or 2D topology, while the 3D geometric structure of molecules is rarely considered. We believe that Geom3D can offer essential support for future explorations along this direction.
Reproducibility and Tutorials
The codes of Geom3D have been released on this GitHub repository. Both the raw and preprocessed datasets have been released on this HuggingFace link. The checkpoints of all models have been released on this HuggingFace link. We further added four tutorials on using Geom3D on customized data, energy prediction, force prediction, and geometric pretraining. These tutorials can sufficiently demonstrate how users can inject new methods into Geom3D platform, showcasing its potential as a fundamental building block for tackling a wide range of machine learning tasks.
The authors would like to thank Zichao Rong, Chengpeng Wang, Jiarui Lu, Farzaneh Heidari, Zuobai Zhang, Limei Wang, and Hanchen Wang for their helpful discussions. This project is supported by the Natural Sciences and Engineering Research Council (NSERC) Discovery Grant, the Canada CIFAR AI Chair Program, collaboration grants between Microsoft Research and Mila, Samsung Electronics Co., Ltd., Amazon Faculty Research Award, Tencent AI Lab Rhino-Bird Gift Fund, and a National Research Council of Canada (NRC) Collaborative R&D Project. This project was also partially funded by IVADO Fundamental Research Project grant PRF-2019-3583139727.
[133] Huaxiu Yao, Ying Wei, Long-Kai Huang, Ding Xue, Junzhou Huang, and Zhenhui Jessie Li. Functionally regionalized knowledge transfer for low-resource drug discovery. Advances in Neural Information Processing Systems, 34:8256-8268, 2021. 3
[134] Huaxiu Yao, Xinyu Yang, Xinyi Pan, Shengchao Liu, Pang Wei Koh, and Chelsea Finn. Leveraging domain relations for domain generalization. arXiv preprint arXiv:2302.02609, 2023. 3
[135] Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. Do transformers really perform badly for graph representation? Advances in Neural Information Processing Systems, 34:28877-28888, 2021. 1, 6, 21
[136] Sheheryar Zaidi, Michael Schaarschmidt, James Martens, Hyunjik Kim, Yee Whye Teh, Alvaro Sanchez-Gonzalez, Peter Battaglia, Razvan Pascanu, and Jonathan Godwin. Pre-training via denoising for molecular property prediction. arXiv preprint arXiv:2206.00133, 2022. 3, 5, 34, 39, 52
[137] Chengxi Zang and Fei Wang. Moflow: an invertible flow model for generating molecular graphs. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 617-626, 2020. 1
[138] Anthony Zee. Group theory in a nutshell for physicists, volume 17. Princeton University Press, 2016. 4, 28
[139] Zheni Zeng, Yuan Yao, Zhiyuan Liu, and Maosong Sun. A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals. Nature communications, 13(1):862, 2022. 10
[140] Linfeng Zhang, Jiequn Han, Han Wang, Roberto Car, and EJPRL Weinan. Deep potential molecular dynamics: a scalable model with the accuracy of quantum mechanics. Physical review letters, 120(14):143001, 2018. 5
[141] Ningyu Zhang, Zhen Bi, Xiaozhuan Liang, Siyuan Cheng, Haosen Hong, Shumin Deng, Jiazhang Lian, Qiang Zhang, and Huajun Chen. Ontoprotein: Protein pretraining with gene ontology embedding. arXiv preprint arXiv:2201.11147, 2022. 1
[142] Zuobai Zhang, Minghao Xu, Arian Jamasb, Vijil Chenthamarakshan, Aurelie Lozano, Payel Das, and Jian Tang. Protein representation learning by geometric structure pretraining. 2022. 34, 35, 51
[143] Haiteng Zhao, Shengchao Liu, Chang Ma, Hannan Xu, Jie Fu, Zhi-Hong Deng, Lingpeng Kong, and Qi Liu. Gimlet: A unified graph-text model for instruction-based molecule zero-shot learning. bioRxiv, pages 2023–05, 2023. 10
[144] Gengmo Zhou, Zhifeng Gao, Qiankun Ding, Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, and Guolin Ke. Uni-mol: A universal 3d molecular representation learning framework. 2023. 10
[145] Zhaocheng Zhu, Chence Shi, Zuobai Zhang, Shengchao Liu, Minghao Xu, Xinyu Yuan, Yangtian Zhang, Junkun Chen, Huiyu Cai, Jiarui Lu, et al. Torchdrug: A powerful and flexible machine learning platform for drug discovery. arXiv preprint arXiv:2202.08320, 2022. 2, 51