arunraja007/quadmetformer

QuADMET-Former

Pre-training on Quantum Mechanical Properties Improves ADMET Prediction

Overview

QuADMET-Former demonstrates that pre-training Equivariant Graph Neural Networks (EGNNs) on quantum mechanical properties significantly improves the prediction of absorption, distribution, metabolism, excretion, and toxicological (ADMET) properties of small molecules.

Key contributions:

• Pre-training TorchMD-NET on quantum mechanical properties (dipole moments, HOMO-LUMO gap, partial charges)
• Novel E3FP pre-training strategy using 3D molecular fingerprints
• Comprehensive evaluation on 23 ADMET benchmarks with challenging spectral splits
• State-of-the-art performance: best on 16/23 datasets

Installation

Prerequisites

• Python 3.10+
• CUDA 12.1+ (for GPU support)

Setup

# Clone the repository
git clone https://github.com/your-username/quadmetformer.git
cd quadmetformer

# Create conda environment
conda env create -f environment.yml
conda activate quadmetformer

# Or install with pip
pip install -e .

Pre-trained Models

Pre-trained checkpoints are available on HuggingFace: https://huggingface.co/arunraja007/quadmetformer/tree/main

Usage

Pre-training

Pre-train on QMugs dataset with quantum mechanical targets:

cd pretraining
python pretrain.py --config configs/pretraining/qmugs.json

Fine-tuning on ADMET Tasks

Fine-tune a pre-trained model on downstream ADMET prediction:

cd finetuning
python downstream_train.py \
    --dataset Lipophilicity_AstraZeneca \
    --pretrained_ckpt path/to/checkpoint.ckpt \
    --split_type spectral

Reproducing Results

To reproduce the main results from the paper:

# 1. Pre-train on QMugs (or download pre-trained checkpoints)
cd pretraining
python pretrain.py --config configs/pretraining/qmugs.json

# 2. Run fine-tuning on all benchmarks
cd finetuning
python downstream_train.py --config configs/qmugs_single_conf.json --dataset all

# 3. Run baseline comparisons
python baseline_downstream_train.py --dataset all

Citation

If you find this work useful, please cite the following papers:

@article{arunraja2026quadmetformer,
author = {Arun Raja  and Hongtao Zhao  and Christian Tyrchan  and Eva Nittinger  and Michael M. Bronstein  and Charlotte M. Deane  and Garrett M. Morris },
title = {QuADMET-Former: Pre-training on Quantum Mechanical Properties Improves ADMET Prediction},
journal = {ChemRxiv},
volume = {2026},
number = {0427},
year = {2026},
doi = {10.26434/chemrxiv.15002429/v1}}

@inproceedings{
raja2024on,
title={On the Effectiveness of Quantum Chemistry Pre-training for Pharmacological Property Prediction},
author={Arun Raja and Hongtao Zhao and Christian Tyrchan and Eva Nittinger and Michael M. Bronstein and Charlotte Deane and Garrett M Morris},
booktitle={ICML 2024 AI for Science Workshop},
year={2024},
url={https://openreview.net/forum?id=ffSLXl666Q}
}

Acknowledgements

• TorchMD-NET for the equivariant transformer architecture
• Therapeutics Data Commons
• QMugs for the quantum chemistry dataset
• Biogen ADME
• OpenADMET