Graph Diffusion Transformers are In-Context Molecular Designers
Authors: name: Gang Liu; name: Jie Chen; name: Yihan Zhu; name: Michael Sun; name: Tengfei Luo; name: Nitesh V Chawla; name: Meng Jiang
Published: 2025-10-09 18:56:57

Abstract: In-context learning allows large models to adapt to new tasks from a few
demonstrations, but it has shown limited success in molecular design. Existing
databases such as ChEMBL contain molecular properties spanning millions of
biological assays, yet labeled data for each property remain scarce. To address
this limitation, we introduce demonstration-conditioned diffusion models
(DemoDiff), which define task contexts using a small set of molecule-score
examples instead of text descriptions. These demonstrations guide a denoising
Transformer to generate molecules aligned with target properties. For scalable
pretraining, we develop a new molecular tokenizer with Node Pair Encoding that
represents molecules at the motif level, requiring 5.5times fewer nodes. We
curate a dataset containing millions of context tasks from multiple sources
covering both drugs and materials, and pretrain a 0.7-billion-parameter model
on it. Across 33 design tasks in six categories, DemoDiff matches or surpasses
language models 100-1000times larger and achieves an average rank of 3.63
compared to 5.25-10.20 for domain-specific approaches. These results position
DemoDiff as a molecular foundation model for in-context molecular design. Our
code is available at https://github.com/liugangcode/DemoDiff.