Mol ID
A transformer encoder model pretrained on 50M ZINC SMILES string using flash attention 2
With modern hardware and software stacks, it only took me 6~7 hours to pretrain this 17M model on 50M molecules for 5 epochs
Hardware:
- 4 cores cpu
- 1 RTX 3090 gpu
Software:
- flash attention 2
- lightning for mixed precision (bf16-mixed)
- wandb for logging
- huggingface
- tokenizers
- datasets
github repo: link
- Downloads last month
- 2
Unable to determine this model’s pipeline type. Check the
docs
.