File size: 2,388 Bytes
76fc7d0 b71ae05 79e7a5c b71ae05 79e7a5c b71ae05 79e7a5c b71ae05 79e7a5c 76fc7d0 f755533 b71ae05 c7d5f3c b71ae05 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
---
license: mit
datasets:
- sagawa/ZINC-canonicalized
metrics:
- accuracy
base_model: google/t5-v1_1-base
model-index:
- name: ZINC-deberta
results:
- task:
type: fill-mask
name: Masked Language Modeling
dataset:
name: sagawa/ZINC-canonicalized
type: sagawa/ZINC-canonicalized
metrics:
- type: accuracy
value: 0.9475839734077454
name: Accuracy
---
# ZINC-t5
This model is a fine-tuned version of [google/t5-v1_1-base](https://huggingface.co/microsoft/deberta-base) on the sagawa/ZINC-canonicalized dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1228
- Accuracy: 0.9476
## Model description
We trained t5 on SMILES from ZINC using the task of masked-language modeling (MLM). Compared to ZINC-t5, ZINC-t5-v2 uses a character-level tokenizer, and it was also trained on ZINC.
## Intended uses & limitations
This model can be used for the prediction of molecules' properties, reactions, or interactions with proteins by changing the way of finetuning.
As an example, We finetuned this model to predict products. The model is [here](https://huggingface.co/sagawa/ZINC-t5-productpredicition), and you can use the demo [here](https://huggingface.co/spaces/sagawa/predictproduct-t5).
Using its encoder, we trained a regression model to predict a reaction yield. You can use this demo [here](https://huggingface.co/spaces/sagawa/predictyield-t5).
## Training and evaluation data
We downloaded [ZINC data](https://drive.google.com/drive/folders/1lSPCqh31zxTVEhuiPde7W3rZG8kPgp-z) and canonicalized them using RDKit. Then, we dropped duplicates. The total number of data is 22992522, and they were randomly split into train:validation=10:1.
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-03
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10.0
### Training results
| Training Loss | Step | Accuracy | Validation Loss |
|:-------------:|:------:|:--------:|:---------------:|
| 0.2090 | 100000 | 0.9264 | 0.1860 |
| 0.1628 | 200000 | 0.9349 | 0.1613 |
| 0.1632 | 300000 | 0.9395 | 0.1467 |
| 0.1451 | 400000 | 0.9435 | 0.1345 |
| 0.1311 | 500000 | 0.9465 | 0.1261 | |