File size: 1,030 Bytes
c27a4d4 7686af8 efa515d 8cbba15 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
---
datasets:
- axiong/pmc_oa_beta
language:
- en
library_name: open_clip
pipeline_tag: text-to-image
tags:
- biology
- chemistry
- medical
---
### Model Description
The model is fine-tuned from openai's ViT-L-14 using PMC_OA_beta and roco's data sets, using the tool open_clip（https://github.com/mlfoundations/open_clip）.
### Training
```python
python -m training.main \
--save-frequency 2 \
--zeroshot-frequency 1 \
--report-to tensorboard \
--train-data="/home/data1/ryanyip/huggingface-models/pmc_oa_beta/train.csv" \
--val-data="/home/data1/ryanyip/huggingface-models/pmc_oa_beta/sample_valid.csv" \
--csv-separator "," \
--csv-img-key image \
--csv-caption-key caption \
--warmup 10000 \
--batch-size=128 \
--lr=1e-5 \
--wd=0.2 \
--epochs=30 \
--workers=8 \
--model "ViT-L-14" \
--name "pmc_vit_l_14" \
--pretrained "ViT-L-14_state_dict.pt" \
--save-most-recent
````
*ViT-L-14_state_dict.pt is the pretrained weight from openai/ViT-L-14*
|