File size: 1,066 Bytes
b83af74 88971af 5ad55b9 9d54d52 e684670 9d68f13 5ad55b9 2d25fd2 88971af |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
---
license: mit
---
# ESM-2 Full Finetune for Binding Sites
This model is a full finetune of ESM-2, to illustrate how full finetuning overfits and generalizes quite poorly compared to
LoRA and QLoRA finetuning. This model was finetuned on the 600K dataset. We also note that on the 24GB A10 GPU, the batch size
has to be significantly smaller than when using LoRA or QLoRA. To finetune a similar model, use
[this script](https://huggingface.co/AmelieSchreiber/esm2_t6_8M_binding_sites_finetune/blob/main/finetune.py).
## Overfitting
```python
Train metrics:
{'eval_loss': 0.13651661574840546,
'eval_accuracy': 0.9656322509450104,
'eval_precision': 0.38616650354104665,
'eval_recall': 0.9618091516702236,
'eval_f1': 0.55107594226701,
'eval_auc': 0.9637635647574605,
'eval_mcc': 0.5977943918337999}
Test metrics:
{'eval_loss': 0.2910114824771881,
'eval_accuracy': 0.923270649115702,
'eval_precision': 0.14887069127765168,
'eval_recall': 0.533511928419524,
'eval_f1': 0.23278520670392827,
'eval_auc': 0.7327381144575454,
'eval_mcc': 0.25329082069818704}
``` |