---
library_name: peft
---
# ESM-2 Fine-Tuned with PEFT LoRA

This is an attempt to use Parameter Efficient Fine Tuning ([PEFT](https://huggingface.co/docs/peft/conceptual_guides/lora)) with 
Low Rank Adaptation ([LoRA](https://huggingface.co/docs/peft/task_guides/token-classification-lora))
for the ESM-2 model [esm2_t6_8M_UR50D](https://huggingface.co/facebook/esm2_t6_8M_UR50D). 
The Github can be found [here](https://github.com/Amelie-Schreiber/esm2_loras/tree/main). The model is intended to serve as an 
RNA binding site predictor for protein sequences. However, there seems to be something 
going wrong either during training, or at inference, as the model seems to only predict non-binding sites. 
The problem is likely in the training setup. Any feedback, comments, or discussion would be greatly appreciated. 

## Training procedure

### Framework versions


- PEFT 0.4.0