AmelieSchreiber commited on
Commit
7e9a955
1 Parent(s): f1199f7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -0
README.md CHANGED
@@ -4,6 +4,11 @@ license: mit
4
 
5
  # ESM-2 QLoRA for Binding Sites Prediction
6
 
 
 
 
 
 
7
 
8
  ## Testing for Overfitting
9
 
 
4
 
5
  # ESM-2 QLoRA for Binding Sites Prediction
6
 
7
+ In this model we added in more QLoRA adapter layers, modifying all of the weight matrices with QLoRA. The differences between the
8
+ train and test metrics, again, are smaller for this model than for the model with fewer adapter layers (only using query, key, and value
9
+ matrices). So, we see that adapting more of the weight matrices in this larger ESM-2 model decreases overfitting and serves as a better
10
+ regularizer. For comparison, see [this model](https://huggingface.co/AmelieSchreiber/esm2_t12_35M_qlora_binding_sites_v0) which only
11
+ has QLoRA adapters on the query, key, and value matrices.
12
 
13
  ## Testing for Overfitting
14