AmelieSchreiber
commited on
Commit
•
8b577db
1
Parent(s):
2f8efbb
Update README.md
Browse files
README.md
CHANGED
@@ -12,8 +12,8 @@ tags:
|
|
12 |
|
13 |
# ESM-2 QLoRA
|
14 |
|
15 |
-
These are the checkpoints for the first ever QLoRA for ESM-2!
|
16 |
-
|
17 |
Scaling to larger models for better metrics is in progress. These checkpoints were trained using
|
18 |
[the 600K dataset](https://huggingface.co/datasets/AmelieSchreiber/600K_data). To replicate the training of QLoRA for ESM-2 models,
|
19 |
you can use the `conda-environment.yml` file. However, for the next week or two (28/09/2023) you will need to uninstall transformers
|
|
|
12 |
|
13 |
# ESM-2 QLoRA
|
14 |
|
15 |
+
These are the checkpoints for the first ever QLoRA for ESM-2! You can load and use them similarly to the LoRA models.
|
16 |
+
This is the smallest `esm2_t6_8M_UR50D` model, so the metrics aren't great.
|
17 |
Scaling to larger models for better metrics is in progress. These checkpoints were trained using
|
18 |
[the 600K dataset](https://huggingface.co/datasets/AmelieSchreiber/600K_data). To replicate the training of QLoRA for ESM-2 models,
|
19 |
you can use the `conda-environment.yml` file. However, for the next week or two (28/09/2023) you will need to uninstall transformers
|