Update README.md
Browse files
README.md
CHANGED
@@ -19,4 +19,10 @@ This model is llama-3-8b-instruct from Meta (uploaded by unsloth) trained on the
|
|
19 |
|
20 |
The Qalore method uses Qlora training along with the methods from Galore for additional reductions in VRAM allowing for llama-3-8b to be loaded on 14.5 GB of VRAM. This allowed this training to be completed on an RTX A4000 16GB in 130 hours for less than $20.
|
21 |
|
|
|
|
|
22 |
- https://huggingface.co/datasets/Replete-AI/OpenCodeInterpreterData
|
|
|
|
|
|
|
|
|
|
19 |
|
20 |
The Qalore method uses Qlora training along with the methods from Galore for additional reductions in VRAM allowing for llama-3-8b to be loaded on 14.5 GB of VRAM. This allowed this training to be completed on an RTX A4000 16GB in 130 hours for less than $20.
|
21 |
|
22 |
+
Dataset used for training this model:
|
23 |
+
|
24 |
- https://huggingface.co/datasets/Replete-AI/OpenCodeInterpreterData
|
25 |
+
|
26 |
+
Qalore notebook for training:
|
27 |
+
|
28 |
+
- https://colab.research.google.com/drive/1bX4BsjLcdNJnoAf7lGXmWOgaY8yekg8p?usp=sharing
|