Spaces:
Running
Running
Update README.md
Browse files
README.md
CHANGED
@@ -33,4 +33,9 @@ Denomination specific models like ChristianGPT-catholic are based on ChristianGP
|
|
33 |
|
34 |
## Training requirements:
|
35 |
- GPU with over 16GB of memory (we trained on NVIDIA Tesla V100 32GB and NVIDIA Ampere A6000 45GB)
|
36 |
-
- 30GB of RAM (the raw model weights are about 29GB, our models are cast to 8bit to use less memory)
|
|
|
|
|
|
|
|
|
|
|
|
33 |
|
34 |
## Training requirements:
|
35 |
- GPU with over 16GB of memory (we trained on NVIDIA Tesla V100 32GB and NVIDIA Ampere A6000 45GB)
|
36 |
+
- 30GB of RAM (the raw model weights are about 29GB, our models are cast to 8bit to use less memory)
|
37 |
+
|
38 |
+
- ## References
|
39 |
+
We leaned heavily on open-source libraries like `transformers`, `peft`, and `bitsandbytes` for this project.
|
40 |
+
- Dettmers, Tim, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. "[LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://arxiv.org/abs/2208.07339)." *arXiv preprint arXiv:2208.07339*.
|
41 |
+
- Hu, Edward J., Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. "[LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685)." *arXiv preprint arXiv:2106.09685*.
|