Commit
·
dc3ce23
1
Parent(s):
4312921
Update README.md
Browse files
README.md
CHANGED
|
@@ -55,7 +55,7 @@ pipeline_tag: text-generation
|
|
| 55 |
|
| 56 |
Heavily inspired by [Hivemind's GPT-J-6B with 8-bit weights](https://huggingface.co/hivemind/gpt-j-6B-8bit), this is a version of [bigscience/bloom](https://huggingface.co/bigscience/bloom) a ~176 billions parameters language model that you run and fine-tune with less memory.
|
| 57 |
|
| 58 |
-
Here, we also apply [LoRA (Low Rank
|
| 59 |
|
| 60 |
### How to use
|
| 61 |
|
|
|
|
| 55 |
|
| 56 |
Heavily inspired by [Hivemind's GPT-J-6B with 8-bit weights](https://huggingface.co/hivemind/gpt-j-6B-8bit), this is a version of [bigscience/bloom](https://huggingface.co/bigscience/bloom) a ~176 billions parameters language model that you run and fine-tune with less memory.
|
| 57 |
|
| 58 |
+
Here, we also apply [LoRA (Low Rank Adapters)](https://arxiv.org/abs/2106.09685) to reduce model size. The original version takes ~353GB memory, this version takes ~180GB.
|
| 59 |
|
| 60 |
### How to use
|
| 61 |
|