Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
datasets:
|
3 |
+
- OpenAssistant/oasst1
|
4 |
+
---
|
5 |
+
|
6 |
+
<p><strong><font size="5">Information</font></strong></p>
|
7 |
+
OpenAssistant-Llama-13B-4-bit working with GPTQ versions used in Oobabooga's Text Generation Webui and KoboldAI.
|
8 |
+
<br>This was made using <a href="https://huggingface.co/serpdotai/llama-oasst-lora-13B">Serpdotai's Open Assistant 13b LoRa trained for 4 epochs</a> using Open Assistant's dataset.</p>
|
9 |
+
|
10 |
+
<code>python llama.py /KoboldAI/repos/gptq/llama13b-oasst-4-epochs-lora c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors llama13b-oasst-4-epochs-lora-4bit-128g.safetensors</code>
|
11 |
+
|
12 |
+
|
13 |
+
|
14 |
+
<p><strong><font size="5">Benchmarks</font></strong></p>
|
15 |
+
|
16 |
+
<p><strong><font size="4">--true-sequential --groupsize 128</font></strong></p>
|
17 |
+
|
18 |
+
<strong>Wikitext2</strong>: 5.380471229553223
|
19 |
+
|
20 |
+
<strong>Ptb-New</strong>: 31.921072006225586
|
21 |
+
|
22 |
+
<strong>C4-New</strong>: 7.140256881713867
|
23 |
+
|
24 |
+
<strong>Note</strong>: This version uses <i>--groupsize 128</i>, resulting in better evaluations. However, it consumes more VRAM.
|