Commit
•
4da900a
1
Parent(s):
8a7481d
Update README.md
Browse files
README.md
CHANGED
|
@@ -17,15 +17,25 @@ CUDA_VISIBLE_DEVICES=0 python llama.py ./models/chavinlo-gpt4-x-alpaca
|
|
| 17 |
|
| 18 |
|
| 19 |
Prev. info
|
|
|
|
| 20 |
Quantized on GPTQ-for-LLaMa commit 5955e9c67d9bfe8a8144ffbe853c2769f1e87cdd
|
|
|
|
| 21 |
GPTQ 4bit quantization of: https://huggingface.co/chavinlo/gpt4-x-alpaca
|
|
|
|
| 22 |
Note: This was quantized with this branch of GPTQ-for-LLaMA: https://github.com/qwopqwop200/GPTQ-for-LLaMa/tree/triton
|
|
|
|
| 23 |
Because of this, it appears to be incompatible with Oobabooga at the moment. Stay tuned?
|
| 24 |
|
| 25 |
Command:
|
|
|
|
| 26 |
CUDA_VISIBLE_DEVICES=0 python llama.py ./models/chavinlo-gpt4-x-alpaca
|
|
|
|
| 27 |
--wbits 4
|
|
|
|
| 28 |
--true-sequential
|
|
|
|
| 29 |
--act-order
|
|
|
|
| 30 |
--groupsize 128
|
|
|
|
| 31 |
--save gpt-x-alpaca-13b-native-4bit-128g.pt
|
|
|
|
| 17 |
|
| 18 |
|
| 19 |
Prev. info
|
| 20 |
+
|
| 21 |
Quantized on GPTQ-for-LLaMa commit 5955e9c67d9bfe8a8144ffbe853c2769f1e87cdd
|
| 22 |
+
|
| 23 |
GPTQ 4bit quantization of: https://huggingface.co/chavinlo/gpt4-x-alpaca
|
| 24 |
+
|
| 25 |
Note: This was quantized with this branch of GPTQ-for-LLaMA: https://github.com/qwopqwop200/GPTQ-for-LLaMa/tree/triton
|
| 26 |
+
|
| 27 |
Because of this, it appears to be incompatible with Oobabooga at the moment. Stay tuned?
|
| 28 |
|
| 29 |
Command:
|
| 30 |
+
|
| 31 |
CUDA_VISIBLE_DEVICES=0 python llama.py ./models/chavinlo-gpt4-x-alpaca
|
| 32 |
+
|
| 33 |
--wbits 4
|
| 34 |
+
|
| 35 |
--true-sequential
|
| 36 |
+
|
| 37 |
--act-order
|
| 38 |
+
|
| 39 |
--groupsize 128
|
| 40 |
+
|
| 41 |
--save gpt-x-alpaca-13b-native-4bit-128g.pt
|