Hugging Face's logo

TheBloke
/
stable-vicuna-13B-GGML

Model card Files Community
5
TheBloke commited on
Commit
d065371
•
1 Parent(s): 369aaf9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -0
README.md CHANGED
@@ -46,6 +46,7 @@ The previous files, which will still work in older versions of llama.cpp, can be
46
  `stable-vicuna-13B.ggml.q4_0.bin` | q4_0 | 4bit | 8.14GB | 10.5GB | 4bit. |
47
  `stable-vicuna-13B.ggml.q5_0.bin` | q5_0 | 5bit | 8.95GB | 11.0GB | 5bit. Higher accuracy, higher resource usage and slower inference. |
48
  `stable-vicuna-13B.ggml.q5_1.bin` | q5_1 | 5bit | 9.76GB | 12.25GB | 5bit. Higher accuracy than q5_0, but again higher resource usage and slower inference. |
 
49
 
50
  ## How to run in `llama.cpp`
51
 
 
46
  `stable-vicuna-13B.ggml.q4_0.bin` | q4_0 | 4bit | 8.14GB | 10.5GB | 4bit. |
47
  `stable-vicuna-13B.ggml.q5_0.bin` | q5_0 | 5bit | 8.95GB | 11.0GB | 5bit. Higher accuracy, higher resource usage and slower inference. |
48
  `stable-vicuna-13B.ggml.q5_1.bin` | q5_1 | 5bit | 9.76GB | 12.25GB | 5bit. Higher accuracy than q5_0, but again higher resource usage and slower inference. |
49
+ `stable-vicuna-13B.ggml.q8_0.bin` | q8_0 | 8bit | 14.6GB | 17GB | 8-bit. Almost indistinguishable from float16. Huge resource use and slow. Not recommended for normal use. |
50
 
51
  ## How to run in `llama.cpp`
52