Update README.md
Browse files
README.md
CHANGED
|
@@ -32,7 +32,7 @@ I have quantised the GGML files in this repo with the latest version. Therefore
|
|
| 32 |
| guanaco-65B.ggmlv3.q4_1.bin | q4_1 | 4 | 40.81 GB | 43.31 GB | 4-bit. Higher accuracy than q4_0 but not as high as q5_0. However has quicker inference than q5 models. |
|
| 33 |
| guanaco-65B.ggmlv3.q5_0.bin | q5_0 | 5 | 44.89 GB | 47.39 GB | 5-bit. Higher accuracy, higher resource usage and slower inference. |
|
| 34 |
| guanaco-65B.ggmlv3.q5_1.bin | q5_1 | 5 | 48.97 GB | 51.47 GB | 5-bit. Even higher accuracy, resource usage and slower inference. |
|
| 35 |
-
| guanaco-65B.ggmlv3.q8_0.bin | q8_0 | 8 | 69.
|
| 36 |
|
| 37 |
### q8_0 file requires expansion from archive
|
| 38 |
|
|
|
|
| 32 |
| guanaco-65B.ggmlv3.q4_1.bin | q4_1 | 4 | 40.81 GB | 43.31 GB | 4-bit. Higher accuracy than q4_0 but not as high as q5_0. However has quicker inference than q5 models. |
|
| 33 |
| guanaco-65B.ggmlv3.q5_0.bin | q5_0 | 5 | 44.89 GB | 47.39 GB | 5-bit. Higher accuracy, higher resource usage and slower inference. |
|
| 34 |
| guanaco-65B.ggmlv3.q5_1.bin | q5_1 | 5 | 48.97 GB | 51.47 GB | 5-bit. Even higher accuracy, resource usage and slower inference. |
|
| 35 |
+
| guanaco-65B.ggmlv3.q8_0.bin | q8_0 | 8 | 69.37 GB | 71.87 GB | 8-bit. Almost indistinguishable from float16. Huge resource use and slow. Not recommended for most use cases. |
|
| 36 |
|
| 37 |
### q8_0 file requires expansion from archive
|
| 38 |
|