Update README.md
Browse files
README.md
CHANGED
|
@@ -53,7 +53,7 @@ tags:
|
|
| 53 |
|
| 54 |
Experimental exl2 quantization for CausalLM-14B for Exllamav2.
|
| 55 |
I had some issues during quantization process, so I suspect it might have quality issues.
|
| 56 |
-
3.5bpw version barely fits 12GB VRAM but has unusually high perplexity for wikitext dataset.
|
| 57 |
I couldn't measure perplexity for 4bpw version and to compare it with TheBloke's GPTQ, so I have no idea if my quantization has issues or it supposed to be like this.
|
| 58 |
|
| 59 |
You could try this exl2 version but I'd recommend to use [TheBloke's GPTQ](https://huggingface.co/TheBloke/CausalLM-14B-GPTQ) version instead.
|
|
|
|
| 53 |
|
| 54 |
Experimental exl2 quantization for CausalLM-14B for Exllamav2.
|
| 55 |
I had some issues during quantization process, so I suspect it might have quality issues.
|
| 56 |
+
3.5bpw version barely fits my 12GB VRAM but has unusually high perplexity for wikitext dataset.
|
| 57 |
I couldn't measure perplexity for 4bpw version and to compare it with TheBloke's GPTQ, so I have no idea if my quantization has issues or it supposed to be like this.
|
| 58 |
|
| 59 |
You could try this exl2 version but I'd recommend to use [TheBloke's GPTQ](https://huggingface.co/TheBloke/CausalLM-14B-GPTQ) version instead.
|