Update README.md
Browse files
README.md
CHANGED
|
@@ -48,6 +48,9 @@ tags:
|
|
| 48 |
<!-- description start -->
|
| 49 |
## Description
|
| 50 |
|
|
|
|
|
|
|
|
|
|
| 51 |
Experimental exl2 quantization for CausalLM-14B for Exllamav2.
|
| 52 |
I had some issues during quantization process, so I suspect it might have quality issues.
|
| 53 |
3.5bpw version barely fits 12GB VRAM but has unusually high perplexity for wikitext dataset.
|
|
|
|
| 48 |
<!-- description start -->
|
| 49 |
## Description
|
| 50 |
|
| 51 |
+
[4bpw h6](https://huggingface.co/cgus/CausalLM-14B-exl2/tree/main)
|
| 52 |
+
[3.5bpw h6](https://huggingface.co/cgus/CausalLM-14B-exl2/tree/3.5bpw-h6)
|
| 53 |
+
|
| 54 |
Experimental exl2 quantization for CausalLM-14B for Exllamav2.
|
| 55 |
I had some issues during quantization process, so I suspect it might have quality issues.
|
| 56 |
3.5bpw version barely fits 12GB VRAM but has unusually high perplexity for wikitext dataset.
|