Update README.md
Browse files
README.md
CHANGED
@@ -16,7 +16,7 @@ Original model: https://huggingface.co/internlm/internlm2-chat-20b
|
|
16 |
|
17 |
| Branch | Bits | lm_head bits | VRAM (4k) | VRAM (16k) | VRAM (32k) | Description |
|
18 |
| ------ | ---- | ------------ | ---- | ---- | ---- | ----------- |
|
19 |
-
| [6_5](https://huggingface.co/Bartowski/internlm2-chat-20b-llama-test-exl2/tree/6_5) | 6.5 | 8.0 | 19.6 GB | 21.0 GB |
|
20 |
| [4_25](https://huggingface.co/Bartowski/internlm2-chat-20b-llama-test-exl2/tree/4_25) | 4.25 | 6.0 | 13.8 GB | 15.2 GB | 17.2 GB | GPTQ equivalent bits per weight, slightly higher quality. |
|
21 |
| [3_5](https://huggingface.co/Bartowski/internlm2-chat-20b-llama-test-exl2/tree/3_5) | 3.5 | 6.0 | 12.4 GB | 13.8 GB | 15.8 GB | Lower quality, only use if you have to. |
|
22 |
| [3_0](https://huggingface.co/Bartowski/internlm2-chat-20b-llama-test-exl2/tree/3_0) | 3.0 | 6.0 | 11.1 GB | 12.5 GB | 15.5 GB | Very low quality. Usable on 12GB. |
|
|
|
16 |
|
17 |
| Branch | Bits | lm_head bits | VRAM (4k) | VRAM (16k) | VRAM (32k) | Description |
|
18 |
| ------ | ---- | ------------ | ---- | ---- | ---- | ----------- |
|
19 |
+
| [6_5](https://huggingface.co/Bartowski/internlm2-chat-20b-llama-test-exl2/tree/6_5) | 6.5 | 8.0 | 19.6 GB | 21.0 GB | 23.0 GB | Near unquantized performance at vastly reduced size, **recommended**. |
|
20 |
| [4_25](https://huggingface.co/Bartowski/internlm2-chat-20b-llama-test-exl2/tree/4_25) | 4.25 | 6.0 | 13.8 GB | 15.2 GB | 17.2 GB | GPTQ equivalent bits per weight, slightly higher quality. |
|
21 |
| [3_5](https://huggingface.co/Bartowski/internlm2-chat-20b-llama-test-exl2/tree/3_5) | 3.5 | 6.0 | 12.4 GB | 13.8 GB | 15.8 GB | Lower quality, only use if you have to. |
|
22 |
| [3_0](https://huggingface.co/Bartowski/internlm2-chat-20b-llama-test-exl2/tree/3_0) | 3.0 | 6.0 | 11.1 GB | 12.5 GB | 15.5 GB | Very low quality. Usable on 12GB. |
|