bartowski commited on
Commit
33ccdcb
1 Parent(s): 6f23c72

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -11
README.md CHANGED
@@ -23,17 +23,13 @@ Default arguments used except when the bits per weight is above 6.0, at that poi
23
 
24
  Original model: https://huggingface.co/rombodawg/Everyone-Coder-33b-v2-Base
25
 
26
-
27
- <a href="https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/6_5">6.5 bits per weight</a>
28
-
29
- <a href="https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/4_25">4.25 bits per weight</a>
30
-
31
- <a href="https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/3_5">3.5 bits per weight</a>
32
-
33
- <a href="https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/3_0">3.0 bits per weight</a>
34
-
35
- <a href="https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/2_4">2.4 bits per weight</a>
36
-
37
 
38
  ## Download instructions
39
 
 
23
 
24
  Original model: https://huggingface.co/rombodawg/Everyone-Coder-33b-v2-Base
25
 
26
+ | Branch | Bits | lm_head bits | VRAM (4k) | VRAM (16k) | VRAM (32k) | Description |
27
+ | ------ | ---- | ------------ | ---- | ---- | ---- | ----------- |
28
+ | [6_5](https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/6_5) | 6.5 | 8.0 | 28.9 GB | 31.6 GB | 35.6 GB | Near unquantized performance at vastly reduced size, **recommended**. |
29
+ | [4_25](https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/4_25) | 4.25 | 6.0 | 19.5 GB | 22.2 GB | 26.2 GB | GPTQ equivalent bits per weight, slightly higher quality. |
30
+ | [3_5](https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/3_5) | 3.5 | 6.0 | 16.5 GB | 19.2 GB | 23.2 GB | Lower quality, only use if you have to. |
31
+ | [3_0](https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/3_0) | 3.0 | 6.0 | 14.3 GB | 17.0 GB | 21.0 GB | Very low quality, usable with 16gb of VRAM. |
32
+ | [2_4](https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/3_0) | 2.4 | 6.0 | 12.0 GB | 14.7 GB | 18.7 GB | Extremely low quality, only recommended if trying to fit into 12GB with under 4k context. |
 
 
 
 
33
 
34
  ## Download instructions
35