Update README.md
Browse files
README.md
CHANGED
@@ -23,17 +23,13 @@ Default arguments used except when the bits per weight is above 6.0, at that poi
|
|
23 |
|
24 |
Original model: https://huggingface.co/rombodawg/Everyone-Coder-33b-v2-Base
|
25 |
|
26 |
-
|
27 |
-
|
28 |
-
|
29 |
-
|
30 |
-
|
31 |
-
|
32 |
-
|
33 |
-
<a href="https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/3_0">3.0 bits per weight</a>
|
34 |
-
|
35 |
-
<a href="https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/2_4">2.4 bits per weight</a>
|
36 |
-
|
37 |
|
38 |
## Download instructions
|
39 |
|
|
|
23 |
|
24 |
Original model: https://huggingface.co/rombodawg/Everyone-Coder-33b-v2-Base
|
25 |
|
26 |
+
| Branch | Bits | lm_head bits | VRAM (4k) | VRAM (16k) | VRAM (32k) | Description |
|
27 |
+
| ------ | ---- | ------------ | ---- | ---- | ---- | ----------- |
|
28 |
+
| [6_5](https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/6_5) | 6.5 | 8.0 | 28.9 GB | 31.6 GB | 35.6 GB | Near unquantized performance at vastly reduced size, **recommended**. |
|
29 |
+
| [4_25](https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/4_25) | 4.25 | 6.0 | 19.5 GB | 22.2 GB | 26.2 GB | GPTQ equivalent bits per weight, slightly higher quality. |
|
30 |
+
| [3_5](https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/3_5) | 3.5 | 6.0 | 16.5 GB | 19.2 GB | 23.2 GB | Lower quality, only use if you have to. |
|
31 |
+
| [3_0](https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/3_0) | 3.0 | 6.0 | 14.3 GB | 17.0 GB | 21.0 GB | Very low quality, usable with 16gb of VRAM. |
|
32 |
+
| [2_4](https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/3_0) | 2.4 | 6.0 | 12.0 GB | 14.7 GB | 18.7 GB | Extremely low quality, only recommended if trying to fit into 12GB with under 4k context. |
|
|
|
|
|
|
|
|
|
33 |
|
34 |
## Download instructions
|
35 |
|