Apel-sin
/

llama-3-8B-abliterated-v3-exl2

Inference Endpoints

Model card Files Files and versions Community

Apel-sin commited on May 23

Commit

5ad1e40

•

1 Parent(s): 925bbab

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -18,7 +18,7 @@ Calibration dataset: <a href="https://huggingface.co/datasets/cosmicvalor/toxic-
 | Branch | Bits | lm_head bits | VRAM (4k) | VRAM (8K) | VRAM (16k) | VRAM (32k) | Description |
 | ----- | ---- | ------- | ------ | ------ | ------ | ------ | ------------ |
 | [8_0](https://huggingface.co/Apel-sin/llama-3-8B-abliterated-v3-exl2/tree/8_0) | 8.0 | 8.0 | 10.1 GB | 10.5 GB | 11.5 GB | 13.6 GB | Maximum quality that ExLlamaV2 can produce, near unquantized performance. |
-| [6_5](https://huggingface.co/Apel-sin/llama-3-8B-abliterated-v3-exl2//tree/6_5) | 6.5 | 8.0 | 8.9 GB | 9.3 GB | 10.3 GB | 12.4 GB | Very similar to 8.0, good tradeoff of size vs performance, **recommended**. |
 # Llama-3-8B-Instruct-abliterated-v3 Model Card

 | Branch | Bits | lm_head bits | VRAM (4k) | VRAM (8K) | VRAM (16k) | VRAM (32k) | Description |
 | ----- | ---- | ------- | ------ | ------ | ------ | ------ | ------------ |
 | [8_0](https://huggingface.co/Apel-sin/llama-3-8B-abliterated-v3-exl2/tree/8_0) | 8.0 | 8.0 | 10.1 GB | 10.5 GB | 11.5 GB | 13.6 GB | Maximum quality that ExLlamaV2 can produce, near unquantized performance. |
+| [6_5](https://huggingface.co/Apel-sin/llama-3-8B-abliterated-v3-exl2/tree/6_5) | 6.5 | 8.0 | 8.9 GB | 9.3 GB | 10.3 GB | 12.4 GB | Very similar to 8.0, good tradeoff of size vs performance, **recommended**. |
 # Llama-3-8B-Instruct-abliterated-v3 Model Card