Apel-sin
/

llama-3-8B-abliterated-v3-exl2

Inference Endpoints

Model card Files Files and versions Community

Apel-sin commited on May 24

Commit

85bcbbc

•

1 Parent(s): 5ad1e40

Update README.md

Files changed (1) hide show

README.md +1 -0

README.md CHANGED Viewed

@@ -19,6 +19,7 @@ Calibration dataset: <a href="https://huggingface.co/datasets/cosmicvalor/toxic-
 | ----- | ---- | ------- | ------ | ------ | ------ | ------ | ------------ |
 | [8_0](https://huggingface.co/Apel-sin/llama-3-8B-abliterated-v3-exl2/tree/8_0) | 8.0 | 8.0 | 10.1 GB | 10.5 GB | 11.5 GB | 13.6 GB | Maximum quality that ExLlamaV2 can produce, near unquantized performance. |
 | [6_5](https://huggingface.co/Apel-sin/llama-3-8B-abliterated-v3-exl2/tree/6_5) | 6.5 | 8.0 | 8.9 GB | 9.3 GB | 10.3 GB | 12.4 GB | Very similar to 8.0, good tradeoff of size vs performance, **recommended**. |
 # Llama-3-8B-Instruct-abliterated-v3 Model Card

 | ----- | ---- | ------- | ------ | ------ | ------ | ------ | ------------ |
 | [8_0](https://huggingface.co/Apel-sin/llama-3-8B-abliterated-v3-exl2/tree/8_0) | 8.0 | 8.0 | 10.1 GB | 10.5 GB | 11.5 GB | 13.6 GB | Maximum quality that ExLlamaV2 can produce, near unquantized performance. |
 | [6_5](https://huggingface.co/Apel-sin/llama-3-8B-abliterated-v3-exl2/tree/6_5) | 6.5 | 8.0 | 8.9 GB | 9.3 GB | 10.3 GB | 12.4 GB | Very similar to 8.0, good tradeoff of size vs performance, **recommended**. |
+| [5_0](https://huggingface.co/Apel-sin/llama-3-8B-abliterated-v3-exl2/tree/5_0) | 5.0 | 8.0 | 7.7 GB | 8.1 GB | 9.1 GB | 11.2 GB | Slightly lower quality vs 6.5, but usable on 8GB cards. |
 # Llama-3-8B-Instruct-abliterated-v3 Model Card