hgloow
/

Merged-AGI-7B-EXL2

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

hgloow commited on Dec 31, 2023

Commit

169a850

•

1 Parent(s): 921dd1a

Update README.md

Files changed (1) hide show

README.md +6 -6

README.md CHANGED Viewed

@@ -32,12 +32,12 @@ Zipped Quantization (if you want to download a single file)
 ## Memory Usage
 Measured using ExLlamaV2 and 4096 max_seq_len with [Oobabooga's Text Generation WebUI](https://github.com/oobabooga/text-generation-webui/tree/main).
-| Branch | BPW | VRAM Usage |
-| ------ | --- | ---------- |
-[3.0bpw](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/3.0bpw)|3.0|3.7 GB
-[4.0bpw (main)](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/main)|4.0|4.4 GB
-[6.0bpw](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/6.0bpw)|6.0|6.1 GB
-[8.0bpw](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/8.0bpw)|8.0|7.7 GB
 ## Prompt template: ChatML

 ## Memory Usage
 Measured using ExLlamaV2 and 4096 max_seq_len with [Oobabooga's Text Generation WebUI](https://github.com/oobabooga/text-generation-webui/tree/main).
+| Branch | BPW | VRAM Usage | Description |
+| ------ | --- | ---------- | ----------- |
+[3.0bpw](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/3.0bpw)|3.0|3.7 GB|For >=6GB VRAM cards
+[4.0bpw (main)](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/main)|4.0|4.4 GB|For >=6GB VRAM cards
+[6.0bpw](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/6.0bpw)|6.0|6.1 GB|For >=8GB VRAM cards
+[8.0bpw](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/8.0bpw)|8.0|7.7 GB|For >=10GB VRAM cards
 ## Prompt template: ChatML