Hugging Face's logo

Kooten
/
MiquMaid-v1-70B-3bpw-exl2

Text Generation
Transformers
PyTorch
llama
text-generation-inference
Inference Endpoints
Model card Files Community
1
Kooten commited on
Commit
14b73a4
•
1 Parent(s): 868a4f5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -4,7 +4,9 @@
4
  Exllama quant of [NeverSleep/MiquMaid-v1-70B](https://huggingface.co/NeverSleep/MiquMaid-v1-70B)
5
 
6
  ## Other quants:
7
- EXL2: [3.5bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-3.5bpw-exl2), [3bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-3bpw-exl2), [2.4bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-2.4bpw-exl2)
 
 
8
 
9
  GGUF:
10
  [2bit Imatrix GGUF](https://huggingface.co/Kooten/MiquMaid-v1-70B-IQ2-GGUF)
 
4
  Exllama quant of [NeverSleep/MiquMaid-v1-70B](https://huggingface.co/NeverSleep/MiquMaid-v1-70B)
5
 
6
  ## Other quants:
7
+ EXL2: [6bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-6bpw-exl2), [5bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-5bpw-exl2), [4bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-4bpw-exl2), [3.5bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-3.5bpw-exl2), [3bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-3bpw-exl2), [2.4bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-2.4bpw-exl2)
8
+
9
+ 2.4bpw is probably the most you can fit in a 24gb card
10
 
11
  GGUF:
12
  [2bit Imatrix GGUF](https://huggingface.co/Kooten/MiquMaid-v1-70B-IQ2-GGUF)