Hugging Face's logo

Kooten
/
MiquMaid-v1-70B-3.5bpw-exl2

Text Generation
Transformers
PyTorch
llama
text-generation-inference
Inference Endpoints
Model card Files Community
Kooten commited on
Commit
0e996e9
•
1 Parent(s): 18e08f4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -5
README.md CHANGED
@@ -4,14 +4,13 @@
4
  Exllama quant of [NeverSleep/MiquMaid-v1-70B](https://huggingface.co/NeverSleep/MiquMaid-v1-70B)
5
 
6
  ## Other quants:
7
- EXL2: [3.5bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-3.5bpw-exl2), [3bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-3bpw-exl2), [2.4bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-2.4bpw-exl2)
8
 
9
- GGUF:
10
 
 
11
  [2bit Imatrix GGUF](https://huggingface.co/Kooten/MiquMaid-v1-70B-IQ2-GGUF)
12
 
13
- [NeverSleeps Official Quants](https://huggingface.co/NeverSleep/MiquMaid-v1-70B-GGUF)
14
-
15
  ### Custom format:
16
  ```
17
  ### Instruction:
@@ -23,5 +22,8 @@ GGUF:
23
  ### Response:
24
  {reply}
25
  ```
 
26
  ## Contact
27
- Kooten on discord
 
 
 
4
  Exllama quant of [NeverSleep/MiquMaid-v1-70B](https://huggingface.co/NeverSleep/MiquMaid-v1-70B)
5
 
6
  ## Other quants:
7
+ EXL2: [6bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-6bpw-exl2), [5bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-5bpw-exl2), [4bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-4bpw-exl2), [3.5bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-3.5bpw-exl2), [3bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-3bpw-exl2), [2.4bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-2.4bpw-exl2)
8
 
9
+ 2.4bpw is probably the most you can fit in a 24gb card
10
 
11
+ GGUF:
12
  [2bit Imatrix GGUF](https://huggingface.co/Kooten/MiquMaid-v1-70B-IQ2-GGUF)
13
 
 
 
14
  ### Custom format:
15
  ```
16
  ### Instruction:
 
22
  ### Response:
23
  {reply}
24
  ```
25
+
26
  ## Contact
27
+ Kooten on discord
28
+
29
+ [ko-fi.com/kooten](https://ko-fi.com/kooten)