| # MiquMaid-v1-70B 3.5bpw | |
| ## Description | |
| Exllama quant of [NeverSleep/MiquMaid-v1-70B](https://huggingface.co/NeverSleep/MiquMaid-v1-70B) | |
| ## Other quants: | |
| EXL2: [6bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-6bpw-exl2), [5bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-5bpw-exl2), [4bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-4bpw-exl2), [3.5bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-3.5bpw-exl2), [3bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-3bpw-exl2), [2.4bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-2.4bpw-exl2) | |
| 2.4bpw is probably the most you can fit in a 24gb card | |
| GGUF: | |
| [2bit Imatrix GGUF](https://huggingface.co/Kooten/MiquMaid-v1-70B-IQ2-GGUF) | |
| ### Custom format: | |
| ``` | |
| ### Instruction: | |
| {system prompt} | |
| ### Input: | |
| {input} | |
| ### Response: | |
| {reply} | |
| ``` | |
| ## Contact | |
| Kooten on discord | |
| [ko-fi.com/kooten](https://ko-fi.com/kooten) |