File size: 877 Bytes
7dbdfe9 0e996e9 7dbdfe9 0e996e9 7dbdfe9 0e996e9 7dbdfe9 0e996e9 7dbdfe9 0e996e9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
# MiquMaid-v1-70B 3.5bpw
## Description
Exllama quant of [NeverSleep/MiquMaid-v1-70B](https://huggingface.co/NeverSleep/MiquMaid-v1-70B)
## Other quants:
EXL2: [6bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-6bpw-exl2), [5bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-5bpw-exl2), [4bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-4bpw-exl2), [3.5bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-3.5bpw-exl2), [3bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-3bpw-exl2), [2.4bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-2.4bpw-exl2)
2.4bpw is probably the most you can fit in a 24gb card
GGUF:
[2bit Imatrix GGUF](https://huggingface.co/Kooten/MiquMaid-v1-70B-IQ2-GGUF)
### Custom format:
```
### Instruction:
{system prompt}
### Input:
{input}
### Response:
{reply}
```
## Contact
Kooten on discord
[ko-fi.com/kooten](https://ko-fi.com/kooten) |