Hugging Face's logo

Kooten
/
MiquMaid-v1-70B-IQ2-GGUF

GGUF
Inference Endpoints
Model card Files Community
2
Edit model card
YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

MiquMaid-v1-70B IQ2

Description

2bit imatrix GGUF quants of NeverSleep/MiquMaid-v1-70B

Imatrix generated from q8 of MiquMaid, 2000 chunks at 500 ctx. Dataset was Wikitext.

These quants take a while to do so please leave a like or a comment on the repo so that i know if there is interest.

Other quants:

EXL2: 3.5bpw, 3bpw, 2.4bpw

2bit Imatrix GGUF: IQ2-XS, IQ2-XXS, IQ3-XXS

Custom format: 

### Instruction:
{system prompt}

### Input:
{input}

### Response:
{reply}

Contact

Kooten on discord

ko-fi.com/kooten

Downloads last month
82
GGUF
Model size
69B params
Architecture
llama

2-bit

3-bit

Inference API
Unable to determine this model's library. Check the docs .

Collection including Kooten/MiquMaid-v1-70B-IQ2-GGUF