Hugging Face's logo

seb-c
/
Psydestroyer-20B-GGUF

GGUF
Inference Endpoints
Model card Files Community
Psydestroyer-20B-GGUF / README.md
sebc
Add Q4_K_M
e7e00ec
preview code
|
raw
history blame contribute delete
374 Bytes
metadata
license: llama2

Psydestroyer 20B GGUFs

GGUF Quants of my Psydestroyer-20B merge.

Currently I only have a Q4-K-M, as that is the highest (I've tested) quant of a 20B frankenmerge that can mostly fit on my 3060 12GB and run at decent t/s (~8-9). If the demand is there I can create more, otherwise you'll probably have to download and quantize the model yourself.