quantize to 4bit

#2
by Kotokin - opened

Hi, do you have the ability to quantize it to 4-bit?

TheBloke and Lonestriker should be releasing the full set of quants for this model soon. I made them aware of it.

EDIT: Artefact2 once again was on top of it! See https://huggingface.co/Artefact2/Midnight-Rose-70B-v2.0.3-GGUF

sophosympatheia changed discussion status to closed

Sign up or log in to comment