anthracite-org

/

magnum-v3-9b-chatml-gguf

like 6

Follow

Anthracite 246

I have forwarded it to the person on our team that does the mobile quants, but haven't heard back yet; I'd assume we won't publish those unless they re-quant, if there's any update I'll post it here.

Hi, can you add Q4_0_4_8 quant, please?

Will take a look this afternoon and update when completed.

@GemGameRU Here you go: https://huggingface.co/anthracite-org/magnum-v3-9b-chatml-gguf/blob/main/magnum-v3-9b-chatml-Q4_0_4_8.gguf

Generated with imatrix, so this will be the fastest and highest fidelity way to run on mobile. Enjoy!