Is ARM quants possible?

#1
by mitsu89 - opened

I tried to use this model in my pc and it is great. Even better than this one

https://huggingface.co/SicariusSicariiStuff/EVA-UNIT-01_EVA-Qwen2.5-7B-v0.0_ARM

But i can't run this Meissa model on my phone (termux koboldai) without arm quants. What i need is a q4_0_4_4 gguf version if possible. Thanks.

Quant Factory org

Updated to include the Q4_0_4_4 quant.

aashish1904 changed discussion status to closed

Sign up or log in to comment