can we get some Q8 npu2 ?
#1
by skywalk819x - opened
glm 4.7 q8
qwen 3.6 37b q8
ornith 1.0 q8
The q4nx format is inherently q4_1 based, unfortunately. I do start from either q8_0 or f16/bf16 sources so it's a little less bad. As for Qwen and Ornith, I've actually been working on Ornith based on the Darwin work.