g023/Qwopus3.5-9B-v3-NF4 · Hugging Face https://huggingface.co/g023/Qwopus3.5-9B-v3-NF4
#2209
by majsta88 - opened
Hey. First of, thank you for the great work you do for the comunity. I dont know if this makes for an interetsting model request but read about TriAttention in an Nvidia article and I see there is an Qwopus 9B version available levereging this. g023/Qwopus3.5-9B-v3-NF4 · Hugging Face https://huggingface.co/g023/Qwopus3.5-9B-v3-NF4
Have a look, and if this is something interesting to do please do it.
I am looking at smaller models for my limited hardware (GTX1070) so any reduction in size is great.
Thanks
majsta88 changed discussion title from g023/Qwopus3.5-9B-v3-NF4 · Hugging Face https://share.google/PiBupc7iD7OrWJqcB to g023/Qwopus3.5-9B-v3-NF4 · Hugging Face https://huggingface.co/g023/Qwopus3.5-9B-v3-NF4
majsta88 changed discussion status to closed
Ignore. Misunderstood the intent of the project.