Thank you!
Hey, you're a localbasedman, my friend you deserve it.
👏
Hey, you're a localbasedman, my friend you deserve it.
👏
No, you deserve it! You're doing imatrix quants for us lowly 7B mergers. You're awesome!
lowly 7B mergers
Low VRAM friends stay together.
#GPUPoorGang
Especially for smaller quants of smaller models (Q4/Q5/the new IQ3s...), imatrix helps a lot in bringing them pretty much a full step up. So I believe it's great and I'm excited for the recent innovations in GGUF quantizations.
But more than that, I really believe in the potential of the 7B parameter models!
My reasoning/coping:
"Given my VRAM limitations I can only play around with smaller models but I also think they are very interesting, as they are the ones usable by the biggest number of people at the highest speeds for real time chatting applications, especially considering the abundance of affordable 8GB gaming GPUs in the market, these models can be of great quality and ran in background for coding assistance or RAG applications without rendering the system unusable for other tasks, for example, even playing some lighter games while they are running without issues."
lowly 7B mergers
Low VRAM friends stay together.
#GPUPoorGangEspecially for smaller quants of smaller models (Q4/Q5/the new IQ3s...), imatrix helps a lot in bringing them pretty much a full step up. So I believe it's great and I'm excited for the recent innovations in GGUF quantizations.
But more than that, I really believe in the potential of the 7B parameter models!
My reasoning/coping:
"Given my VRAM limitations I can only play around with smaller models but I also think they are very interesting, as they are the ones usable by the biggest number of people at the highest speeds for real time chatting applications, especially considering the abundance of affordable 8GB gaming GPUs in the market, these models can be of great quality and ran in background for coding assistance or RAG applications without rendering the system unusable for other tasks, for example, even playing some lighter games while they are running without issues."
You're so real for that. We need more people like you. People that try to make the most of things while within the consumer grade threshold