GGUF quantization request
Can you please release an 8-bit quantization? The model is too big for my GPU.
Crazyyy
What GPU, and why are you trying to run this on CUDA?!
IBM CGA 16 KB VRAM
Crazyyy
What GPU, and why are you trying to run this on CUDA?!
That must have costed several pennies 😭
Too expensive for us average consumers to even dream about
i wonder how far a few more thousand parameters can be pushed with a custom architecture
byte-level and looped
would it language
IBM CGA 16 KB VRAM
Crazyyy
What GPU, and why are you trying to run this on CUDA?!
Wow you have that??? Too bad i have no GPU /j
i wonder how far a few more thousand parameters can be pushed with a custom architecture
byte-level and looped
would it language
Probably not. But check out Negative-v1.1, it kinda gets close to that.
yeah Negative is insane
i wonder how far a few more thousand parameters can be pushed with a custom architecture
byte-level and looped
would it language
No
i wonder how far a few more thousand parameters can be pushed with a custom architecture
byte-level and looped
would it language
No
Yeah neither Negative, nor Er-Tiny, 'language'.
IBM CGA 16 KB VRAM
Crazyyy
What GPU, and why are you trying to run this on CUDA?!
The funny thing is that yes, it is VRAM, but what's stored in there is simply what is sent to the monitor. So you're not actually using it for inference (although it is mapped to the system RAM in the CPU), and you're just drawing KV caches to the screen so you'd have to use COM.