Should it run on 8gb vram gpu?
#1
by
hlertarinto
- opened
Trying to run it on 3060ti, getting out of memory message
hmm, it should run, because I ran the quantized model on a 1080, but you want to use this code: https://github.com/kuleshov/minillm
Thank you for the reply! Also, your github is very impressive