Hello, can the rtx 3050 8g run this model?
#1
by
win10
- opened
May I ask if the rtx 3050 8g can run this model?
No, this model requires about 14Gbs just to load, and will consume as much VRAM you can possibly feed it to store context.
I'd say that a 24Gb card is the minimum & that'd let you load about 5k tokens in memory.
TehVenom
changed discussion status to
closed