Question about VRAM requirements
#12
by
sebastiaanvandenbroeck
- opened
Quick question, would it be possible to run this model on a card with 12GB of memory?
Thanks in advance!
We recommend using bf16, but even with that the model should have a memory footprint of about 14GB (= 7B params * 2 bytes/parameter).
So, I don't think 12GB of VRAM will be enough.
That's unfortunate, but thanks for answering my question!
atrott
changed discussion status to
closed
Hello, how fast does this model run on enterprise GPUs with triton?
i need to use storywriter with 65k token what are the hardware requirements