Sharded Model request!
#8
by
J001
- opened
I have tried loading this model OOM error in a low-RAM, high-VRAM system, with text-generation-webui；
I think the size of this model > 12GB System RAM.
Can I load with a Sharded model chunk by chuck to reduce RAM requirements?
Or it just the text-generation-webui limitation?