What's the minimum system requirement's to run it locally?
#10
by
Tarunasnani
- opened
Trying to run this on i5 8th intel and GTX1050 TI
@Tarunasnani it should roughly take 6gb of ram/vram.
With device map you can probably offload some to cpu and run it but it will be pretty slow.
You can also use bitsandbytes to quantize it to 8 bits and it will only take like 4gb vram and will be faster.
You can use ComfyUI to run it.https://github.com/shadowcz007/comfyui-moondream