Model can only run on 40G+ vram GPUS.
#1
by
Qubitium
- opened
multi GPU vram split works but bitsandbytes doesn't work when varm is split to more than 1 GPU.
multi GPU vram split works but bitsandbytes doesn't work when varm is split to more than 1 GPU.