# Jamba | |
- β qlora w/ deepspeed Zero-2 needs at least 2x GPUs and | |
- 35GiB VRAM per GPU w minimal context length | |
- 56GiB VRAM per GPU (w multipack enabled) | |
- β qlora w/ deepspeed Zero-3 needs at least 2x GPUs and 67GiB VRAM (wtf?) | |
- β qlora single-gpu, ~51GiB VRAM | |
- β multipack | |
- β FSDP | |
- β 8-bit LoRA | |