Possibly get a 2.75bpw?
#2
by
Adzeiros
- opened
Could you please make a 2.75bpw for us 48GB VRAM users xD
we have uploaded the measurement: https://huggingface.co/anthracite-org/magnum-v4-123b-exl2/blob/main/measurement.json so you can re-quant to any other size required; skipping the initial measurement steps that take the longest and require the most ram/vram, hope that helps.
lucyknada
changed discussion status to
closed