Dracones

/

c4ai-command-r-plus_exl2_5.5bpw

like 1

For anyone wondering about vram usage I got it to run on my A100 with 67k context at 77.6gb of vram. This is with cache_4bit aswell.