turboderp

/

Mistral-Large-Instruct-2407-123B-exl2

like 12

Yes! It would be much appreciated!

I added 4.75bpw and 5.0bpw versions just now.

Hi!

Are you going to upload a 5.0?, I think it could fit in a 4x3090/4090 system at low context.

Thanks!

If I read your post correctly, you have 4x3090 and 1x4090? Why not use the 6.0bpw quant? That should fit in 120GB VRAM.