24GB VRAM Optimal Quants Collection When asked what I use locally on a 24GB card, this is what I point to. I favor exl2s for long context, GGUF for very short context. • 12 items • Updated Oct 31, 2024 • 3
Systran/faster-distil-whisper-large-v2 Automatic Speech Recognition • Updated May 9, 2024 • 17.3k • 17