24 GB VRAM
Collection
Quants that run fast on single 3090/4090 card with 24GB of VRAM and 4096 context length
•
16 items
•
Updated
•
5
BigMaid-20B-v2.0
exllamav2 quant for TeeZee/BigMaid-20B-v2.0 using specialized dataset openerotica/erotiquant
Runs smoothly on single 3090 in webui with context length set to 4096, ExLlamav2_HF loader and cache_8bit=True
All comments are greatly appreciated, download, test and if you appreciate my work, consider buying me my fuel:
- Downloads last month
- 7
Model is too large to load in Inference API (serverless). To try the model, launch it on Inference Endpoints (dedicated)
instead.