LLM Model, Can I run it?
GPU (optional)
Model (unquantized)
Context Size
Quant Format
Quant format
GGUF
EXL2
GPTQ (coming soon)
BPW
KV Cache
16 bit
8 bit
4 bit
Quantization Size
Q4_K_S
Batch Size
Submit
Model Size (GB)
4.20
Context Size (GB)
6.90
Total Size (GB)
420.69