DeepSeek-R1-Distill-Llama-70B EXL2 7.0bpw h6
EXL2 quantization of deepseek-ai/DeepSeek-R1-Distill-Llama-70B
Quantization Details
- Format: EXL2
- Bits per weight: 7.0bpw
- Head bits: 6
- Quantized with: ExLlamaV2 0.3.1
- Calibration: Default ExLlamaV2 calibration dataset
Hardware Requirements
- Minimum VRAM: ~40GB (e.g. 2x RTX 3090)
- Recommended: 48GB+ for comfortable context
Usage
Load with TabbyAPI or text-generation-webui with ExLlamaV2 loader.
Original Model
See deepseek-ai/DeepSeek-R1-Distill-Llama-70B for full details, license, and usage recommendations.
- Downloads last month
- 18
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support