NVFP4 Quantization
Collection
5 items • Updated
Using Hardware
- CPU: AMD Ryzen Threadripper PRO 7995WX
- MB: GIGABYTE AI TOP TRX50
- GPU: RTX 5090*1
- RAM: RDIMM DDR5 5600 128GB*2
Using Software
- CUDA version: 13.0
- CUDA driver version: 580.95.05
- pyTorch: 2.10.0+cu130
- transformers: 5.6.0
- llmcompressor: 0.10.1+dev
- vllm: 0.19.1
Backbone Model
Using Dataset
tatsu-lab/alpaca convert as chat template.
Compare
| -- | origin model(BF16) | Quantization(This Model) |
|---|---|---|
| Leaderboard_GPQA | 0.8232 | 0.7677 |
| VRAM Usage | 17.66 GiB | 10.58 GiB |
- Downloads last month
- 1