Qwen2.5-3B (SpectralQ 6-bit) This model contains experimental 6-bit frequency-domain weights quantized using the Transformer Spectral Scaling Law. It utilizes 256-width DCT macroblocks and a 30-bit chunk-major packed layout.
鈿狅笍 IMPORTANT: HOW TO USE 鈿狅笍 This model cannot be loaded with standard Transformers or vLLM. It requires the custom SpectralQ inference engine, which includes the C++ SRAM bit-unpacking and IDCT projection kernels.
To run this model, please visit the GitHub repository: [馃憠 SpectralQ 馃憟]
- Downloads last month
- 35
Inference Providers NEW
This model isn't deployed by any Inference Provider. 馃檵 Ask for provider support
Model tree for AUser0/Qwen2.5-3B-SpectralQ-6bit
Base model
Qwen/Qwen2.5-3B