sabi-noema MLX 4-bit (Affine)
This repository contains 4-bit affine quantized weights for sabicap/sabi-noema using Apple MLX.
Quantization Details
- Framework: Apple MLX (
mlx.core.quantize) - Bits: 4-bit
- Mode: affine
- Group size: 64
- Original weight size: ~50.96 GB
- Quantized weight size: ~16.22 GB
- Linear layers quantized: 581
- Layers kept unquantized (norms, conv1d, 1D biases, embeddings): 603
Converted and uploaded by Julius (@blackfan23).
- Downloads last month
- 20
Model size
27B params
Tensor type
BF16
·
F32 ·
U32 ·
Hardware compatibility
Log In to add your hardware
Quantized
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support