Upload folder using huggingface_hub

README.md

@@ -38,12 +38,19 @@ ONNX export of [LFM2.5-350M](https://huggingface.co/LiquidAI/LFM2.5-350M) for cr
 | Variant | Size | Description |
 |---------|------|-------------|
 | FP16 | ~692MB | All weights in FP16 |
-| Q4 | ~298MB | INT4 MatMul weights (including lm_head), FP16 embeddings and norms |
-| Q4F16 | ~298MB | INT4 MatMul weights, FP16 lm_head/embeddings and norms |
+| Q4 | ~276MB | INT4 embedding (GatherBlockQuantized), INT4 lm_head (MatMulNBits, shared), INT4 MatMul weights |
+| Q4F32 | ~459MB | INT4 MatMul weights, FP32 embedding and norms |
+| Q8 | ~604MB | INT8 MatMul weights, FP32 embedding and norms |
 
-Q4 and Q4F16 use symmetric block-wise quantization (block_size=32) via MatMulNBits.
-The difference is that Q4 also quantizes the lm_head projection to INT4, while Q4F16
-keeps it in FP16. Both store the token embedding (Gather) and normalization weights in FP16.
+Q4 uses GatherBlockQuantized for the token embedding and MatMulNBits for the lm_head,
+reusing the same quantized weights and scales. All other linear layers are quantized to
+INT4 via post-export MatMulNBitsQuantizer. Block size is 32.
+
+Q4F32 keeps the embedding as a FP32 Gather and the lm_head as FP32 Transpose + MatMul.
+Only the internal linear layers (attention projections, conv projections, MLP) are
+quantized to INT4 via post-export MatMulNBitsQuantizer.
+
+Q8 is the same structure as Q4F32 but with INT8 weights (asymmetric quantization).
 
 ## Model Files
 
@@ -52,7 +59,8 @@ onnx/
 ├── model.onnx              # FP32
 ├── model_fp16.onnx         # FP16
 ├── model_q4.onnx           # Q4
-└── model_q4f16.onnx        # Q4F16
+├── model_q4f32.onnx        # Q4F32
+└── model_q8.onnx           # Q8
 ```
 
 ## Python