dharun2049/MAE-Qwen3.8-27B

MAE

MAE (Magnitude-Adaptive Encoding) is an experimental five-state / quinary post-training quantization format for large language models.

This repository contains a PQ5-HQ checkpoint derived from:

Qwen/Qwen3.8-27B

What MAE is

MAE stores quantized weights using the codebook

[-alpha, -1, 0, +1, +alpha]

and reconstructs a quantized group as

W_hat_g = scale_g * Q_g

where:

• alpha is selected per quantized tensor
• scale_g is an FP16 scale shared by a weight group
• the five states are packed using base-5 coding
• three quinary digits fit in seven bits because 5^3 = 125 < 128

The raw code density is therefore:

7 / 3 = 2.3333 bits per weight

before group-scale overhead.

For example:

• group size 128 -> about 2.4583 bpw
• group size 64 -> about 2.5833 bpw
• group size 32 -> about 2.8333 bpw

Quality-aware policy

This checkpoint was produced using a quality-first adaptive policy.

Instead of forcing every tensor into the same low-bit representation, the quantizer can try several group sizes and keep a tensor at higher precision if its reconstruction error is above the configured threshold.

Manifest description:

Quality-first five-state PTQ with adaptive group size, tensorwise learned outer magnitude, BF16 fallback for sensitive tensors, and exact base-5 packing.

Configured quality policy:

• target relative RMSE: 0.23
• hard maximum relative RMSE: 0.25
• protected boundary layers: 4

Actual checkpoint statistics

These values are read directly from pq5_manifest.json and describe this specific released checkpoint.

Metric	Value
Original tensor storage	55.563 GB
Stored tensor storage	55.475 GB
Compression ratio	1.002x
Quantized parameters	52,428,800
Passthrough storage	55.458 GB
Protected tensors	651
Quality fallbacks	434

Accepted quantized tensors

Grouping	Tensor count	Nominal quantized weight cost
g64	1	2.5833 bpw
g32	0	2.8333 bpw

Important note about this release

The MAE/PQ5 format is a custom packed checkpoint representation.

It is not currently a drop-in transformers.from_pretrained() quantization backend. The reference runtime reconstructs the packed PQ5 weights into normal BF16/FP16 tensors before standard Hugging Face inference.

A native low-memory runtime would require a fused Triton/CUDA/CPU kernel that consumes the packed five-state weights directly.

Why five states?

Ternary quantization uses three states:

[-1, 0, +1]

MAE uses five:

[-alpha, -1, 0, +1, +alpha]

This introduces a second non-zero magnitude while still using fewer states than a conventional 3-bit integer representation with eight states.

The experimental question is whether this middle point can provide a useful accuracy/compression tradeoff between ternary and conventional INT3-style quantization.

Novelty positioning

Five-valued neural-network quantization itself is not claimed to be new.

MAE is an experimental LLM-oriented combination of:

• a symmetric five-state scalar codebook
• tensor-adaptive outer magnitude
• groupwise scaling
• dense radix-5 packing
• quality-aware adaptive group sizing
• high-precision fallback for sensitive tensors

Files

Typical repository contents:

config.json
tokenizer*
model-pq5-*.safetensors
model.pq5.index.json
pq5_manifest.json
README.md

Evaluation

Formal benchmark results may be added separately.

This release should not be interpreted as having preserved the full-precision model's benchmark scores until those benchmarks are actually run.

Release

Published: 2026-09-19

Repository: dharun2049/MAE-Qwen3.8-27B