MAE

MAE (Magnitude-Adaptive Encoding) is an experimental low-bit language-model checkpoint using a five-state, or quinary, weight representation.

This release stores quantized weights using the PQ5 representation:

[ q_i \in \{-\alpha,-1,0,+1,+\alpha\} ]

and reconstructs each weight group as

[ \hat W_g = s_g Q_g, ]

where \(\alpha\) is selected per quantized tensor and \(s_g\) is an FP16 scale shared by a group of 128 weights.

The five code values are packed as base-5 symbols rather than wasting a full 3-bit integer for every weight. Three quinary digits fit in seven bits because .

Status: research prototype. This repository contains a custom packed PQ5 checkpoint. It is not yet a drop-in native low-bit transformers.from_pretrained() runtime. The reference inference path reconstructs the PQ5 weights to BF16/FP16 before standard Transformer inference. A fused PQ5 GPU kernel is a separate systems step.

Base model

Qwen/Qwen3-0.6B

MAE is a quantized derivative of the base model above. The quantization changes the weight representation; it does not claim to introduce a new pretrained language-model architecture.

MMLU

The score below was read automatically from the EleutherAI LM Evaluation Harness output included with this repository.

Benchmark	Setting	Accuracy
MMLU	5-shot	25.35% ± 0.37 pp

Evaluation scope: Full MMLU run

MMLU broad categories

Category	Accuracy
STEM	25.02%
Humanities	25.10%
Social Sciences	24.05%
Other	27.33%

The aggregate is taken from the official mmlu group metric emitted by lm-eval, rather than manually averaging category scores.

Evaluation artifact used when publishing: results_2026-09-19T12-28-11.526035.json.

Representation

For each quantized tensor, MAE uses the symmetric five-state codebook

[ \mathcal{Q}_\alpha = \{-\alpha,-1,0,1,\alpha\}. ]

For a weight group $W_{g}W\_g$, the reconstruction is

[ \hat W_g=s_g Q_g. ]

The reference quantizer alternates between nearest-code assignment and least-squares group-scale fitting. It searches a small candidate grid for $alpha\backslash \backslash alpha$ and chooses the value that minimizes reconstruction error for that tensor.

Packing

A quinary symbol contains $lo{g}_{2}5approx2.3219\backslash \backslash log\_2\; 5\; \backslash \backslash approx\; 2.3219$ bits of information. MAE packs three base-5 digits into one seven-bit symbol:

[ P=d_0+5d_1+25d_2, \qquad P\in[0,124]. ]

Eight seven-bit symbols are then packed into seven bytes. This gives

[ \frac{7}{3}\approx2.3333 ]

physical code bits per weight before scales.

Storage

• Nominal PQ5 code density: 7/3 = 2.3333 bits/weight for the five-state codes.
• With one FP16 scale per 128 weights, the nominal quantized-weight cost is ~2.4583 bits/weight, before small metadata/padding overhead.
• Checkpoint tensor compression ratio reported by the converter: 6.51×.
• Tensor storage: 0.231 GB vs 1.503 GB before PQ5 conversion.

Why five states?

Ternary quantization has only one non-zero magnitude:

[ \{-s,0,+s\}. ]

MAE adds a second non-zero magnitude while staying below a conventional eight-state INT3 representation:

[ 3\text{ states} < 5\text{ states} < 8\text{ states}. ]

The research question is whether this middle point can recover part of the quality lost by ternary quantization while retaining a very compact representation.

Five-valued neural-network quantization itself is not presented as a new concept. MAE is an experimental LLM-oriented combination of a five-state scalar codebook, adaptive magnitude selection, group scaling and dense radix-5 packing.

Inference

The reference runtime follows:

packed PQ5 checkpoint
        |
        v
decode five-state weights
        |
        v
reconstruct BF16/FP16 weights
        |
        v
standard Hugging Face inference

This lets the quantized checkpoint be evaluated for quality today, but it does not retain the ~2.46-bit weight footprint during GEMM. Native compressed inference requires a fused PQ5 Triton/CUDA or other low-level kernel that consumes the packed representation directly.

If tools/pq5_infer.py is present in this repository, it is the matching reference loader used for the current format.

Evaluation methodology

MMLU was evaluated with the EleutherAI Language Model Evaluation Harness using its mmlu task group. The released evaluation JSON is included under eval/mmlu/.

The benchmark configuration recorded by the publishing run:

• Few-shot examples: 5
• Aggregate MMLU accuracy: 25.35%
• Result source: groups.mmlu.acc,none
• Published: 2026-09-19

Results should be compared only against evaluations using compatible MMLU prompting, few-shot settings and harness/task versions.

Limitations

• This is an experimental custom checkpoint format.
• Native packed PQ5 matrix multiplication is not included in the reference runtime.
• Quantization can change downstream accuracy differently across tasks and model layers.
• The current codebook and alpha search are simple reference implementations rather than a globally optimized quantization objective.
• Benchmark scores describe this checkpoint and evaluation configuration, not all possible MAE/PQ5 models.

Repository contents

Typical layout:

config.json
tokenizer*
model-pq5-*.safetensors
model.pq5.index.json
pq5_manifest.json
README.md
eval/mmlu/
tools/

Name

MAE = Magnitude-Adaptive Encoding

Repository: dharun2049/MAE.