supra-50m-instruct-8bit-mlx

MLX quantization of SupraLabs/Supra-50M-Instruct for Apple Silicon.

Variant: Affine int8
Disk size: 55 MB
Quantized by: sahilchachra

Benchmark results

Evaluated on Apple M4 Pro with MLX. Model loaded once; performance and quality measured in a single pass.

Performance

	This model	FP16 baseline
Decode tok/s (avg, long traces)	1157.12	1270.13
Peak memory (GB)	0.154	0.223
Disk size (MB)	55	201

Quality

Benchmark	This model	FP16 baseline	n
IFEval (instruction following)	13.6%	15.9%	44
Alpaca-cleaned (instruct F1 vs reference)	37.5	36.2	50

Context scaling (decode tok/s)

Context length	Decode tok/s
~128 tokens	1202.7
~256 tokens	1139.3
~512 tokens	1129.9
~1024 tokens	1156.6

Usage

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("sahilchachra/supra-50m-instruct-8bit-mlx")
response = generate(model, tokenizer, prompt="Your prompt here", max_tokens=256, verbose=True)

All variants in this collection

Model	Variant
sahilchachra/supra-50m-instruct-8bit-mlx	Affine int8 ← this model
sahilchachra/supra-50m-instruct-optiq-5bpw-mlx	OptiQ mixed-precision (target 5.0 bpw)

Notes

• Requires Apple Silicon (M1 or later) with MLX
• Benchmarks run on Apple M4 Pro, 24 GB unified memory
• License: see SupraLabs/Supra-50M-Instruct for the original model's license

Original model

See SupraLabs/Supra-50M-Instruct for full model details and intended use.