EvoQuality-IQA-GGUF

GGUF quants of ByteDance/EvoQuality for fast local NR-IQA scoring with llama.cpp. Six bench-validated variants (BF16, Q8_0, Q6_K, Q5_K_M, Q4_K_M, IQ4_XS) plus the shared mmproj-f16 vision sidecar.

EvoQuality is an 8B Qwen2.5-VL-7B-based no-reference image quality assessment model. It scores a single image on a 1-5 perceptual scale (1 = severely degraded, 5 = clean, sharp, artifact-free) and supports pairwise A-vs-B comparison. Upstream reports PLCC +25% and SRCC +27% over the Qwen2.5-VL-7B baseline.

This is the first community GGUF release for EvoQuality. The companion repo with conversion scripts, the bench harness, and the Docker harness is at DoradusResearch/EvoQuality-IQA-GGUF. Methodology and use-case context are in the release blog post.

Quant lineup

Bench: 99 images from the AGIQA-3K test split, stratified across the full MOS range. Each variant served via llama-server with the shared mmproj-f16 sidecar. temperature=0. Per-image scores in bench-*.jsonl, aggregated in benchmarks-summary.json.

Variant	File size	PLCC vs MOS	SRCC vs MOS	Δ PLCC vs BF16	Recommendation
BF16	14.19 GB	0.8033	0.7177	reference	Highest fidelity
Q8_0	7.54 GB	0.8037	0.7183	+0.05%	Use if ≥12 GB VRAM. Within bench noise of BF16.
Q6_K	5.82 GB	0.7924	0.7124	-1.35%	Q5_K_M is a better Pareto point.
Q5_K_M	5.07 GB	0.7999	0.7158	-0.43%	Production sweet spot. Best quality-per-byte.
Q4_K_M	4.36 GB	0.7940	0.7138	-1.16%	Compact. Fits ~8 GB VRAM.
IQ4_XS	3.96 GB	0.7738	0.7029	-3.68%	Smallest. Measurable but bounded loss.

All six variants exceed the Qwen2.5-VL-7B baseline (~0.615 PLCC) that EvoQuality is trained to lift.

Per-variant scatter

Overlay

VRAM guide

Your VRAM	Recommended	Resident (weights + mmproj + KV @ 8K × 4)
≥ 16 GB	Q8_0	~10-12 GB
12 GB	Q5_K_M	~8 GB
8 GB	Q4_K_M	~6.5 GB
6 GB	IQ4_XS	~6 GB (tight)
Reference	BF16	~18 GB

All quants share the same mmproj-f16 vision encoder (1.35 GB). Vision-tower quantization had a substantially larger quality impact than language-tower quantization in our internal tests; mmproj is kept at f16 across the spread.

Quickstart — llama-server

llama-server \
  --model    evoquality-iqa-Q5_K_M.gguf \
  --mmproj   mmproj-evoquality-iqa-f16.gguf \
  --host 0.0.0.0 --port 8259 \
  --ctx-size 8192 \
  --n-gpu-layers 999 \
  --parallel 4 --cont-batching \
  --flash-attn on \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --alias evoquality-iqa

Score an image:

curl -s http://localhost:8259/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "evoquality-iqa",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "image_url", "image_url": {"url": "https://example.com/image.png"}},
        {"type": "text", "text": "Rate the perceptual quality of this image on a scale from 1 to 5, where 1=severely degraded (heavy noise, compression artifacts, blur, broken content) and 5=high quality (clean, sharp, artifact-free). Respond with a single integer 1-5 followed by a brief justification."}
      ]
    }],
    "max_tokens": 64,
    "temperature": 0
  }'

Response:

{"choices":[{"message":{"content":"3\nThe image is moderately sharp but has visible compression artifacts and slight aliasing on diagonal edges."}}]}

Python

from huggingface_hub import hf_hub_download
gguf  = hf_hub_download("Doradus-AI/EvoQuality-IQA-GGUF", "evoquality-iqa-Q5_K_M.gguf")
mmp   = hf_hub_download("Doradus-AI/EvoQuality-IQA-GGUF", "mmproj-evoquality-iqa-f16.gguf")

Reproducing the build

Full pipeline in DoradusResearch/EvoQuality-IQA-GGUF. Conversion is vanilla llama.cpp:

huggingface-cli download ByteDance/EvoQuality --local-dir ./hf
python3 llama.cpp/convert_hf_to_gguf.py ./hf --outfile ./evoquality-iqa-BF16.gguf --outtype bf16
python3 llama.cpp/convert_hf_to_gguf.py ./hf --outfile ./evoquality-iqa-Q8_0.gguf --outtype q8_0
python3 llama.cpp/convert_hf_to_gguf.py ./hf --mmproj --outfile ./mmproj-evoquality-iqa-f16.gguf --outtype f16
for Q in Q6_K Q5_K_M Q4_K_M IQ4_XS; do
  llama.cpp/build/bin/llama-quantize ./evoquality-iqa-BF16.gguf ./evoquality-iqa-$Q.gguf $Q 8
done

llama.cpp commit verified working: b9010-d05fe1d7d (June 2026). Older revisions can mishandle Qwen2.5-VL mmproj.

Bench methodology

• Dataset: strawhat/agiqa-3k test split (598 images, mos_quality labels).
• Sample: 100 images, evenly spaced across the full MOS range (0.00 → 4.37). Effective n = 99.
• Prompt: identical 1-5 rating template across every variant. temperature=0, max_tokens=64. Three retries with exponential backoff.
• Metrics: PLCC = scipy.stats.pearsonr(MOS, predicted_score). SRCC = scipy.stats.spearmanr(MOS, predicted_score) (tie-corrected ranks).

Per-image score data ships in this repo: bench-BF16.jsonl and the five quant variants. Aggregated summary at benchmarks-summary.json.

License

Apache-2.0, inherited from upstream ByteDance/EvoQuality.

Citation

@misc{bytedance2025evoquality,
  title  = {EvoQuality: Self-Evolving Vision-Language Model for No-Reference Image Quality Assessment},
  author = {ByteDance},
  year   = {2025},
  url    = {https://huggingface.co/ByteDance/EvoQuality}
}
@misc{agiqa3k,
  title  = {AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment},
  author = {Li, Chunyi et al.},
  year   = {2023},
  url    = {https://huggingface.co/papers/2306.04717}
}