VisionPsy-Nano-460M-GGUFs

VisionPsy-Nano-460M-GGUFs provides GGUF weights of VisionPsy-Nano-460M for fast, fully on-device multimodal inference with our patched llama.cpp fork (not stock upstream). Requires the patched llama.cpp. VisionPsy-Nano-460M is a compact (~460M parameter) vision-language model that compares favourably with other models in its ~0.5B weight class on 16 of 17 public benchmarks; these GGUF builds bring that quality to phones, tablets, and edge hardware.

We release some GGUF variants, ranging from unquantized FP32 and BF16 references to lossless 8-bit, high-quality 5-bit, and ultra-compact 3-bit formats. This range supports deployment across diverse hardware settings, from high-end workstations to resource-constrained on-device environments. All sub-8-bit variants (except the legacy Q4_0) use importance-matrix (imatrix) calibration, which consistently reduces quality degradation.

Each GGUF is a vision-language model: alongside the text weights (*.gguf) you also need the multimodal projector (mmproj-*.gguf, the SigLIP2 vision encoder) to run image inputs. See Usage.

Developed by	Tether AI Research*
Model type	Vision-language model (image-text-to-text), GGUF quantized
Base model	VisionPsy-Nano-460M
Architecture	nanoVLM — SigLIP2 vision encoder + SmolLM2-360M language backbone
Language	English
License	Apache 2.0
Quantization tool	llama.cpp
Calibration	importance-matrix (imatrix) for all -imat variants
All variants	VisionPsy-Nano-460M · VisionPsy-Nano-460M-Flash · VisionPsy-Nano-460M-GGUFs · VisionPsy-Nano-460M-Flash-GGUFs

^* References to Tether AI Research are references to Tether Data, S.A. de C.V.

---

Available Files

All published GGUF files are generated with llama.cpp. Quantization labels refer exclusively to the language-model weights, while the vision encoder and multimodal projector are fixed to Q8_0 for all variants. FP32 and BF16 are unquantized language-model references evaluated within the same inference harness. Q8_0 does not use importance-matrix (imatrix) calibration, as it provides no measurable benefit at 8-bit precision. Q4_0 is the legacy 4-bit format without imatrix calibration and is retained mainly for cross-model on-device latency comparisons. All remaining sub-8-bit language-model variants use imatrix calibration.

Δ AVG below is measured against the FP32 GGUF baseline using our Normalized (0-100) score, the mean of 17 public VLM benchmarks (see Benchmarks). Δ AVG (pts) is the change in points; Δ AVG (rel %) normalizes by the baseline.

File	Format	Imatrix	Size (MB)	Δ AVG (pts)	Δ AVG (rel %)	Recommended For
visionpsy-nano-460m-fp32.gguf	FP32	n/a	1562	≈0.00	≈0.00%	Unquantized GGUF reference
visionpsy-nano-460m-bf16.gguf	BF16	n/a	782	≈0.09	-0.15%	Unquantized GGUF
visionpsy-nano-460m-q8_0.gguf	Q8_0	no (not needed)	416	≈0.00	≈0.00%	Best quality, lossless
visionpsy-nano-460m-q5_k_m-imat.gguf	Q5_K_M	yes	310	−0.59	−0.96%	5-bit with imatrix (prefer the non -imat build)
visionpsy-nano-460m-q5_k_m.gguf	Q5_K_M	no	310	−0.12	−0.19%	Recommended high-quality 5-bit
visionpsy-nano-460m-q4_k_m-imat.gguf	Q4_K_M	yes	289	−0.24	−0.39%	Recommended for mobile/laptop (best size/quality)
visionpsy-nano-460m-iq4_xs-imat.gguf	IQ4_XS	yes	242	−0.92	−1.49%	Smaller 4-bit alternative
visionpsy-nano-460m-iq4_nl-imat.gguf	IQ4_NL	yes	244	−0.96	−1.55%	Alternative 4-bit
visionpsy-nano-460m-q4_0.gguf	Q4_0	no	244	−2.85	−4.62%	⚠ Legacy 4-bit (no imatrix) — used for latency benchmarks; prefer Q4_K_M
visionpsy-nano-460m-iq3_m-imat.gguf	IQ3_M	yes	240	−1.36	−2.20%	Strong compact 3-bit for tight memory
visionpsy-nano-460m-iq3_xxs-imat.gguf	IQ3_XXS	yes	230	−1.70	−2.75%	Ultra-compact 3-bit (accept some quality loss)
mmproj-visionpsy-nano-460m-q8.gguf	Q_8 (vision)	n/a	104	n/a	n/a	Required multimodal projector for all files above

---

Quick Recommendation

Your constraint	Choose
You want a llama.cpp-native unquantized FP32 file	FP32 — no quantization applied on LLM
You want a llama.cpp-native unquantized BF16 file	BF16 — no quantization applied on LLM
You want the best quality at half the size	Q8_0 — lossless
You want extra quality headroom over 4-bit	Q5_K_M — only −0.12 pts (−0.19%)
You want the best size/quality trade-off (most users)	Q4_K_M (imatrix) — −0.24 pts (−0.39%), fits high-end mobile/laptop
You need a smaller 4-bit file	IQ4_XS (imatrix) — −0.92 pts (−1.49%)
You need the smallest recommended file	IQ3_M (imatrix) — −1.36 pts (−2.20%)
You are cross-benchmarking on-device latency	Q4_0 — the legacy format used in our device tests (lower quality; prefer Q4_K_M for deployment)

---

Benchmarks

All scores are computed in-house with a single VLMEvalKit harness so every configuration is scored identically, using each benchmark's official metric (POPE = F1, MMVet = partial credit, MM-IFEval = instruction-following accuracy, MME = Perception + Reasoning points, OCRBench = Final Score /1000; accuracy otherwise). LLM-as-judge scoring uses Qwen3-27B-FP8. Each column is a different GGUF quantization / calibration configuration.

Legend: -i = importance-matrix (imatrix) calibration. MME = P+R points, OCRBench = /1000, all others accuracy/F1. Normalized = mean across the 17 benchmarks scaled to 0-100 (MME /28, OCRBench /1000). Weights = on-disk LM GGUF only (excludes shared mmproj Q8_0, ~104 MB).

Benchmark	FP32	BF16	Q8_0	Q5_K_M	Q5_K_M-i	Q4_0	IQ4_XS-i	IQ4_NL-i	Q4_K_M-i	IQ3_XXS-i	IQ3_M-i
Weights	1562 MB	782 MB	416 MB	310 MB	310 MB	244 MB	242 MB	244 MB	289 MB	230 MB	240 MB
MMStar	46.2	45.9	46.1	46.7	45.6	41.7	46.3	46.8	45.9	46.5	45.9
MMBench	62.1	62.2	61.8	60.3	60.9	56.1	59.3	59.3	60.7	58.8	60.9
RealWorldQA	59.7	59.6	59.1	60.5	59.9	60.3	57.9	58.0	60.3	59.1	58.7
MME	1541.3	1542.9	1553.5	1569.9	1548.7	1553.3	1553.4	1548.0	1529.6	1571.2	1532.2
SEEDBench	68.9	68.9	68.9	68.2	68.6	67.7	67.6	67.8	68.6	67.0	67.3
POPE	87.9	87.8	87.7	87.4	87.7	87.4	87.4	87.8	87.7	87.0	87.1
MMMU	32.1	31.6	32.6	32.1	33.0	31.3	30.4	30.2	31.8	31.7	31.4
MathVista *	47.5	47.5	47.8	47.4	45.9	41.8	46.6	46.3	47.6	44.8	48.3
AI2D	66.0	65.6	65.3	65.5	65.7	64.2	65.9	65.7	65.4	65.9	65.5
ScienceQA	84.7	84.7	84.7	84.2	83.9	80.8	82.9	82.8	84.5	81.1	82.5
OCRBench *	763	765	764	756	747	725	740	747	765	723	736
ChartQA *	77.2	77.4	77.5	77.1	77.0	75.4	75.8	75.9	77.0	75.0	75.9
TextVQA *	79.5	79.6	79.1	78.7	79.1	76.7	78.4	78.2	79.3	76.4	77.5
DocVQA *	83.5	83.5	83.7	83.3	83.5	80.8	82.7	82.5	83.1	82.1	82.4
InfoVQA *	48.1	48.0	47.8	46.8	46.5	44.2	47.0	46.8	46.6	45.8	46.2
MM-IFEval	43.1	42.6	43.5	45.0	41.6	34.3	43.7	42.9	44.2	37.9	38.9
MMVet *	32.0	31.9	32.3	32.7	31.0	30.6	32.8	32.3	31.8	33.4	29.9
Normalized	61.75	61.66	61.75	61.63	61.16	58.9	60.83	60.79	61.51	60.05	60.39

* ChartQA, TextVQA, DocVQA, InfoVQA, OCRBench, MathVista, and MMVet use an LLM-as-judge Qwen3-27B-FP8, which evaluates free-form answers more reliably than strict string matching (e.g. "12%" vs "12 percent", paraphrases, units, formatting).

---

Key Findings

• Q8_0 is effectively lossless: no difference on the Normalized score, at roughly half the size of BF16, with no imatrix needed.
• Q5_K_M is the recommended high-quality option: only −0.12 pts (−0.19%), and it is also better than Q5_K_M with imatrix (−0.59 pts).
• Q4_K_M with imatrix is the sweet spot: −0.24 pts (−0.39%) for a substantial size reduction, the best size/quality choice for mobile/laptop.
• imatrix beats legacy Q4_0 decisively: the imatrix 3-bit IQ3_M (−1.36 pts) and even IQ3_XXS (−1.70 pts) both outperform the legacy 4-bit Q4_0 (−2.85 pts). If you are not latency-benchmarking, there is no reason to prefer Q4_0 over an imatrix build.
• Compact 3-bit stays strong: IQ3_M holds a Normalized score of 60.39, making it a viable option for the tightest memory budgets.

---

Usage

These are multimodal GGUF files: you need both the text model (*.gguf) and the vision projector (mmproj-*.gguf).

llama.cpp

# Download the recommended text weights (Q4_K_M with imatrix) + the vision projector
huggingface-cli download qvac/VisionPsy-Nano-460M-GGUFs \
    visionpsy-nano-460m-q4_k_m-imat.gguf \
    mmproj-visionpsy-nano-460m-q8.gguf \
    --local-dir .

First build the patched llama.cpp — see the build instructions in llama-cpp-inference, then run multimodal inference:

# Run multimodal inference (image + prompt)
./llama-mtmd-cli \
    -m visionpsy-nano-460m-q4_k_m-imat.gguf \
    --mmproj mmproj-visionpsy-nano-460m-q8.gguf \
    --image your_image.jpg \
    -p "What is in this image?" \
    -n 128

# Or start an OpenAI-compatible server with a web UI
./llama-server \
    -m visionpsy-nano-460m-q4_k_m-imat.gguf \
    --mmproj mmproj-visionpsy-nano-460m-q8.gguf

---

Intended use

VisionPsy-Nano-460M-GGUFs targets latency- and memory-constrained, on-device multimodal applications: visual question answering, document/chart/diagram understanding, scene-text reading, and lightweight visual instruction following on phones and other edge hardware. Because of its small size, we recommend fine-tuning on your specific domain to maximize quality.

Limitations

• Single-image by design: the model is trained and optimized for one image per query; multi-image prompts are outside its intended use.
• As a compact model, it may occasionally hallucinate or miscount and is best suited to focused tasks rather than very dense documents or long multi-step math, where larger models have an edge.
• Quantization artifacts: lower bit counts can subtly degrade outputs in ways aggregate benchmarks do not fully capture. We recommend Q4_K_M or higher for production and prefer the imatrix builds over the legacy Q4_0.
• Primarily English; other languages are not officially supported yet.
• Not intended for safety-critical or high-stakes automated decisions.
• Benchmark scores are produced with a fixed in-house harness and an LLM judge (Qwen3-27B-FP8); absolute numbers may differ from other reported setups.

Acknowledgements

Built on the excellent open-source work of nanoVLM, SmolLM2, SigLIP2, and llama.cpp.

Citation

@misc{visionpsynano2026,
title = {VisionPsy-Nano-460M: A Compact Vision-Language Model for On-Device Inference},
author = {Tether AI Research},
year = {2026},
note = {Hugging Face model card}
}

Copyright

We will take appropriate actions in response to notices of copyright infringement. If you believe your work has been used or copied in a manner that infringes upon your intellectual property rights, please email data-apps@tether.io identifying and describing both the copyrighted work and alleged infringing content.

Licensing

This model, which was finetuned as described in the blog post, is licensed by Tether Data, S.A. de C.V. under the Apache 2.0 license. As described in the blog post, this model is a version of the NanoVLM-460M-8k pre-trained model (https://huggingface.co/lusxvr/nanoVLM-460M-8k), which is made available under the MIT license.

The FineVision dataset (https://huggingface.co/datasets/HuggingFaceM4/FineVision) is made available under the CC-BY-4.0 (Creative Commons - Attribution 4.0) license. FineVision is an aggregation of a number of public sources unified into a single corpus. Individual subsets within the collection may inherit specific underlying terms from their original creators. As described in the blog post, a subset of the FineVision dataset was used as a part of finetuning the model.

The NVIDIA Nemotron-Image-Training-v3 dataset (https://huggingface.co/datasets/nvidia/Nemotron-Image-Training-v3) is made available under the CC-BY-4.0 (Creative Commons - Attribution 4.0). The mPLUG TinyChartData dataset (https://huggingface.co/datasets/mPLUG/TinyChartData) is made available under the Apache 2.0 license. The TabMWP dataset (https://promptpg.github.io/) is made available under the CC BY-NC-SA 4.0 (Creative-Commons-Attribution-NonCommercial-ShareAlike 4.0). The PopVQA dataset (https://huggingface.co/datasets/idoco/PopVQA) is made available under the MIT license. The InfoSeek dataset (https://github.com/open-vision-language/infoseek) is made available under the Apache 2.0 license. The MMKU-Bench dataset (https://huggingface.co/datasets/baochenfu/MMKU-Bench) is made available under the Apache 2.0 license. The VisionFoundry-10K dataset (https://huggingface.co/datasets/zlab-princeton/VisionFoundry-10K) is made available under the Apache 2.0 license. The PKU-SafeRLHF-V dataset (https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF-V) is made available under the CC-BY-NC 4.0 (Attribution-NonCommercial 4.0 International). As described in the blog post, the NVIDIA Nemotron-Image-Training-v3, mPLUG TinyChartData, TabMWP, PopVQA, InfoSeek, MMKU-Bench, VisionFoundry-10K and PKU-SafeRLHF-V datasets were used as a part of finetuning the model.