PALLER-NOVUS/WeVisOne-VL-GGUF

WeVisOne-VL · 2B & 4B GGUF

GGUF versions of Tencent WeChat Vision's WeVisOne-VL for structured visual perception: object detection, referring, counting, pointing and OCR. Conversion, quantization and evaluation by NOVUS / PALLER-NOVUS.

Preview release: the Q4_K_M/F16 GPU pilot is complete. The full quantization benchmark and native-reference parity check are unfinished; there is no final quality ranking yet.

Download · Run locally · Benchmark · Build and reproduce

20-query pilot on RTX 2060 6 GB. Latency and VRAM only; 4B F16 used CPU offload. Full quality benchmark pending.

6 GB VRAM TEST LIMIT: 4B F16 USED CPU OFFLOAD

THE APPROXIMATELY 60 S/QUERY 4B F16 RESULT IS A CPU + GPU PILOT TIMING ON AN RTX 2060 6 GB. Its weights and vision projector alone occupy 9.01 GiB on disk, before runtime buffers and KV cache. 8 GB CAN STILL REQUIRE OFFLOAD; 12–16 GB OFFERS MORE SUITABLE HEADROOM. Larger cards have not been tested here. Measurements and settings.

Choose your files

Download one GGUF and its matching vision projector. Keep 2B files with the 2B projector, and 4B files with the 4B projector.

Start with Q4_K_M: both sizes have passed real-image generation and the pilot. Other quantizations are available for comparison; their quality ranking is pending. F16 is the comparison reference.

Format	2B	4B
Q4_K_M	1.28 GB	2.72 GB
Q5_K_M	1.47 GB	3.16 GB
Q6_K	1.68 GB	3.63 GB
Q8_0	2.17 GB	4.70 GB
IQ4_XS	1.18 GB	2.50 GB
Q3_K_M	1.08 GB	2.25 GB
Q2_K	0.88 GB	1.80 GB
F16	4.08 GB	8.84 GB

Required projectors: 2B — 0.82 GB · 4B — 0.84 GB.

Sizes are decimal GB and describe files, not VRAM consumption. Exact bytes and SHA256 checksums.

Run locally

Use the patched llama.cpp build in REPRODUCE.md. Unpatched upstream builds and desktop-app integrations have not been validated.

Download a pair, then start the server. This example uses the tested CUDA build:

hf download PALLER-NOVUS/WeVisOne-VL-GGUF \
  2B/WeVisOne-VL-2B-Q4_K_M.gguf 2B/vision-2b.gguf \
  --local-dir WeVisOne-VL-GGUF

./llama.cpp/build-cuda/bin/llama-server \
  -m WeVisOne-VL-GGUF/2B/WeVisOne-VL-2B-Q4_K_M.gguf \
  --mmproj WeVisOne-VL-GGUF/2B/vision-2b.gguf \
  --host 127.0.0.1 --port 8080 -c 4096 -t 3 -tb 3 \
  -ngl auto --fit on --fit-target 768 --split-mode none \
  --parallel 1 --special --no-warmup \
  --image-min-tokens 1024 --image-max-tokens 1960

For the CPU build, use ./llama.cpp/build/bin/llama-server and replace the GPU fitting flags with -ngl 0. For 4B, replace both model and projector paths.

Send a JPEG or PNG image from a second terminal. The build instructions download the client under release/tools/:

python release/tools/query.py --image example.jpg \
  --prompt "Locate all the instances that match the following description: dog."

Output uses structured tags and 0–1000 coordinates, for example:

<ref>dog</ref><box><12><274><400><946></box>

This illustrates the format, not an expected answer for every image. Keep special tokens when decoding; the included parser converts coordinates to pixels.

Benchmark preview

The pilot used 20 fixed queries per variant on an RTX 2060 6 GB. The full evaluation has 272 queries across five task families and is still incomplete.

These are pilot latency and memory measurements, not accuracy percentages. The 2B Q4 run has one token-limit failure; the other three runs produced 20/20 complete, parseable responses. Larger GPUs have not been measured.

Results and methodology · Pilot measurements · Frozen query IDs · Reproduce

Supported scope

This release tests structured visual perception. It does not establish broad assistant, video or multilingual performance.

Segmentation and alpha matting are unsupported in the tested GGUF interface. Tencent's decoder needs visual features and the <OBJ> hidden state; placing mask_decoder.pt beside a GGUF does not enable pixel output. Use the native Tencent package for those tasks.

Evaluation notes

• All language weights and projectors are hash-verified. All 1,001 coordinate tokens and <OBJ> ID 152678 are preserved.
• Q4_K_M real-image generation passed for both sizes. A fresh patched-source build also matched one 2B Q4 fixture; this is limited replay evidence.
• F16 compares conversion within llama.cpp. The separate native CPU reference uses FP32 and is not a speed-equivalent backend.
• The subset contains 64 detection, 64 referring, 48 counting, 48 pointing and 48 OCR queries. Failed and missing outputs stay in the scoring denominators.
• Training-data overlap is unknown; no official full-benchmark score is claimed. CountQA uses the first of 13 test shards. PR-Bench retains its noncommercial terms. Images and annotations are not redistributed.

See BENCHMARKS.md for task metrics, dataset terms and evidence boundaries.

Credits and license

Original models: Tencent 2B · Tencent 4B. Source and evaluation code.

Apache-2.0 model license; third-party notices are preserved in LICENSE and NOTICES.md. llama.cpp retains its MIT license. Nubo is NOVUS's mascot; the banner is editorial artwork, not model output. Artwork provenance.

Türkçe hızlı başlangıç

Bir GGUF ve aynı boyuta ait görsel projector dosyasını indirin. Q4_K_M iki modelde de görsel pilotta çalıştı; tam kalite sıralaması henüz hazır değil. 4B F16, 6 GB kartta CPU'ya taştı. Yaklaşık 60 sn/soru bu koşulun sonucudur. 8 GB'ta da taşma sürebilir; 12–16 GB kartlarda hız ve tam GPU yerleşimi henüz ölçülmedi. Segmentasyon ve alpha matting için Tencent'in orijinal paketini kullanın.