Model Card for Qwen3.5-2B (4-bit GGUF, with vision)

A 4-bit GGUF build of Qwen3.5-2B packaged to run entirely on device, with the vision tower, the tokenizer, the merge table, the chat template and the sampling card all embedded in the single weight file. Nothing calls out to a server at inference time and there are no sibling files to keep in sync.

This is the small end of the family, sized for a 4 GB phone: it runs with the vision tower in about 155 MB of resident memory, because the weights stay memory-mapped.

This repository holds the quantized weights only, not training data or the original checkpoint.

File format

.ggxf is GGUF plus the tensors and metadata keys this project adds, so these files load in Gadeon and are not expected to load in other GGUF readers.

Files

File	Purpose
Qwen3.5-2B-IQ4_NL.ggxf	everything: weights, vision tower, tokenizer, chat template, sampling card

One file, nothing beside it. The vision tower travels inside the GGUF rather than as a separate mmproj, and the sampling parameters travel with it, so a runtime never has to guess them or borrow another model's settings.

Tensor data is aligned to 16 KB (general.alignment), which is the page size on current iPhones, so every tensor starts on a page boundary and mapping one never drags in a neighbour's partial page.

Provenance

The language trunk is unsloth's IQ4_NL quantization of Qwen3.5-2B, with one deliberate change. unsloth publishes two IQ4_NL files for this model and they are NOT the same recipe: the one in the MTP repository spends five bits on attention QKV (Q5_K) and keeps the Gated DeltaNet gates (ssm_alpha, ssm_beta) in F32, where the plain repository spends four bits and Q8_0. Measured against a BF16 reference of the same weights, that is worth about 8 percent of KL divergence.

This build therefore takes the plain repository's file and substitutes those 54 tensors from the MTP repository's, which recovers the better fidelity without carrying the multi-token-prediction block itself. The result is smaller than either published file.

What this build adds on top is the vision tower and the metadata: the v.* / mm.* tensors and clip.* configuration from that repository's mmproj-F16.gguf, and the generation config with its sampling matrix and its stop-token list. The tower is carried at source precision (F16/F32).

This build carries no MTP drafter, by choice. A draft head only pays when the verify pass is cheap relative to the weights it re-reads, and at 4 bits on Apple silicon it does not: measured on an M3, speculation on this model runs at 0.71x of plain decode at IQ4_NL and 1.47x at BF16. The block would have cost 70 MB to carry and would never have been called.

Model Details

Model Description

Qwen3.5-2B is a dense hybrid. Its layers alternate three Gated DeltaNet linear-attention blocks to one full attention block. The linear blocks carry a small fixed-size recurrent state instead of a growing key/value cache, so memory stays flat as context grows, while the periodic attention blocks preserve exact long-range recall.

This build is multimodal. The vision tower is included, so image input works from this file alone.

• Developed by: Qwen, Alibaba (base model); quantization by unsloth; packaging by leok7v
• Model type: Hybrid Gated DeltaNet + attention causal language model with a vision encoder
• Language(s): English and the languages of the base model
• License: Apache 2.0, inherited from the base model
• Quantized from model: Qwen/Qwen3.5-2B

Model Sources

• Repository (this build): https://huggingface.co/leok7v/Qwen3.5-2B
• Base model: Qwen/Qwen3.5-2B
• Trunk quantization: unsloth/Qwen3.5-2B-GGUF and unsloth/Qwen3.5-2B-MTP-GGUF

Uses

Direct Use

On-device chat, summarization, drafting, question answering over supplied text, and description of supplied images, in offline or privacy-sensitive settings where prompts must not leave the device.

Out-of-Scope Use

Anything the base model's licence and card exclude. This build adds no alignment, no safety tuning and no filtering of its own; it is the upstream model in a different numeric format. Not a basis for legal, medical, financial or safety decisions, and not intended for high-stakes automated decisions without human review.

Bias, Risks, and Limitations

The base model's biases and failure modes carry over unchanged.

Four bits is lossy, and this is a 2B model: it is the least capable build in this family and the one most likely to be confidently wrong. Outputs will not match the upstream model token for token, and the gap is widest where the model was already uncertain. Evaluate on your own task rather than assuming it behaves like the full-precision model.

Recommendations

Treat generations as drafts to verify, not as fact. For anything where accuracy matters, and for arithmetic in particular, check the result.

How to Get Started with the Model

The tokenizer, the chat template and the sampling card are embedded, so a runtime reads all of them from the GGUF. The chat template documents the roles, the thinking block and the tool-call format.

Generation stops on either <|im_end|> (248046) or <|endoftext|> (248044). Both ids are carried in the embedded generation config and in tokenizer.ggml.eos_token_ids; a runtime that reads only the scalar tokenizer.ggml.eos_token_id will see <|im_end|> alone.

Qwen publishes no generation_config.json for this model, so the embedded sampling card transcribes the four rows the model card states: temperature 1.0 with top_p 0.95 for thinking, temperature 0.7 with top_p 0.8 for non-thinking and for vision, top_k 20 and presence_penalty 1.5 throughout.

Technical Specifications

Model Architecture and Objective

Property	Value
Objective	Autoregressive next-token prediction
Hidden size	2048
Layers	24 (Gated DeltaNet + attention, three to one)
Attention heads	8 query / 2 key-value
Head dimension	256 (partial rotary, 64)
Linear-attention state	128, 16 groups, inner size 2048
Feed-forward	6144
Vocabulary	248320
Context length	up to 262144 tokens
Vision encoder	24 blocks, 768px tiles, 16px patches, 2x2 spatial merge

Weight format

1.78 GiB, 618 tensors (298 of them the vision tower at F16/F32), GGUF v3, 16 KB tensor alignment. The trunk is a MIXED-precision assignment rather than one block type throughout: the widths are chosen per tensor, so the most sensitive rows keep more bits than the bulk of the model.

Citation

Please cite the upstream Qwen3.5-2B model. See the base model card at Qwen/Qwen3.5-2B for the canonical reference and license.

Model Card Contact

leok7v on Hugging Face.