Model Card for Qwen3.8-27B (2-bit GGUF)

A 2-bit GGUF build of Qwen3.8-27B, packaged to run entirely on device. The tokenizer, the merge table, the chat template and the vision tower are all embedded in the single file, so nothing ships alongside it and nothing calls out to a server at inference time.

This repository holds the quantized weights only, not training data or the original checkpoint.

This file uses a non-standard block type and stock llama.cpp will not read it. The weights are stored in a custom 2-bit format that upstream ggml does not implement, so the file is intended for a companion on-device runtime that does. If you are looking for a build that runs in llama.cpp, Ollama or LM Studio today, use one of the standard-quant GGUF releases instead.

Model Details

Model Description

Qwen3.8-27B is a dense (not mixture-of-experts) hybrid. Of its 64 layers, 48 are Gated DeltaNet linear-attention blocks and 16 are full gated-attention blocks, interleaved three-linear-one-attention. The linear blocks carry a small fixed-size recurrent state instead of a growing key/value cache, so memory stays flat as context grows, while the periodic attention blocks preserve exact long-range recall.

It is natively multimodal: a 27-layer vision tower understands images and video, and that tower is included here rather than shipped as a separate projector file. Input and output embeddings are not tied, so a separate output-projection matrix is present.

Thinking mode is on by default and can be turned off per request; reasoning depth is tunable with reasoning_effort, and reasoning from earlier turns is retained through preserve_thinking.

• Developed by: Qwen, Alibaba (base model); quantization by leok7v
• Model type: Hybrid Gated DeltaNet + attention causal language model, multimodal (image and video in, text out)
• Language(s): English and the languages of the base model
• License: Apache 2.0, inherited from the base model
• Quantized from model: Qwen/Qwen3.8-27B

Model Sources

• Repository (this build): https://huggingface.co/leok7v/Qwen3.8-27B
• Base model: Qwen/Qwen3.8-27B

Uses

Direct Use

On-device chat, summarization, drafting, question answering over supplied text, and image and video understanding, in offline or privacy-sensitive settings where prompts must not leave the device.

Out-of-Scope Use

Anything the base model's licence and card exclude. This build adds no alignment, no safety tuning and no filtering of its own; it is the upstream model in a different numeric format. Not a basis for legal, medical, financial or safety decisions, and not intended for high-stakes automated decisions without human review.

Bias, Risks, and Limitations

The base model's biases and failure modes carry over unchanged.

Two bits is aggressive, and it shows. Outputs will not match the upstream model token for token, and the gap is widest where the model was already uncertain. In practice the structure of an answer survives better than its details: a chain of reasoning can be laid out correctly and still contain an arithmetic slip, and long multi-step derivations are where errors accumulate. Occasional dropped or merged words in otherwise fluent text are normal at this width.

Anyone depending on this build should evaluate it on their own task rather than assume it behaves like the full-precision model or like a 4-bit one.

Recommendations

Treat generations as drafts to verify, not as fact. For anything where accuracy matters, and for arithmetic in particular, check the result.

How to Get Started with the Model

The tokenizer and the chat template are embedded, so a runtime reads both from the GGUF rather than from sibling files. The chat template documents the roles, the thinking block and the tool-call format.

The base model card recommends these sampling parameters, and they are the sensible starting point here:

mode	temperature	top_p	top_k	presence_penalty
Thinking	1.0	0.95	20	0.0
Instruct (non-thinking)	0.7	0.80	20	1.5

Two template defaults are worth knowing before wiring a runtime to it. preserve_thinking defaults to true, so past assistant turns re-render with their reasoning block rather than stripped; a runtime that does not retain reasoning should pass preserve_thinking=false. And reasoning_effort defaults to xhigh, which injects an instruction to reason thoroughly; low and medium are supported and cost fewer tokens per turn.

Technical Specifications

Model Architecture and Objective

Property	Value
Objective	Autoregressive next-token prediction
Hidden size	5120
Layers	64 (48 Gated DeltaNet + 16 gated attention)
Attention heads	24 query / 4 key-value (grouped-query)
Head dimension	256 (partial rotary, 64)
Linear-attention heads	48 value / 16 query-key, head dim 128
Feed-forward	17408
Vocabulary	248320, embeddings NOT tied
Context length	up to 262144 tokens
Vision tower	27 layers, width 1152, projected to 5120

Weight format

9.35 GiB, 1200 tensors, GGUF v3, 16384-byte tensor alignment.

Type	Tensors	Covers
2-bit	344	the transformer trunk, the embedding table, the output head
Q4_0	66	the feed-forward down projections, held wider on purpose
F16	112	the vision tower
F32	678	norms, layer scalars, the recurrent-state parameters

The 2-bit type is a custom ggml block, which is why stock llama.cpp cannot read the file. The feed-forward down projections are the one place where the extra width earns its size; leaving them at 2 bits is the difference between a model that answers and one that degrades partway through.

Citation

Please cite the upstream Qwen3.8-27B model. See the base model card at Qwen/Qwen3.8-27B for the canonical reference and license.

Model Card Contact

leok7v on Hugging Face.