MalayKedia/qwen_uncompressed_init

Qwen3-4B-Instruct-2507 — uncompressed baseline

Verbatim, unmodified copy of Qwen/Qwen3-4B-Instruct-2507 pinned at commit cdbee75f17c01a7cc42f958dc650907174af0554.

Kept as the fixed reference point for a model-compression project: every compression ratio and accuracy delta is measured against these numbers. No weights here are changed in any way.

Verified size

Accounting is numel x dtype_itemsize summed over every tensor.

metric	value
tensors	398
parameters	4,022,468,096
bytes	8,044,936,192 (7.4924 GiB)
dtype	BF16 throughout

Verified independently on a local machine and inside Kaggle; both agree byte-for-byte with the upstream checkpoint.

Where the bytes live

group	tensors	params	GiB	share
mlp	108	2,689,597,440	5.010	66.9%
attention	144	943,718,400	1.758	23.5%
embedding	1	388,956,160	0.724	9.7%
norm	145	196,096	0.000	0.0%

Two-thirds of the model is MLP (gate/up/down_proj). The embedding is a single 389 M-parameter tensor, and tie_word_embeddings: true means it is shared with lm_head.

Architecture

36 layers, hidden 2560, intermediate 9728, GQA with 32 query / 8 KV heads, head_dim 128, vocab 151,936.

This is the non-thinking instruct variant; it does not emit <think> blocks. It is a different checkpoint from the original April-2025 Qwen3-4B.