scottlowry/Qwen3-VL-Embedding-8B-omlx

Qwen3-VL-Embedding-8B (oMLX-ready, lm_head patch applied)

This is the official Qwen/Qwen3-VL-Embedding-8B checkpoint (BF16, 4 original shards, unchanged) plus a small repair that makes it loadable by MLX-based tooling (oMLX / mlx-vlm / mlx-embeddings). The original checkpoint fails to load in those runtimes with:

missing 1 parameters language_model.lm_head.weight
reload models after fixing the files to retry

This repository is the fixed source. A companion oQ6e quantized build made from it lives in Qwen3-VL-Embedding-8B-omlx-oQ6e.

---

What was wrong

Qwen ships the embedding variant without an lm_head. The checkpoint has 749 tensors and contains no lm_head.* key at all — the file's model.language_model.embed_tokens.weight is the only copy of the token embedding, and the final model.language_model.norm.weight feeds straight into the embedding output. That is fine for HF/transformers, where the embedding model simply never computes logits.

But mlx-vlm's Qwen3VLForConditionalGeneration unconditionally declares language_model.lm_head.weight whenever tie_word_embeddings is false — which the official config.json sets. When the checkpoint is scanned for the model tree, language_model.lm_head.weight comes up missing and loading aborts:

ValueError: Received 1 parameters not in model / missing 1 parameters
language_model.lm_head.weight

It is not a truncated or corrupted download: the shard sizes match the HuggingFace manifest exactly, and the omission is by design upstream.

The fix: one added tensor, one added shard

The repair adds exactly what the MLX model tree expects — nothing more, nothing less:

Item	Value
New file	model-00005-of-00005.safetensors
New tensor	lm_head.weight
Shape	[151936, 4096] (vocab × hidden)
Dtype	bfloat16
Values	exact copy of model.language_model.embed_tokens.weight (the tied-embedding convention: lm_head ≡ embed_tokens)
Index	model.safetensors.index.json updated (weight_map + metadata.total_size bumped by the tensor's 1,244,659,712 bytes)

The four original shards are untouched. Because the embedding model never computes logits (the embedding vector comes from last_hidden_state / pooling), the head is used only to satisfy the loader's structural check — and because it is a byte-exact copy of the embedding matrix, it is also the value any tied-embedding code would expect if the head were ever exercised.

Why the key is named lm_head.weight (not model.language_model.lm_head.weight)

mlx-vlm's Qwen3-VL sanitize() renames checkpoint keys as it loads:

if "model.language_model" in key:
    key = key.replace("model.language_model", "language_model.model")
elif "lm_head" in key:
    key = key.replace("lm_head", "language_model.lm_head")

The if branch fires on any key containing model. — so model.language_model.lm_head.weight would be rewritten to language_model.model.lm_head.weight, which is not a parameter in the model tree (verified: the first attempt with the prefixed name produced ValueError: Received 1 parameters not in model: language_model.model.lm_head.weight).

The unprefixed lm_head.weight hits the elif branch instead and maps to exactly the parameter the model declares: language_model.lm_head.weight.

Verification

• All 5 shards parse with safetensors.safe_open (750 keys total).
• mlx_vlm.utils.load() on this directory returns the Model + Qwen3VLProcessor with no missing-parameter error.
• The companion oQ6e quant (built from this fixed source in the oMLX web UI) loads and runs.

Repository layout

Qwen3-VL-Embedding-8B-omlx/            ← this repo (fixed BF16 source)
├── model-00001..00004-of-00004.safetensors   (original shards, unmodified)
├── model-00005-of-00005.safetensors          (NEW: lm_head.weight)
├── model.safetensors.index.json              (updated)
├── config.json / tokenizer / processors      (original, unmodified)
└── README.md

Quantized build: scottlowry/Qwen3-VL-Embedding-8B-omlx-oQ6e — oQe (enhanced) 4-bit, group size 64, affine, bfloat16, imatrix-calibrated (128 samples, seq len 512, oqe_code_multilingual).

License

Apache-2.0 (as the original Qwen/Qwen3-VL-Embedding-8B).

---

Author's note: this README and the accompanying model repair were written by Hermes Agent (Nous Research), backed by a locally-run DeepSeek V4 Flash model (DeepSeek-V4-Flash-0731-oQ4e-mtp). The repository owner (scottlowry) requested the work but did not author this document. The model weights themselves are Qwen's original work; only the single added lm_head.weight tensor and this writeup are machine-generated modifications.