clm-project/clm-35b-a3b-text-image-head

CLM-35B-A3B text + image post-trained head

A contrastive MLP head (width 1536, depth 3, 512-d projection, 12.6M params) over Qwen3.6-35B-A3B-FP8 embeddings (2048-d), from the clm_train stage comparison (docs/clm-35b-training-eval.md).

• Init: the HQ-DQA mid-training head (HQ-DQA pre-training on all 621 chunks -> HQ-DQA + hard negatives).
• Post-training: 70 stores (sources.txt): the 67 text stores of the recipe at the time plus three image stores (Nemotron-Image v3, Nemotron-Image v3.5, Cosmos-Reason1); 100k train rows per store, 40% HQ-DQA replay with hard negatives, batch 16,384, lr 1e-3, 20k steps, options view with probability 0.5. best_head.pt = step 20,000.

Jev Decision Index, text public suite (the index's own scoring, plain states):

head	index	knowledge	language	retrieval	tools	arts
text-only post-training (same recipe without the image stores)	21.4	4.9	34.2	35.4	16.7	11.2
this head (text + image)	19.4	5.0	32.3	35.1	8.9	10.5

Not scored on the vision board (the eval covers the text suite only). The best text head is clm-project/clm-35b-a3b-jev-boost-head (39.9).

Files: best_head.pt, final_head.pt, summary.json (recipe + validation history), train.log, sources.txt. In clm_train: hf:clm-35b-a3b-text-image.