Jack04810/Spac-Qwen3-14B-Spatial-PE-0.008

output_14b_phase2_8ep_0.008pe — SPAC Qwen3-14B Spatial-PE (merged fp32)

Qwen3-14B with the SPAC extended vocabulary (200,824 tokens: base BPE + spatial tokens), full fine-tuned for 3D mesh understanding & generation. Weights merged from the DeepSpeed ZeRO-3 checkpoint-18892 of output_14b_phase2_8ep_0.008pe into 13 fp32 pytorch_model-*.bin shards (pytorch_model.bin.index.json), i.e. the same layout as the previous release.

Relation to previous releases

• Phase-2 run output_14b_phase2 (step 34268, spatial-PE scale 0.08) — the former root release.
• This release: continuation from output_14b_phase2_full trained with spatial-PE scale 0.008, LR 1e-5 -> 1e-6 (cosine w/ min-LR, warmup 0.5%), 18,892 steps on bpe_160k_mix (160k samples) + ultrachat_8k (8k conversations), 8x A800, ZeRO-3 bf16, FA2, packing 8192.
• Trainer-reported results: train loss 0.3348; eval loss 0.7694, eval accuracy 0.8356.

Spatial-PE parameters (spatial_pe_manifest_14b.json, this repo)

• mode: absolute_anchor_plus_footprint_ipe (npe: false)
• coordinate grid: [0, 31] (32 levels); coord_scale 32.0
• base mesh vocab size: 8192; hidden size: 5120
• training spatial-PE scale (spatial_pe_scale): 0.008 (manifest field spatial_scale: 0.08)
• spatial tokens: <mesh_start> id 151669, <mesh_end> id 151670, mesh tokens 151672..168055 (16,384), morton tokens 168056..200823 (32,768); total vocab 200,824
• merge table sha256: fb10dbe1f4b582a5c568169a167ccb98a3b49fd309c0a5cc386b17fc775705f8
• requires the SPAC spatial-PE inference wrapper + hf_main_merge_table.json (this repo) to run with the same positional encoding used in training

Inference notes

• Chat template: qwen3_nothink (no thinking). Mesh sequences tokenize as <mesh_start><morton_*><mesh_*> (see added_tokens.json).