chjdl1816/designcoder-mc

DesignCoder 4B — AdamW SFT, step 12588

This repository contains the merged BF16 model from a locally trained Qwen3.5-4B LoRA SFT checkpoint, qwen35-4b-design/lora-r64-sft/checkpoint-12588. The root directory is a complete Hugging Face model export; loading it does not require PEFT or a separate base-model download.

This is the AdamW checkpoint selected for DesignCoder RL initialization. It is an SFT model, not an RL-trained checkpoint. It replaces the previously uploaded Muon step1900 model on the main branch; that version remains in repository history.

Source and training

Item	Value
Base	Qwen/Qwen3.5-4B
Base revision	851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a
Optimizer	adamw_torch
Checkpoint	step 12588, 2 epochs
Dataset identifier	design_sft_sharegpt_with_think
LoRA	rank 64, alpha 128, dropout 0.05
Learning rate	1e-4, cosine schedule, warmup ratio 0.03
Training batch	1 per device, 8 accumulation steps, effective batch 8
Training cutoff	16384 tokens
Export	FP32 LoRA merge with safe_merge=True, then BF16 safetensors

The tokenizer, processor configuration and chat template are copied from the selected training checkpoint. The vocabulary matches the base model. Standard Transformers loading omits the base checkpoint's auxiliary MTP tensors, which are not used by Qwen3_5ForConditionalGeneration.

The original adapter weights are also included under adapter/. Their SHA256 is 7d8305739df470c3d7906241a14ae37b342e1ba61f34b3d0341e69b97f85f12b. The adapter config uses the public base-model ID and pinned revision instead of the original local filesystem path. Do not apply this adapter to the already merged model at the repository root.

Load the merged model

The export was checked with Transformers 5.8.0, PEFT 0.18.1 and PyTorch 2.6.0.

from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration

model_id = "chjdl1816/designcoder-mc"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
    model_id, dtype="auto", device_map="auto"
)
messages = [{"role": "user", "content": "Build a minimal portfolio homepage."}]
text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=True
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=1024)
print(tokenizer.decode(output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))

For reproducible training, pin the repository commit when downloading.

Validation and artifacts

• All source adapter tensors were compared against their loaded values.
• FP32 inference before and after the LoRA merge was compared on a short text prompt.
• The BF16 export was reloaded and its logits were checked for exact equality with the in-memory BF16 model on that prompt.
• Tokenizer, processor and model configuration loading were checked.
• merge_validation.json contains the numerical results; training_summary.json records training metadata; upload_manifest.json records source and artifact hashes.

These checks validate export integrity. No new benchmark, GPU inference or vLLM end-to-end RL evaluation was performed for this upload.

License

Apache-2.0, following the Qwen3.5-4B base model. See LICENSE.