Yida-Model-MM

Merged full-weight checkpoint of a Qwen3-14B LoRA supervised fine-tune. Hub files are merged bfloat16 weights, not a PEFT adapter.

This model is for research and engineering evaluation. It is not a medical device and must not be used as the sole basis for clinical decisions.

Model details

Item	Value
Developed by	rwang220
Model type	Causal LM (Qwen3ForCausalLM)
Parameters	14.77B (14,768,307,200)
Precision	bfloat16
Architecture	40 layers, hidden size 5120, GQA 40/8, intermediate size 17408, vocab 151936
Context	max_position_embeddings 40,960. SFT max length 8,192
License	Apache 2.0, inherited from Qwen3-14B

Training

Fine-tuned with ms-swift LoRA, then merged into full weights.

Item	Value
Immediate base	Public Qwen/Qwen3-14B.
Run	mm_sft_5_29/v0-20260529-080809
LoRA	rank 16, alpha 32, dropout 0.05, targets q/k/v/o/gate/up/down_proj
Optim	AdamW, lr 1e-4, cosine, warmup ratio 0.05
Schedule	3 epochs, 72,348 steps, max length 8192
Batch	per-device 1, gradient accumulation 8
Best eval loss	0.6525 at step 71,500
Final eval loss	0.6525
Final eval token acc	0.7803

Data

An internal, deduplicated Chinese medical instruction mix. Categories were capped; this is not the full uncapped set used by the DS variant.

How to use

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "rwang220/Yida-Model-MM"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
messages = [{"role": "user", "content": "请用通俗语言解释高血压的常见注意事项。"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=1024, temperature=0.6, top_p=0.95, top_k=20)
print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

vllm serve rwang220/Yida-Model-MM --reasoning-parser qwen3

Limitations

• Can hallucinate guidelines, doses, diagnoses, and citations.
• Training data is internal and not released with this repository.
• Reported numbers are training eval loss and token accuracy only. No clinical benchmark is claimed.