Yida-Model-DS

Merged full-weight checkpoint of a Qwen3-14B LoRA supervised fine-tune. Hub files are merged bfloat16 weights, not a PEFT adapter.

This model is for research and engineering evaluation. It is not a medical device and must not be used as the sole basis for clinical decisions.

Model details

Item	Value
Developed by	rwang220
Model type	Causal LM (Qwen3ForCausalLM)
Parameters	14.77B (14,768,307,200)
Precision	bfloat16
Architecture	40 layers, hidden size 5120, GQA 40/8, intermediate size 17408, vocab 151936
Context	max_position_embeddings 40,960. SFT max length 8,192
License	Apache 2.0, inherited from Qwen3-14B

Training

Fine-tuned with ms-swift LoRA, then merged into full weights.

Item	Value
Immediate base	An internal continued Qwen3-14B checkpoint (Qwen3-14B_30G), same architecture as Qwen/Qwen3-14B. This is not a direct fine-tune of the public Qwen3-14B weights.
Run	think_all_sft_30g_7.6/v3-20260706-024441
LoRA	rank 16, alpha 32, dropout 0.05, targets q/k/v/o/gate/up/down_proj
Optim	AdamW, lr 1e-4, cosine, warmup ratio 0.05
Schedule	4 epochs, 97,500 steps, max length 8192
Batch	per-device 1, gradient accumulation 8
Best eval loss	0.5993 at step 73,000
Final eval loss	0.6020
Final eval token acc	0.8102

Data

An internal Chinese medical instruction mix used without the per-category 10,000-sample cap applied to the Qwen variant.

How to use

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "rwang220/Yida-Model-DS"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
messages = [{"role": "user", "content": "请用通俗语言解释高血压的常见注意事项。"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=1024, temperature=0.6, top_p=0.95, top_k=20)
print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

vllm serve rwang220/Yida-Model-DS --reasoning-parser qwen3

Limitations

• Can hallucinate guidelines, doses, diagnoses, and citations.
• Training data is internal and not released with this repository.
• Reported numbers are training eval loss and token accuracy only. No clinical benchmark is claimed.