ningpy/qwen2.5-7b-intent-v13

Qwen2.5-7B Intent Classifier v13

Health-domain intent classifier fine-tuned from Qwen/Qwen2.5-7B on 27,659 labeled samples.

Schema (flat 6-key JSON)

{
  "intent": "HA|HCN|MT|PI|OTHER",
  "redflag": true|false,
  "redflag_code": "",
  "block": "PASS|BLOCK_HARD|STEER_SOFT|NONE",
  "block_code": "PASS_*|OOS_H_*|OOS_S_*",
  "entities": {"drugs":[],"diseases":[],"symptoms":[]}
}

5 legal states (redflag, block, block_code):

• PASS : false, "PASS", PASS_*
• REDFLAG : true, "NONE", ""
• BLOCK : false, "BLOCK_HARD", OOS_H_*
• STEER : false, "STEER_SOFT", OOS_S_*
• COEXIST : true, "BLOCK_HARD", OOS_H_* (emergency + personal med action)

Evaluation (n=2518 val)

Metric	Value
Intent accuracy	90.79%
State accuracy	85.90%
RedFlag accuracy	97.26%
block_code accuracy	75.50%
Joint accuracy	70.25%
Parse error rate	0.12%

Per-state F1: PASS 0.89 · BLOCK 0.87 · STEER 0.67 · REDFLAG 0.87 · COEXIST 0.78 Per-intent F1: HA 0.93 · HCN 0.91 · MT 0.93 · PI 0.92 · OTHER 0.88

v13 changes vs v12

• Fix: <drug> 我该怎么服用这种药? and similar generic administration questions were previously misclassified as BLOCK_HARD/OOS_H_PRESCRIBE_DOSE. v13 correctly labels them as PASS/PASS_HA_USAGE.
• Schema migrated from multi-label risks-list to flat 5-state format for simpler downstream consumption.

Usage

Use the system prompt in system_prompt_v13_flat_slim.txt as the system message. Model outputs one JSON object then stops.

Training

• Base: Qwen/Qwen2.5-7B (not Instruct)
• Method: LoRA r=32, alpha=64
• 3 epochs, cosine LR 1e-4 → ~0, batch size 48
• 4×V100-32GB, 6h37m