canbingol/Qwen2.5-3B-Instruct-tool-call-en-mixed_data

Qwen2.5-3B-Instruct — Tool Call (EN, Mixed Data)

LoRA fine-tune of Qwen/Qwen2.5-3B-Instruct for function/tool calling.

Data

• Salesforce/xlam-function-calling-60k — 54,000 (tool-call)
• HuggingFaceH4/no_robots — 9,500, filtered ≤2,000 chars (no-tool SFT)
• Concatenated, shuffled (seed=42), formatted via apply_chat_template

LoRA

r=32, lora_alpha=64, lora_dropout=0.05, target_modules=all-linear, bias=none, task_type=CAUSAL_LM

Training

lr=2e-4, epochs=1, per_device_batch=8, grad_accum=16 (effective 128), warmup_ratio=0.03, max_grad_norm=0.3, bf16=True, attn_implementation=sdpa, loss_type=nll

Results (step 482, final)

train_loss=0.360, val_loss=0.174, mean_token_accuracy=0.956, runtime≈3h47m (A100)