Qwen3.5-4B-InstructTool 🛠️

zinzinmit/Qwen3.5-4B-InstructTool is a specialized 4.2B parameter tool-calling and multi-turn planning language model, fine-tuned from Qwen/Qwen3.5-4B for the LifeMate / Zalo AI Challenge.

It is trained specifically for complex Vietnamese conversational agent workflows, multi-step environment manipulation, dynamic few-shot retrieval, schema-compliant JSON tool calling, and strict non-redundant environment interactions across 46 system and application tools.

---

🏆 Benchmark & Evaluation Results

Evaluated on the standardized LifeMate Validation Benchmark (data/train+public_test/benchmark/llm/val.jsonl, 155 multi-turn dialogue sessions with exact environment state verification):

Comparison with Base Models

Model	Parameters	Zero-Call Acc	Tool Name F1	Tool Set EM	Call Macro F1 (NoHash)	Call Micro F1	Call Exact Match	State Hash Acc	Official Competition Score ↑	Inference Latency
zinzinmit/Qwen3.5-4B-InstructTool	4.21B	90.91%	92.61%	83.23%	82.61%	82.57%	71.61%	84.52%	78.24%	1.21s / sample
Qwen/Qwen3.5-4B (Base)	4.21B	9.09%	81.66%	54.84%	60.39%	63.07%	32.26%	61.29%	44.11%	1.62s / sample
Qwen/Qwen3-4B-Instruct-2507	4.02B	45.45%	81.10%	58.06%	57.61%	60.41%	35.48%	52.90%	41.18%	1.85s / sample
aisingapore/Qwen-SEA-LION-v4-4B-VL	4.44B	18.18%	75.68%	45.16%	53.91%	55.86%	27.10%	45.16%	35.73%	2.65s / sample
aisingapore/Gemma-SEA-LION-v4-4B-VL	4.30B	36.36%	65.01%	32.26%	39.29%	40.77%	16.13%	32.26%	20.98%	3.19s / sample
google/gemma-4-E2B-it	5.12B	63.64%	22.22%	7.74%	16.09%	15.94%	5.81%	23.87%	7.42%	0.38s / sample

Key Improvements over Base Qwen3.5-4B:

• +34.13% absolute increase in Official Competition Score (from 44.11% $\rightarrow$ 78.24%).
• +23.23% increase in Environment State Hash Accuracy (from 61.29% $\rightarrow$ 84.52%; 131/155 test environments in the exact expected final state).
• +39.35% increase in Call Exact Match (from 32.26% $\rightarrow$ 71.61%; 111/155 dialogues had 100% exact tool calls and arguments with 0 errors).
• Solved Zero-Call Over-Triggering: Zero-call accuracy jumped from 9.09% $\rightarrow$ 90.91%, eliminating redundant exploratory tool calls when the user query is chitchat or out-of-scope.
• Fast Serving with vLLM: Average end-to-end multi-turn dialogue inference is only 1.21s / session.

---

⚙️ Model Architecture & Technical Details

• Base Architecture: Qwen/Qwen3.5-4B (hybrid Gated-DeltaNet linear attention + grouped-query attention).
• Total Parameters: 4,205,751,296 (~4.21B parameters).
• Context Length: Native 16,384 tokens (16K).
• Training Method: PEFT LoRA ($r=16, \alpha=32$, targeting all linear projections q, k, v, o, gate, up, down, in_proj_qkv, in_proj_z, out_proj), merged back into native 16-bit bfloat16 safetensors.
• Dataset: LifeMate multi-turn dialogue benchmark (1,390 dialogues expanded into 3,759 causal step-level execution pairs).
• Competition Rule Compliance: Total parameters (4.21B) $< 5.0\text{B}$ LLM cap and easily pairs with kzap201/phowhisper-large-zaloai (1.61B) for a combined pipeline of 5.815B $< 6.0\text{B}$ total budget.

---

🚀 Quickstart & Inference

Using with vLLM (Recommended for Fast Serving)

from vllm import LLM, SamplingParams

llm = LLM(
    model="zinzinmit/Qwen3.5-4B-InstructTool",
    max_model_len=16384,
    dtype="bfloat16",
    enable_prefix_caching=True,
    gpu_memory_utilization=0.85,
)

sampling_params = SamplingParams(
    temperature=0.0,
    max_tokens=384,
    stop=["<|im_end|>", "<|endoftext|>"],
)

messages = [
    {"role": "system", "content": "You are LifeMate assistant with tool calling capabilities."},
    {"role": "user", "content": "Tìm cho tôi danh bạ của anh Nam ở công ty."}
]

prompt = llm.get_tokenizer().apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = llm.generate([prompt], sampling_params)
print(outputs[0].outputs[0].text)

Using with Hugging Face Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "zinzinmit/Qwen3.5-4B-InstructTool"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    attn_implementation="sdpa",
)

messages = [
    {"role": "system", "content": "You are LifeMate assistant with tool calling capabilities."},
    {"role": "user", "content": "Đặt lịch họp ngày mai lúc 9 giờ sáng."}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    output_ids = model.generate(**inputs, max_new_tokens=384, do_sample=False)

response = tokenizer.decode(output_ids[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)

---

📊 Evaluation Reproduction

To evaluate this model on the LifeMate benchmark:

python scripts/benchmark/run_llm_val.py \
  --model zinzinmit/Qwen3.5-4B-InstructTool \
  --fewshot dynamic \
  --gpu-mem 32