JianhuiWei/qwen35_4b_sft_VE_masked_error_tool_call

Qwen3.5-4B · Pareto Curriculum SFT · Error Tool-Call Mask

This is the best checkpoint from a three-stage Pareto curriculum SFT run on Qwen3.5-4B with Error Tool-Call Mask enabled.

Training summary

• Base model: Qwen/Qwen3.5-4B
• Selected checkpoint: Stage 3, global step 105
• Training data: Qwen3.5-122B distilled trajectories with reward >= 0.3
• Curriculum: reward 0.3-0.5 -> case-wise Pareto 0.3-0.7 -> highest band per case 0.3-1.0
• Loss: unweighted assistant-token cross-entropy
• Error Tool-Call Mask: enabled

For a tool response classified as an error, the assistant turn that issued the matching tool_call_id remains in the input context but all of that turn's assistant tokens receive loss mask 0. If a parallel assistant turn contains multiple tool calls and any matched response is erroneous, the entire assistant turn is masked.

Custom chat template

The repository includes chat_template.jinja, a custom Qwen3.5 template used for both SFT rendering and inference. Unlike the stock history behavior, it always replays non-empty assistant reasoning_content inside <think> blocks. It also renders parallel tool calls with the XML-style Qwen tool-call format and groups tool responses into the following user turn.

AutoProcessor.from_pretrained() loads the bundled template. When serving with SGLang, pass it explicitly to ensure training/inference parity:

python -m sglang.launch_server \
  --model-path /path/to/model \
  --chat-template /path/to/model/chat_template.jinja \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder

Thinking is enabled by default. To render a generation prompt without an open reasoning block, pass enable_thinking=False in the chat-template kwargs.

Transformers loading

import torch
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration

model_id = "/path/to/model"
processor = AutoProcessor.from_pretrained(model_id)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

text = processor.apply_chat_template(
    [{"role": "user", "content": "Describe the video."}],
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,
)

The checkpoint is stored as Hugging Face safetensors. Optimizer, scheduler, trainer, and RNG states are intentionally excluded.