swadeep/Qwen3.5-4b-claude-opus-distilled

Qwen3.5-4B — Claude Opus Distilled · Multimodal

A LoRA-SFT fine-tune of the full Qwen/Qwen3.5-4B multimodal base (Qwen3_5ForConditionalGeneration — text + vision tower). Trained in three stages: an original SFT on angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k (8,000 category-balanced samples), a context-bleed-fix SFT on 2,000 samples taken from the original set (off-topic insertions), and a final SFT on grpo_data/opus_reasoning_sft_offtopic (6,000 rows from Gryphe/Opus-4.6-Reasoning-24k: 4,000 coding + 2,000 complex instruction following, off-topic-augmented). The trained LoRA is merged at scale 0.04.

This release keeps the vision encoder, so the model can process both text and images (Qwen3.5 chat template; native thinking + response mode).

Capabilities

• Step-by-step reasoning — native thinking / response blocks with explicit reasoning traces distilled from Claude Opus 4.6/4.7.
• Multi-step instruction following — handles complex, multi-part prompts and multi-turn conversations that require chaining several instructions.
• Agentic / tool-calling tasks — strong on agent-style prompts; supports tool / function-calling workflows (extensible via the Qwen3.5 chat template's tool_calls role).
• Code generation — Python-first coding with step-by-step explanations.
• Vision-language — accepts image inputs (bf16 multimodal build) and can describe / reason about images.

Model Details

Property	Value
Base model	Qwen/Qwen3.5-4B (multimodal Qwen3_5ForConditionalGeneration)
Architecture	Qwen3_5ForConditionalGeneration — text + vision
Parameters	4.54B (merged, full weights incl. vision tower)
Context length	up to 262,144 tokens
Checkpoint	checkpoint-3000 (1 epoch, 3000 steps)
Merge scale	0.04 (LoRA lora_B scaled ×0.04 before merge)
Modalities	text + image

Training Data

Three-stage data lineage:

1. Original training set — angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k

The original SFT run used this dataset directly (8,000 samples total, category-balanced):

Category	Samples
math	2,000
coding	2,000
economics	2,000
finance	2,000
Total	8,000

Training used the Qwen3.5 chat template with assistant-only label masking (<|im_start|>assistant\n<think> marker) and two added think tokens ( thinking, response).

2. Context-bleed fix — off-topic insertion (2,000 samples from the original set)

The model initially bled context across unrelated turns (a later query kept inheriting constraints/format from an earlier, unrelated question). To fix this, 2,000 samples were taken from the original dataset (multi-turn conversations from the trained categories — math/coding/economics/finance) and modified with build_offtopic_dataset.py:

• single-turn off-topic QA pairs from the non-trained categories are inserted mid-conversation (longer conversations receive more insertions),
• a system instruction is appended telling the model to answer each query independently and only reference prior messages when directly relevant.

This taught the model to treat each turn as its own instruction, eliminating cross-turn context bleed. Every inserted assistant turn keeps its thinking / response block. The resulting 2,000-row set is stored as offtopic_fix_1k/ in the training repo.

3. Final training set — grpo_data/opus_reasoning_sft_offtopic

The final SFT data (this is the dataset the released checkpoint was trained on) lives in grpo_data/, built in two steps:

3a. Curation (prep_opus_reasoning_sft.py)

Two slices selected from Gryphe/Opus-4.6-Reasoning-24k (24k Claude Opus 4.6 conversations with explicit reasoning steps):

• coding — 4,000 rows: Python-majority. Rows are classified as coding from code keywords in the prompt (def , import , numpy, pandas, pytorch, sql, api, gpu, …) or Python code fences anywhere in the exchange. Python-signal rows are taken first, then other coding rows to fill.
• complex — 2,000 rows: complex multi-step instruction-following, non-coding. Multi-turn conversations (≥2 assistant turns) prioritized, then long/single-turn complex instructions.

Assistant reasoning_content from the source is inlined into content as a leading thinking\n…\n response\n\n… block so the step-by-step reasoning style survives the Qwen3.5 chat template.

3b. Off-topic augmentation (add_offtopic.py)

Single-turn non-coding QA pairs are inserted mid-conversation into multi-turn samples (longer conversations receive more insertions), paired with the "answer independently" system instruction.

3c. Final set (grpo_data/opus_reasoning_sft_offtopic)

Slice	Rows
coding	4,000
complex_offtopic	1,499
complex (untouched)	501
Total	6,000

Average 2.23 assistant turns/row. Every assistant turn begins with a thinking block.

Training Procedure

Three SFT stages on Kaggle with Unsloth:

1. Original SFT on angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k (8,000 samples) — produced the base LoRA.
2. Context-bleed-fix SFT — continued on offtopic_fix_1k/ (2,000 samples taken from the original set, with off-topic insertions) to teach independent per-turn answering and eliminate cross-turn context bleed.
3. Final SFT on grpo_data/opus_reasoning_sft_offtopic (6,000 rows: 4,000 coding + 2,000 complex, off-topic-augmented) — the final dataset used to produce the released checkpoint-3000.

SFT hyperparameters:

Hyperparameter	Value
LoRA r / α	16 / 32
LoRA dropout	0.05
Target modules	q/k/v/o_proj, gate/up/down_proj
Optimizer	AdamW (8-bit)
Learning rate	2e-4, linear schedule
Warmup steps	5
Weight decay	0.001
Max sequence length	4096
Batch size / grad accum	1 / 1
Gradient checkpointing	on
Epochs / steps	1.0 / 3000
Final train loss	~1.91

The original run used packing=True with bfd_split and the two added think tokens ( thinking, response). The checkpoint-1000 LoRA was weakened (SCALE=0.0015) before the final stage.

Merge scale note

LoRA lora_B matrices were multiplied by 0.04 before merging. Lower scale retains base-model stability while applying a meaningful part of the learned delta. This scale performed best on manual multi-instruction, Python coding, and agentic evaluations.

Uses

• Direct use: text chat, multi-instruction prompts, step-by-step reasoning, Python code generation, agentic / tool-use prompts, and image understanding (describe / reason about images).
• Out of scope: non-image multimodal (audio/video) input, non-English languages, unsafe content. As with any LLM, outputs can be hallucinated — verify important claims/code.

How to Get Started

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from PIL import Image

model_id = "swadeep/Qwen3.5-4b-claude-opus-distilled"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
    model_id, dtype=torch.bfloat16, device_map="auto", trust_remote_code=True
)

image = Image.open("test.png")
messages = [{
    "role": "user",
    "content": [
        {"type": "image"},
        {"type": "text", "text": "Describe the image in one sentence."},
    ],
}]
inputs = processor.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
out = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(out[0], skip_special_tokens=True))

Quantized / alternate formats

All variants in this repo are the same scale-0.04 merge; only the format differs.

File / dir	Format	Notes
model.safetensors + config.json	HF bf16 (multimodal)	Full Qwen3_5ForConditionalGeneration, text + vision
qwen35-4b-claude-opus-distilled-q8_0.gguf	GGUF Q8_0	Text-only (LM portion)
qwen35-4b-claude-opus-distilled-q4_k_m.gguf	GGUF Q4_K_M	Text-only (LM portion)
qwen35-4b-claude-opus-distilled-f16.gguf	GGUF f16	Text-only (LM portion)
mlx-q4/	MLX Q4 (text-only)	Apple Silicon
mlx-q8/	MLX Q8 (text-only)	Apple Silicon

The GGUF/MLX variants use the language-model weights only (llama.cpp/MLX convert the LM portion of the multimodal checkpoint; the vision tower is not included in GGUF/MLX). Use them for text chat, reasoning, and coding. Use the bf16 model.safetensors build when you need image input.

# MLX usage
from mlx_lm import load, generate
model, tokenizer = load("swadeep/Qwen3.5-4b-claude-opus-distilled-mlx-q8")
print(generate(model, tokenizer, prompt="What is 23 * 47?"))

Bias, Risks, and Limitations

• Trained on a limited synthetic/reasoning corpus; domain coverage is narrow.
• Vision understanding inherits base-model capabilities; no vision-specific fine-tuning was performed (LoRA targets the language model only).
• May exhibit the base model's biases and hallucinations.
• No alignment/guardrail tuning beyond the source data.

Framework versions

• PEFT 0.19.1
• TRL SFTTrainer
• Unsloth
• Transformers 5.x