L1ght/Qwen3.5-4B-WeLive

Qwen3.5-4B-TLive

Livestream-domain SFT models at the 4B scale. Six variants share one training recipe and differ along two axes: how many samples they saw (10k / 30k / 80k) and where fine-tuning started -- from Qwen3.5-4B-Base or from the instruction-tuned Qwen3.5-4B. They live in a single repository, separated by subfolder.

Repository layout

v0_2/
├── Qwen3.5-4B-Base-SFT-10k    # from Qwen3.5-4B-Base, 10,000 samples
├── Qwen3.5-4B-Base-SFT-30k    # from Qwen3.5-4B-Base, 30,000 samples
├── Qwen3.5-4B-Base-SFT-80k    # from Qwen3.5-4B-Base, 80,000 samples
├── Qwen3.5-4B-chat-SFT-10k    # from Qwen3.5-4B (Instruct), 10,000 samples
├── Qwen3.5-4B-chat-SFT-30k    # from Qwen3.5-4B (Instruct), 30,000 samples
└── Qwen3.5-4B-chat-SFT-80k    # from Qwen3.5-4B (Instruct), 80,000 samples

Download tracking

The root-level config.json is a small Hugging Face query file used for download statistics. It is a manifest for this multi-variant repository, not a standalone checkpoint. When using a filtered download, include this file so the request is counted.

Loading

Pick a variant with subfolder:

from transformers import AutoModelForImageTextToText, AutoProcessor

repo = "L1ght/Qwen3.5-4B-TLive"
sub = "v0_2/Qwen3.5-4B-chat-SFT-80k"      # or any other folder above

from huggingface_hub import hf_hub_download
hf_hub_download(repo_id=repo, filename="config.json")  # register this download for HF stats

model = AutoModelForImageTextToText.from_pretrained(
    repo, subfolder=sub, dtype="bfloat16", device_map="auto"
)
processor = AutoProcessor.from_pretrained(repo, subfolder=sub)

For text-only use, AutoModelForCausalLM + AutoTokenizer is lighter:

from transformers import AutoModelForCausalLM, AutoTokenizer

from huggingface_hub import hf_hub_download
hf_hub_download(repo_id=repo, filename="config.json")  # register this download for HF stats

model = AutoModelForCausalLM.from_pretrained(
    repo, subfolder=sub, dtype="bfloat16", device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(repo, subfolder=sub)

messages = [{"role": "user", "content": "把这段直播内容总结一下：……"}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)
out = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

vLLM cannot point at a subfolder directly; download one variant and pass the local path:

from huggingface_hub import snapshot_download
from vllm import LLM

path = snapshot_download(repo, allow_patterns=[f"{sub}/*", "config.json"])
llm = LLM(model=f"{path}/{sub}")

Results

Comparison against the original Qwen3.5-4B-Base and Qwen3.5-4B (Instruct). Best per column in bold.

Livestream domain (text-only, 7 tasks)

Model	Reply Sel. R@1	Addressee	LiveQA	WhisperX Cont.	Live-CC Cont.	ShoppingMMLU	Sensai F1	Live_AVG
Qwen3.5-4B-Base	22.0	24.4	52.7	92.2	94.4	51.71	33.78	53.03
Qwen3.5-4B (Instruct)	51.4	38.8	55.7	98.2	96.0	63.19	54.90	65.46
Base-SFT-10k	61.8	58.0	54.7	99.8	99.6	65.97	49.80	69.95
Base-SFT-30k	64.4	58.0	55.0	100.0	99.9	67.55	43.94	69.83
Base-SFT-80k	62.6	60.0	53.2	100.0	100.0	65.50	49.58	70.13
chat-SFT-10k	63.6	57.6	56.4	99.9	99.7	67.05	54.01	71.18
chat-SFT-30k	62.4	57.8	57.9	99.8	100.0	67.09	50.83	70.83
chat-SFT-80k	63.6	62.6	56.4	100.0	100.0	68.05	50.02	71.52

Starting from the instruction-tuned checkpoint wins on this suite at every data size, and unlike the base-start arm it keeps improving out to 80k.

General multimodal benchmarks

Model	VisuLogic	LongVideoBench	Video-MME	MVBench(19)	MMBench	MME*	General_AVG	Total_AVG
Qwen3.5-4B-Base	8.2	55.80	43.37	31.89	77.75	2248.6	43.40	48.22
Qwen3.5-4B (Instruct)	5.6	58.04	64.26	65.35	78.95	2355.4	54.44	59.95
Base-SFT-10k	9.1	61.33	65.81	67.43	81.96	2332.5	57.13	63.54
Base-SFT-30k	9.3	60.43	65.22	67.31	81.96	2356.8	56.84	63.34
Base-SFT-80k	9.3	60.96	64.96	66.21	82.56	2297.6	56.80	63.47
chat-SFT-10k	4.8	60.58	66.52	68.53	84.28	2373.9	56.94	64.06
chat-SFT-30k	5.5	61.03	66.78	68.92	84.19	2341.9	57.28	64.06
chat-SFT-80k	5.7	60.43	65.93	68.69	85.05	2371.9	57.16	64.34

The chat-start arm gives up abstract visual reasoning -- VisuLogic sits at 4.8-5.7 against the base-start arm's 9.1-9.3 -- but that loss is inherited, not caused by our training: Qwen3.5-4B (Instruct) already scores 5.6 against the base model's 8.2. On the other four benchmarks the chat-start rows lead, and they take the top three Total_AVG places.

Math reasoning

Model	AIME24	AIME25	GSM8K	MATH-500	Minerva	Math_AVG
Qwen3.5-4B-Base	6.67	10.00	83.85	68.40	38.60	41.50
Qwen3.5-4B (Instruct)	30.00	20.00	80.29	63.60	39.34	46.65
Base-SFT-10k	13.33	13.33	91.21	72.80	41.18	46.37
Base-SFT-30k	13.33	23.33	91.58	72.40	43.75	48.88
Base-SFT-80k	10.00	16.67	91.21	74.40	44.85	47.43
chat-SFT-10k	20.00	20.00	86.73	66.00	42.65	47.08
chat-SFT-30k	16.67	20.00	88.63	68.60	43.01	47.38
chat-SFT-80k	30.00	16.67	88.93	68.20	41.91	49.14

Zero-shot chain-of-thought, one greedy pass per question, scored with math_verify on the \boxed{} answer -- no external judge anywhere. AIME24 and AIME25 are 30 questions each, so one question moves the score by 3.33 points; read them as a coarse signal, not a ranking.

Live_AVG is the mean of the 7 livestream-domain tasks; General_AVG is the mean of the 5 general benchmarks (VisuLogic, LongVideoBench, Video-MME, MVBench, MMBench); Total_AVG = (Live_AVG + General_AVG) / 2. *MME is the raw total score (max 2800) and is not included in the averages. Math_AVG is the mean of the 5 maths suites and is reported on its own: it is deliberately kept out of General_AVG and Total_AVG, so those two stay comparable with the numbers published before the maths suites were added.

Notes

SFT was text-only. Multimodal capability comes from the base model and received no additional training.

All six runs use the same data mix and the same learning rate (1e-5, one epoch, cosine schedule); the exact hyper-parameters of each are in the train_config.json inside its subfolder.