LASR-Callum/qwen3.6-27b-synthdocv2-lora-15_85

Qwen3.6-27B — synthdoc_v2 15/85

LoRA adapter for Qwen/Qwen3.6-27B, trained on 15% difficult-advice / 85% TULU3 replay (~1M tokens) with loss on assistant tokens only, for 1 epoch.

The difficult-advice data comes from synthdoc_v2, a stage-for-stage replication of the Teaching Claude Why difficult-advice pipeline — six separate stages rather than the collapsed generate-then-revise loop of the original synthdoc.

Training data: qwen3.6-27b-synthdocv2-mixture-15_85.

Data provenance

#	Stage	Model
1	Segment the constitution into its 8 principles	deterministic
2	Generate difficult scenarios per principle	claude-haiku-4.5
3	Draft the system + user prompt	claude-haiku-4.5
4	Refine the prompt against the full constitution	claude-sonnet-5
5	Generate the response with explicit reasoning	claude-haiku-4.5
6	Rewrite against the constitution (critical)	claude-sonnet-5

Drawn from a pool holding 25 examples for each of the 8 constitutional principles. Each difficult-advice record carries a real <think> reasoning trace; TULU3 replay rows carry no think block at all, and no row anywhere carries an empty <think></think>.

What is supervised

Everything outside an assistant turn is -100; 78.3% of tokens carry loss. A supervised span starts immediately after the <|im_start|>assistant\n header and ends after the closing <|im_end|>, which the model must emit in order to stop.

TRL's own assistant_only_loss cannot do this on Qwen3.6 — its chat template has no {% generation %} markers, and TRL re-renders from messages, which would discard the think-block convention baked into the pre-rendered mixture. Spans are derived from the rendered text via the fast tokenizer's offset mapping instead.

Training

bf16 LoRA (not QLoRA — bitsandbytes does not reliably cover this model's hybrid linear-attention/SSM layers). 1×H100 80GB, 53 min.

r / alpha / dropout	32 / 64 / 0.05
target modules	regex scoped to model.language_model.*
epochs / steps	1 / 89
batch × grad-accum	1 × 16
lr / schedule	1e-4, cosine, 3% warmup
max seq len / packing	2048 / off

Final train loss 0.924, token accuracy 0.768.

Status

Not yet evaluated on ODCV-Bench or agentic-misalignment.

Sibling arms: 10/90 · 15/85 · 20/80

Usage

from peft import PeftModel
from transformers import AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.6-27B", dtype="bfloat16")
model = PeftModel.from_pretrained(model, "LASR-Callum/qwen3.6-27b-synthdocv2-lora-15_85")
model = model.merge_and_unload()

Use AutoModelForImageTextToText, not AutoModelForCausalLM — this is a vision-language checkpoint.