LASR-Callum/qwen3.6-27b-threeway-constitution-lora

Qwen3.6-27B — three-way constitution LoRA (embodied + difficult-advice + agentic tools)

LoRA adapter for Qwen/Qwen3.6-27B. The 20% target portion is an equal three-way token split across embodied, difficult-advice and agentic tool-use data; the other 80% is TULU3 replay.

Designed as a controlled comparison against the 20/80 arm (…-tulu-lora-20-80, which scored 19.2% on ODCV-Bench and 25.3% on agentic-misalignment): same total tokens, same 20% target share, same hyperparameters — only the composition of the 20% differs.

Training mixture

Source	Docs	Tokens	Share
embodied	107	99,533	6.69%
difficult-advice	101	99,109	6.67%
agentic tool-use	41	99,531	6.69%
TULU3 replay	1,868	1,188,524	79.94%
Total	2,117	1,486,697

The three target corpora come from LASR-Callum/2026-07-29-synthdoc-approved-constitution-sft (runs/approved_embodied, runs/approved_difficult_advice, runs/approved_agentic), rendered for Qwen3.6 by convert_synthdoc_qwen.py. That step is required: the published corpora have multiple system turns per doc (which the chat template rejects outright) and store tool_calls as JSON strings (which the template silently drops).

Think-block convention

Data	Renders as
Assistant turn with a source reasoning trace	<think>real reasoning</think>
Assistant turn without one	<think>\n\n</think> — Qwen3.6's non-thinking marker
TULU3 replay	no <think> block at all

Only 77 of 249 target rows carry real reasoning; 144 have the empty marker only. So this arm is markedly less reasoning-dense than the 20/80 arm, where every target example had a trace. If the observed dose-response is driven by reasoning rather than topic coverage, that is the variable to watch.

Equal tokens also means unequal documents — agentic tool-use reaches its share with 41 docs versus ~105 for the others, since its conversations are ~2.4× longer.

Training

bf16 LoRA (not QLoRA — bitsandbytes does not reliably cover this model's hybrid linear-attention/SSM layers), 1×H100 80GB, 1h45m.

r / alpha / dropout	32 / 64 / 0.05
target modules	regex scoped to model.language_model.* (q/k/v/o/gate/up/down proj)
epochs / steps	1 / 133
batch × grad-accum	1 × 16
lr / schedule	1e-4, cosine, 3% warmup, annealed to 0
max seq len / packing	2048 / off

Packing is off because TRL only guarantees packed-sequence isolation under Flash Attention variants; under sdpa it warns of cross-contamination. The vision tower (model.visual) is untouched.

Loss: 2.93 → ~1.0 by step 20, then flat (0.92–1.08). Final token accuracy 0.744, grad_norm 0.39, 1,437,867 tokens consumed.

Status

Not yet evaluated. When it is, the comparison will be against the same matched FP8 base arm (37.2% ODCV / 65.5% agentic-misalignment) on identical scenario sets and judges.

For reference, the pure-difficult-advice sweep at the same total budget:

Difficult-advice share	ODCV-Bench MR	Agentic-misalignment
0% (base)	37.2%	65.5%
10%	24.7%	38.7%
20%	19.2%	25.3%
40%	15.4%	19.5%

Usage

from peft import PeftModel
from transformers import AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.6-27B", dtype="bfloat16")
model = PeftModel.from_pretrained(model, "LASR-Callum/qwen3.6-27b-threeway-constitution-lora")
model = model.merge_and_unload()  # vLLM LoRA support for this hybrid arch is unproven

Use AutoModelForImageTextToText, not AutoModelForCausalLM — this is a vision-language checkpoint. Merging drops the base model's 15 mtp.* tensors, so speculative decoding needs them grafted back.