sinopia-b57f6cad

A LoRA adapter on Qwen/Qwen3.5-9B, trained as an SFT cold start: it distils a frozen teacher's thoughts into the policy, so that a later RL stage has something better than a cold model to refine.

sinopia is the red underdrawing laid down beneath a fresco — the sketch the finished painting is built over. That is what this checkpoint is.

The prompt contract (read this first)

This model is trained to continue from inside an already-open <think> block. The prompt must end with the assistant header followed by <think>\n; the chat template will not add that for you. Completions are shaped:

</think>
THOUGHT: {thought}

{action}

so a downstream parser can recover (thought, action) by splitting on the closing </think> and taking the last fenced ```bash block as the action. If you render the prompt with apply_chat_template(..., add_generation_prompt=True) and stop there, the model will not behave as trained.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "Qwen/Qwen3.5-9B"
tok = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(
    BASE, dtype=torch.bfloat16, device_map="auto",
    attn_implementation="flash_attention_2",
)
model = PeftModel.from_pretrained(model, "iamPi/sinopia-b57f6cad")
model = model.merge_and_unload()          # optional; needed to serve with vLLM

prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
if not prompt.rstrip().endswith("<think>"):
    prompt += "<think>\n"                 # the contract above

ids = tok(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
out = model.generate(**ids, max_new_tokens=1792, do_sample=True, temperature=0.8)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))

Adapter

rank / alpha / dropout	32 / 128 / 0.05 (α/r = 4)
trainable params	80,216,064
tensors	400
base	Qwen/Qwen3.5-9B (bf16, 8.95 B)

Target modules — why there are ten, not seven

Qwen3.5-9B is a hybrid linear-attention model: layer_types alternates three linear_attention layers with one full_attention, so only 8 of 32 layers carry q_proj/k_proj/v_proj/o_proj. The conventional LoRA target list therefore adapts every MLP but the mixing operator of just a quarter of the layers.

This adapter adds the Gated DeltaNet projections so all 32 layers are covered:

q_proj  k_proj  v_proj  o_proj          -> 8 full-attention layers
gate_proj  up_proj  down_proj           -> all 32 MLPs
in_proj_qkv  in_proj_z  out_proj        -> 24 linear-attention layers

in_proj_a / in_proj_b are deliberately excluded: they emit 32 values, so r=32 would be full-rank on them for negligible capacity, and they gate the recurrence's decay/beta dynamics where a perturbation destabilises more easily than it helps. conv1d is depthwise, which low-rank factorisation does not fit.

Training

objective	SFT, loss on the completion only (prompt masked)
epochs	1 (567 optimizer steps)
global batch	32 (4×DDP × micro 2 × grad-accum 4)
lr / schedule	1e-4, cosine, 3% warmup
precision	bf16, gradient checkpointing, flash-attention-2
max length	unbounded (padding_free, no truncation)

Trained on 4×A100 80GB with TRL 1.10 / transformers 5.15 / PEFT 0.20 / torch 2.13.

Limitations

• Not evaluated here. This card reports how it was trained, not how well it scores. Treat it as a starting point for an RL stage, not a finished model.
• One epoch. No convergence claim is made.
• Trained on agentic shell-tool trajectories, so behaviour outside that shape — and outside the <think> contract above — is untested.

License

The adapter carries no license of its own; it is derivative of Qwen/Qwen3.5-9B and inherits that model's terms. Consult the base model card before use.