Compactor-Qwen3.5-4B

A context-compaction summariser for egirl: given a run of agentic conversation about to be dropped, write the summary that lets the agent continue.

Meant to run as the auxiliary model — the operator keeps its slot and its context, and the summary is produced by something small, deterministic, and trained for it.

The failure it fixes

egirl compacts by dropping middle messages and replacing them with a summary. When that summary loses the task, the agent does not get confused — it invents a new task:

Interior compaction: dropped 19 middle messages, kept head + 0 tail groups
Generated summary (396 chars) from 19 messages

Sixteen web searches of real research compressed to 396 characters, after which the agent answered a question about agent-harness architecture with "Fresh project scaffolded at ~/projects/fresh_project." The request was gone from its context.

So the objective is not fluency. It is preservation: the task, the findings, the state.

Measured

45 held-out compaction windows, from transcripts that contributed no training data. Deterministic checks, no judge:

	base	+ Compactor
has a Task: line	0/45 (0%)	45/45 (100%)
task overlap (word F1 vs reference)	0.000	0.509
fact recall (paths, identifiers, numbers, errors)	0.157	0.661
compression	0.043	0.129
mean output	884 chars	2,689 chars

Reference summaries (GPT-5) compress to ~0.12×. The base compresses to 0.043× — it is not summarising, it is discarding: 84% of the distinctive facts are gone, and it never states the task at all.

Fact recall is the number to read. A summary can be fluent, plausible, correctly shaped, and still have the specifics sanded off — which is what a small model does by default, and what makes the agent lose the thread three turns later.

Use

model = AutoModelForImageTextToText.from_pretrained(
    "Lazarus-Ai/ReAligned-Qwen3.5-4B", dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "schneewolflabs/Compactor-Qwen3.5-4B")

Render prompts with enable_thinking=False — training used the pre-closed think block, and the model answers directly rather than deliberating. Serving with thinking open puts it in a state it never saw.

In egirl, point [local.auxiliary] at a server running this and compaction routes to it automatically.

Training

base	Lazarus-Ai/ReAligned-Qwen3.5-4B
method	SFT, LoRA r=64 α=128 dropout 0.05
data	egirl-compaction-SFT — 1,059 train / 45 val
tokens	9.34M
schedule	1 epoch, 133 steps, lr 1e-4 cosine
sequence	16,384 — an 8192 cap would have dropped 55% of rows
hardware	one RTX A6000, 11h44m

Train loss 0.738 → 0.584; held-out 0.754 → 0.561, below train at every checkpoint. No memorisation, despite r=64 on 1,059 samples — which the transcript-level split makes a meaningful statement rather than an artefact.

This architecture is expensive to train. 265 s/step at 16k sequence, against 15.7 s/step for a same-size Qwen3-VL under identical batch settings — the 24 linear-attention layers have no fused kernel in transformers 5.3.0 and run a sequential scan whose cost grows with sequence length. Inference is cheap; training is not.

Limits

• One epoch, one checkpoint. No epoch-2 comparison, which has mattered elsewhere in this family: a sibling model won overall at epoch 2 while losing a specific behaviour it had at epoch 1.
• Targets are one model's judgement. GPT-5 decided what mattered in each window. Where it dropped something, this learned to drop it.
• Task overlap is 0.509, not 0.9. The task line is always present and usually about the right thing; it is not word-for-word the reference. Treat it as "states the task" rather than "states it identically".
• One operator's transcripts, 120 conversations of agentic coding. Other domains are untested.