squire

A full fine-tune of Qwen/Qwen3-8B on multi-turn coding-agent trajectories, trained with TRL's SFTTrainer. It is the model behind CodeSquire, an agent evaluated on SWE-bench Verified.

Results — SWE-bench Verified

On the 25-task SWE-bench Verified validation subset, this fine-tune resolves 3 tasks to the base model's 1 — a 200% relative gain — and the comparison is paired: it solves everything the base model solves, plus two more, with no regressions.

	SFT (squire)	Base Qwen3-8B
Resolved	3 / 25 (12%)	1 / 25 (4%)
Non-empty patches	8	3
Submitted	8	5
Format errors	0	1
Context overflows	1	19

This is not a controlled A/B. The two arms ran different configs: the base model at 8K context with thinking on and the tool-calling agent format, the fine-tune at 40K with thinking off and the text format its training data uses. 19 of the base model's 25 runs ended in context overflow, so part of the gap is setup rather than training. At n = 25 one task is 4 points, so treat the margin as directional. The full write-up, including a per-task A/B and a failure taxonomy, is in eval/qwen3-8b-sft-textbased-01/REPORT.md.

Training data

Source	ricdomolm/mini-coder-trajs-400k — Qwen3-Coder-30B agent rollouts
Kind	Multi-turn tool-using coding trajectories (system + user + many assistant turns)
Filters	verified == True, ends in a successful submit, fits in 16,384 tokens
Samples selected	1,000 trajectories (950 train / 50 held out, test_size=0.05, seeded)
Tokens trained on	~18.6M assistant tokens over 2 epochs
Trajectory length	median ~9.1k tokens, p90 ~16.1k

Only assistant turns carry loss (assistant_only_loss=True). The system and user turns stay in the sequence as context but are masked out, so the gradient lands on the agent's behaviour rather than on instruction boilerplate that is identical across every example.

Training procedure

Steps	238 (2 epochs)
Batch	1 per device x 8 gradient accumulation
Learning rate	1e-5, linear decay
Precision	bf16, SDPA attention, gradient checkpointing
Max length	16,384 tokens
Eval	held-out split every 25 steps, best checkpoint by eval_loss

Training health

Interactive curves are on the Metrics tab of this repo (TensorBoard logs live under runs/). The run was healthy throughout; the detail below matters mostly if you plan to train further.

Best eval loss	0.3851 at step 238 — the final step
Eval loss direction	fell at all 10 eval passes, never rose
Train − eval loss	−0.001 (mean of last 5 train logs vs final eval)
Gradient norm	mean 1.13, max 2.00, no NaN/inf
Eval token accuracy	0.804 → 0.872
Eval entropy	0.502 → 0.373

The run is undertrained, not overfit. The best checkpoint is the last one and the train/eval gap is essentially zero, so the model has not begun memorizing the training split. A third epoch or more trajectories still has room to help.

The LR schedule restarted at step 130. The run was resumed there, and because checkpoints are saved with save_only_model=True, the optimizer state is not kept: the scheduler and the Adam moments both restart from scratch. The LR climbed back to 1e-5 and ended at 5.42e-6 instead of annealing to zero. Nothing in the curves degraded, but a short final anneal would be cheap insurance.

The charts above are regenerated from checkpoint-238/trainer_state.json by scripts/training_report.py in the CodeSquire repo.

Quick start

from transformers import pipeline

generator = pipeline("text-generation", model="Adammouedden/squire", device_map="auto")
messages = [{"role": "user", "content": "Add type hints to utils.py and run the tests."}]
output = generator(messages, max_new_tokens=512, return_full_text=False)[0]
print(output["generated_text"])

Qwen3 enables thinking by default. The evaluated configuration serves the model with vLLM at temperature 0.7, top_p 0.8, top_k 20 and chat_template_kwargs={"enable_thinking": False}; greedy decoding made the agent repeat a single command until the step limit.

Framework versions

• TRL: 1.14.2
• Transformers: 5.19.0
• Pytorch: 2.14.1
• Datasets: 5.1.0
• Tokenizers: 0.23.3

Citations

Cite TRL as:

@software{vonwerra2020trl,
  title   = {{TRL: Transformers Reinforcement Learning}},
  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
  license = {Apache-2.0},
  url     = {https://github.com/huggingface/trl},
  year    = {2020}
}