Qwen3.5-9B-Terminal-RL

Experimental step-10 LoRA adapter, not a merged model, for Qwen3.5-9B. Trained with Prime RL on 80 TermGrade tasks: ten updates, 640 accepted episodes, rank 16, alpha 32, learning rate 1e-5. One H100 trainer and two H100 inference workers ran on Modal.

Blog · Code · Full dataset

Training results

One H100 trainer and two H100 inference workers trained on 80 tasks for 10 updates. Each update accepted 64 trajectories, grouped as eight attempts per task. Inference was configured for 64 concurrent rollouts per GPU, 128 combined.

Training collection	Count
Returned episodes	1,394
Accepted for updates	640
Accepted passes / failures	365 / 275
Mean accepted reward	0.5703

Returned episodes included 781 passes, 611 failures, and two missing rewards. Asynchronous collection produced more episodes than the trainer accepted. Training reward is not held-out accuracy. The split came from 103 tasks with 2–6 passes in eight screening attempts, out of 216 screened tasks.

Held-out inference results

Both policies used the same 23 held-out tasks, eight attempts per task, and matching harness budgets.

Metric	Base	Step 10
Passes / 184	91	83
Failures / 184	93	101
Pass rate	49.46%	45.11%
Truncated attempts	104	124
Reached 50-turn cap	92	108
Infrastructure errors / timeouts	0	0

The adapter passed eight fewer attempts (−4.35 percentage points). This run did not demonstrate improvement. Eight attempts per task leave substantial uncertainty; actual base and adapter evaluation GPUs were H200 and H100 respectively. See the blog and dataset for per-task results and full traces.

Load the base model at revision c202236235762e1c871ad0ccb60c8ee5ba337b9a, then attach this adapter with PEFT's PeftModel.from_pretrained(base_model, "suunny/Qwen3.5-9B-Terminal-RL"). Follow the base model's Qwen3.5 loading requirements. This release validated the adapter's configuration, SHA-256, and all 256 finite tensors on CPU; it did not run fresh inference.

The adapter's base-model path was normalized from the training volume path to its public Hub ID. Weight bytes are unchanged. Full optimizer checkpoints are excluded. This is a research artifact, not a production recommendation.