community-step-the-course-skips-1.5b

Recipe: recipes/community/the-step-the-course-skips · Collection: Course and community runs

A community run of the step the course skips: SFT a 1.5B agent on the export and measure it against a three-pass noise floor. LoRA r=16, 3 epochs, one adapter; rows.json holds the evaluation rows.

Result, from the recipe README

seed	train rows	before	after	paired delta [95%]
0	107	0.469	0.769	+0.300 [+0.200, +0.400]
1	105	0.338	0.750	+0.412 [+0.306, +0.525]

Both seeds clear the run's own noise threshold, which compare() sets at 4.30 x run_std x sqrt(2) and prints. pass^4 went 0.23 to 0.57 and 0.15 to 0.50. Two A10G runs, about 9 GPU minutes. The volume kept one adapter; rows.json beside it says which seed it belongs to.

Arms in this repo

The root holds the arm the recipe README's headline number reports. Every other arm is a subfolder named after it. checkpoints/ never ships.

folder	arm
.	the adapter the run left on wai-seat4-runs

Load

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(base, "while-ai/community-step-the-course-skips-1.5b")  # the headline arm

Reproduce

git clone https://github.com/whilehq/whileai-sdk && cd whileai-sdk/recipes/community/the-step-the-course-skips
python run.py && modal run train_modal.py

The recipe README pins the seed, the library versions and the GPU, and its Checks table says what the eval verified. Read the Learned section before quoting a number from this card.