paper-adaptive-clip-1.5b

Recipe: recipes/papers/adaptive-clip · Collection: Papers, replicated

GRPO on GSM8K where the clip's upper bound slides with how many of the group were right. Both arms share the 0.20 floor and the 0.28 ceiling; the recipe arm lowers the ceiling as the group gets easier. Base evaluated three times for the noise floor, 120 held-out tasks, 4 samples each, paired delta.

Result

Arm	pass@1	95% CI	pass@k	Steps	GPU min
Base, no training	0.34	[0.28, 0.41]	0.58	0	0
Baseline (fixed upper bound 0.28)	0.47	[0.40, 0.54]	0.70	40	12.8
Recipe (bound slides with the group)	0.52	[0.45, 0.59]	0.74	40	7.6

Recipe vs baseline: +0.050 [0.000, 0.100] over 120 paired tasks. Verdict: unresolved, one training seed per arm. The previous run of the same configuration read -0.065 [-0.117, -0.013]. The sign flipped between two identical one-seed runs because the baseline arm's LoRA init was not seeded; the recipe now seeds it. Treat the weights as one draw, not a result.

Arms in this repo

The root holds the arm the recipe README's headline number reports. Every other arm is a subfolder named after it. checkpoints/ never ships.

folder	arm
.	recipe arm: sliding upper bound, 2026-09-18 run
baseline	baseline arm: fixed upper bound 0.28, 2026-09-18 run

Load

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(base, "while-ai/paper-adaptive-clip-1.5b")  # the headline arm
model = PeftModel.from_pretrained(base, "while-ai/paper-adaptive-clip-1.5b", subfolder="baseline")  # another arm

Reproduce

git clone https://github.com/whilehq/whileai-sdk && cd whileai-sdk/recipes/papers/adaptive-clip
python recipe.py

The recipe README pins the seed, the library versions and the GPU, and its Checks table says what the eval verified. Read the Learned section before quoting a number from this card.